Fusion vs. Two-Stage for Multimodal Retrieval

Arampatzis, Avi ; Zagoris, Konstantinos ; Chatzichristofis, Savvas A. (2011)

We compare two methods for retrieval from multimodal collections. The first is a score-based fusion of results, retrieved visually and textually. The second is a two-stage method that visually re-ranks the top-K results textually retrieved. We discuss their underlying hypotheses and practical limitations, and contact a comparative evaluation on a standardized snapshot of Wikipedia. Both methods are found to be significantly more effective than single-modality baselines, with no clear winner but with different robustness features. Nevertheless, two-stage retrieval provides efficiency benefits over fusion.

© Springer-Verlag Berlin Heidelberg 2011
Except where otherwise noted, this item's license is described as © Springer-Verlag Berlin Heidelberg 2011