The Real Problem of Bridging the “Semantic Gap”

Abstract

The “semantic gap” is a well-know problem in multimedia. The challenge is to accurately classify and effectively search multimedia content from automatically extracted low-level audio-visual features. While much effort is focused on developing the best machine learning approaches, not enough attention is placed on the required semantic coverage and the real utility of the classifiers in multimedia systems. Exacerbating the problem is the tremendous dearth of reliable training data across the diversity of semantics required for effective multimedia search in practice. Until we address the problems of what classifiers are needed in the large scale and how they can be sufficiently trained, research on bridging the “semantic gap” will not produce any real impact.