Storage and Index Design for Community Systems

Abstract

In recent years, we have witnessed a rapid growth in the number of web services. The popularity of these services has driven much activity in digital information publishing and searching. While these systems vary in the services they provide, they share the same operation mode - users publish data items such as URLs, photos and advertisements; in addition, users collaboratively contribute descriptions such as tags and attributes which are used by the system to organize the published data to facilitate searching (by browsing or querying the tags). Such collaborative but unsophisticated way of organizing information with user-created metadata is known as folksonomy, and such systems are sometimes also referred to as collaborative tagging systems. Besides simple tags, richer and flexible data structures could be used to describe a published item to provide more powerful expressiveness to the user. Some systems allow users to define their own attributes, and to describe their published objects with variable number of attributes. The freedom from a strict syntax of the published data items is very convenient to users. However, it is a challenging task to design efficient and scalable mechanisms to organize, classify, and index the data items with variable schemas and topics to facilitate searching.