What’s New? What’s Certain? – Scoring Search Results in the Presence of Overlapping Data Sources
Data integration projects in the life sciences often gather data on a particular subject from multiple sources. Some of these sources overlap to a certain degree. Therefore, integrated search results may be supported by one, few, or all data sources. To reflect these differences, results should be ranked according to the number of data sources that support them. How such a ranking should look like is not clear per se. Either, results supported by only few sources are ranked high because this information is potentially new, or such results are ranked low because the strength of evidence supporting them is limited.
We present two scoring schemes to rank search results in the integrated protein annotation database Columba. We define a surprisingness score, preferring results supported by few sources, and a confidence score, preferring frequently encountered information. Unlike many other scoring schemes our proposal is purely data-driven and does not require users to specify preferences among sources. Both scores take the concrete overlaps of data sources into account and do not presume statistical independence. We show how our schemes have been implemented efficiently using SQL.
Unable to display preview. Download preview PDF.
- 2.Bleiholder, J., Khuller, S., Naumann, F., Raschid, L., Wu, Y.: Query Planning in the Presence of Overlapping Sources. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., Boehm, K., Kemper, A., Grust, T., Boehm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 811–828. Springer, Heidelberg (2006)CrossRefGoogle Scholar
- 4.Florescu, D., Koller, D., Levy, A.: Using Probabilistic Information in Data Integration.. In: Proceedings of the VLDB, pp. 216–225. Morgan Kaufmann, San Francisco (1997)Google Scholar
- 8.Lacroix, Z., Murthy, H., Naumann, F., Raschid, L.: Links and Paths through Life Sciences Data Sources. In: DILS 2004. LNCS (LNBI), vol. 2994, pp. 203–211 Springer, Heidelberg (2004)Google Scholar
- 13.Naumann, F., Leser, U., Freytag, J.-C.: Quality-driven Integration of Heterogenous Information Systems. In: Proceedings of the VLDB, pp. 447–458. Morgan Kaufmann, San Francisco (1999)Google Scholar
- 14.Rother, K., Müller, H., Trissl, S., Koch, I., et al.: Columba: Multidimensional Data Integration of Protein Annotations. In: DILS 2004. LNCS (LNBI), vol. 2994, pp. 156–171. Springer, Heidelberg (2004)Google Scholar
- 17.Yanai, I., DeLisi, C.: The society of genes: networks of functional links between genes from comparative genomics. Genome Biology 3(11) : (research0064) (October 2002)Google Scholar