Automatically Constructing a Directory of Molecular Biology Databases

  • Luciano Barbosa
  • Sumit Tandon
  • Juliana Freire
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4544)


There has been an explosion in the volume of biology-related information that is available in online databases. But finding the right information can be challenging. Not only is this information spread over multiple sources, but often, it is hidden behind form interfaces of online databases. There are several ongoing efforts that aim to simplify the process of finding, integrating and exploring these data. However, existing approaches are not scalable, and require substantial manual input. Notable examples include the NCBI databases and the NAR database compilation. As an important step towards a scalable solution to this problem, we describe a new infrastructure that automates, to a large extent, the process of locating and organizing online databases. We show how this infrastructure can be used to automate the construction and maintenance of a Molecular Biology database collection. We also provide an evaluation which shows that the infrastructure is scalable and effective—it is able to efficiently locate and accurately identify the relevant online databases.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barbosa, L., Freire, J.: Siphoning Hidden-Web Data through Keyword-Based Interfaces. In: Proceedings of SBBD, pp. 309–321 (2004)Google Scholar
  2. 2.
    Barbosa, L., Freire, J.: Searching for Hidden-Web Databases. In: Proceedings of WebDB, pp. 1–6 (2005)Google Scholar
  3. 3.
    Barbosa, L., Freire, J.: An Adaptive Crawler for Locating Hidden-Web Entry Points. In: Proceedings of WWW (To appear 2007)Google Scholar
  4. 4.
    Barbosa, L., Freire, J.: Combining Classifiers to Organize Online Databases. In: Proceedings of WWW (To appear 2007)Google Scholar
  5. 5.
    Bharat, K., Broder, A., Henzinger, M., Kumar, P., Venkatasubramanian, S.: The connectivity server: Fast access to linkage information on the Web. Computer Networks 30(1-7), 469–477 (1998)Google Scholar
  6. 6.
    Chakrabarti, S., van den Berg, M., Dom, B.: Focused Crawling: A New Approach to Topic-Specific Web Resource Discovery. Computer Networks 31(11-16), 1623–1640 (1999)CrossRefGoogle Scholar
  7. 7.
    Chang, K.C.-C., He, B., Zhang, Z.: Toward Large-Scale Integration: Building a MetaQuerier over Databases on the Web. In: Proceedings of CIDR, pp. 44–55 (2005)Google Scholar
  8. 8.
    Diligenti, M., Coetzee, F., Lawrence, S., Giles, C.L., Gori, M.: Focused Crawling Using Context Graphs. In: Proceedings of VLDB, pp. 527–534 (2000)Google Scholar
  9. 9.
    Galperin, M.: The molecular biology database collection: 2007 update. Nucleic Acids Res, 35 (2007)Google Scholar
  10. 10.
    Hsieh, W., Madhavan, J., Pike, R.: Data management projects at Google. In: Proceedings of ACM SIGMOD, pp. 725–726. ACM Press, New York (2006)Google Scholar
  11. 11.
    Knox, C., Shrivastava, S., Stothard, P., Eisner, R., Wishart, D.S.: BioSpider: A Web Server for Automating Metabolome Annotations. In: Pacific Symposium on Biocomputing, pp. 145–156 (2007)Google Scholar
  12. 12.
  13. 13.
    Menczer, F., Pant, G., Ruiz, M., Srinivasan, P.: Evaluating topic-driven Web crawlers. In: Proceedings of SIGIR, pp. 241–249 (2001)Google Scholar
  14. 14.
    Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)zbMATHGoogle Scholar
  15. 15.
    Ngu, A.H., Rocco, D., Critchlow, T., Buttler, D.: Automatic discovery and inferencing of complex bioinformatics web interfaces. World Wide Web 8(3), 463–493 (2005)CrossRefGoogle Scholar
  16. 16.
    Raghavan, S., Garcia.-Molina, H.: Crawling the Hidden Web. In: Proceedings of VLDB, pp. 129–138 (2001)Google Scholar
  17. 17.
    Srinivasan, P., Mitchell, J., Bodenreider, O., Pant, G., Menczer, F.: Web Crawling agents for Retrieving Biomedical Information. In: Workshop on Agents in Bioinformatics (NETTAB-02) (2002)Google Scholar
  18. 18.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  19. 19.
    Wu, W., Yu, C., Doan, A., Meng, W.: An Interactive Clustering-based Approach to Integrating Source Query interfaces on the Deep Web. In: Proceedings of ACM SIGMOD, pp. 95–106. ACM Press, New York (2004)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Luciano Barbosa
    • 1
  • Sumit Tandon
    • 1
  • Juliana Freire
    • 1
  1. 1.School of Computing, University of Utah 

Personalised recommendations