Advertisement

Knowledge Discovery via Machine Learning for Neurodegenerative Disease Researchers

  • I. Burak Özyurt
  • Gregory G. Brown
Protocol
  • 959 Downloads
Part of the Methods in Molecular Biology™ book series (MIMB, volume 569)

Summary

Ever-increasing size of the biomedical literature makes more precise information retrieval and tapping into implicit knowledge in scientific literature a necessity. In this chapter, first, three new variants of the expectation–maximization (EM) method for semisupervised document classification (Machine Learning 39:103–134, 2000) are introduced to refine biomedical literature meta-searches. The retrieval performance of a multi-mixture per class EM variant with Agglomerative Information Bottleneck clustering (Slonim and Tishby (1999) Agglomerative information bottleneck. In Proceedings of NIPS-12) using Davies–Bouldin cluster validity index (IEEE Transactions on Pattern Analysis and Machine Intelligence 1:224–227, 1979), rivaled the state-of-the-art transductive support vector machines (TSVM) (Joachims (1999) Transductive inference for text classification using support vector machines. In Proceedings of the International Conference on Machine Learning (ICML)). Moreover, the multi-mixture per class EM variant refined search results more quickly with more than one order of magnitude improvement in execution time compared with TSVM. A second tool, CRFNER, uses conditional random fields (Lafferty et al. (2001) Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML-2001) to recognize 15 types of named entities from schizophrenia abstracts outperforming ABNER (Settles (2004) Biomedical named entity recognition using conditional random fields and rich feature sets. In Proceedings of COLING 2004 International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA)) in biological named entity recognition and reaching F1 performance of 82.5% on the second set of named entities.

Key words

Relevance ranking Information extraction EM Cluster validity Conditional random fields Machine learning Knowledge discovery Schizophrenia 

Notes

Acknowledgment

This research was supported by 1 U24 RR021992 to the Function Biomedical Informatics Research Network (BIRN, http://www.nbirn.net) that is funded by the National Center for Research Resources (NCRR) at the National Institutes of Health (NIH). Special thanks to Sinem Özyurt MS Chem, MS ChE for annotating biological named-entities.

References

  1. 1.
    Nigam, K., McCallum, A. K., Thrun, S., and Mitchell, T. (2000) Text classification from labeled and unlabeled documents using EM. Machine Learning, 39, 103–134.CrossRefGoogle Scholar
  2. 2.
    Slonim, N., and Tishby, N. (1999) Agglomerative information bottleneck. In Proceedings of NIPS-12.Google Scholar
  3. 3.
    Davies, D. L., and Bouldin, D. W. (1979) A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 224–227.PubMedCrossRefGoogle Scholar
  4. 4.
    Joachims, T. (1999) Transductive inference for text classification using support vector machines. In Proceedings of the International Conference on Machine Learning (ICML).Google Scholar
  5. 5.
    Lafferty, J., McCallum, A., and Pereira, F. (2001) Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML-2001, 282–289.Google Scholar
  6. 6.
    Settles, B. (2004) Biomedical named entity recognition using conditional random fields and rich feature sets. In Proceedings of COLING 2004 International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA).Google Scholar
  7. 7.
    Brown, G. G., Pieper, S., Martone, M., Aucoin, N., Joyner, A., Bischoff-Grethe, A., and Torvik, V. (2004) The query atlas: A brain referenced knowledge discovery tool. In Annual Neuroscience mMeeting.Google Scholar
  8. 8.
    Baeza-Yates, R., and Ribeiro-Neto, B. (1999) Modern Information Retrieval. ACM Press, New York.Google Scholar
  9. 9.
    van Rijsbergen, C. J. (1979) Information Retrieval. Butterworths, London.Google Scholar
  10. 10.
    Croft, W. B., Cronen-Townsend, S, and Lavrenko, V. (2001) Relevance feedback and personalization: A language modeling perspective. In DELOS Workshop: Personalisation and Recommender Systems in Digital Libraries.Google Scholar
  11. 11.
    van Rijsbergen, C. (1977) A theoretical basis for the use of co-occurrence data in information retrieval. Journal of Documentation, 33(2), 106–119.CrossRefGoogle Scholar
  12. 12.
    Friedman, J. H. (1997) On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery, 1 (1), 55–77.CrossRefGoogle Scholar
  13. 13.
    Duda, R. O., Hart, P. E., and Stork, D. G. (2001) Pattern Classification. Second edition. Wiley, New York.Google Scholar
  14. 14.
    Dunn, J. C. (1974) Well separated clusters and optimal fuzzy partitions. Journal of. Cybernetics, 4, 95–104.CrossRefGoogle Scholar
  15. 15.
    Maulik, U., and Bandyopadhyay, S. (2002) Performance evaluation of some clustering algorithms and validity indices. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(12), 1650–1654.CrossRefGoogle Scholar
  16. 16.
    Gelatt, C., Kirkpatrick S., and Vecchi, M. (1983) Optimization by simulated annealing. Science, 220, 671–680.PubMedCrossRefGoogle Scholar
  17. 17.
    Ozyurt, I. B., and Brown, G. G. (2007) Search result refinement via machine learning from labeled-unlabeled data for meta-search. In IEEE Symposium on Computational Intelligence and Data Mining CIDM 2007, 186–193.Google Scholar
  18. 18.
    Levine, E., and Domany, E. (2001) Resampling method for unsupervised estimation of cluster validity. Neural Computation, 13, 2573–2593.PubMedCrossRefGoogle Scholar
  19. 19.
    Geman, S., and Geman, D. (1984) Stochastic relaxation, Gibbs distribution, and Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.PubMedCrossRefGoogle Scholar
  20. 20.
    Peng, F., and McCallum, A. (2004) Accurate information extraction from research papers using conditional random fields. In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics (HLT/NAACL-04).Google Scholar
  21. 21.
    Kindermann, R., and Snell, J. L. (1980) Markov Random Fields and Their Applications. American Mathematical Society, Providence.Google Scholar
  22. 22.
    Hammersley, J., and Clifford, P. (1971) Markov fields on finite graphs and lattices. Unpublished manuscript.Google Scholar
  23. 23.
    Sutton, C., and McCallum, A. (2006) An introduction to conditional random fields for relational learning. In Introduction to Statistical Relational Learning (Getoor, L., and Taskar, B., eds.). MIT Press, Cambridge.Google Scholar
  24. 24.
    Byrd, R. H., Nocedal, J., and Schnabel, R. B. (1994) Representations of quasi-Newton matrices and their use in limited memory methods. Mathematical. Programming, 63(2), 129–156.CrossRefGoogle Scholar
  25. 25.
    Charniak, E. (2000) A maximum-entropy-inspired parser. In Proceedings of NAACL, 132–139.Google Scholar
  26. 26.
    Ngai, G., and Florian, R. (2001) Transformation-based learning in the fast lane. In Proceedings of North American ACL 2001, 40–47.Google Scholar
  27. 27.
    Collins, M. (2002) Ranking algorithms for named-entity extraction: Boosting and the voted perceptron. In Proceedings of Association for Computational Linguistics Conference, 489–496.Google Scholar
  28. 28.
    Fellbaum, C. (ed.) (1998) WordNet: An Electronic Lexical Database. MIT Press, Cambridge.Google Scholar
  29. 29.
    McCallum, A. K. (2002) A machine learning for language toolkit. http://mallet.cs.umass.edu.
  30. 30.
    Sebastiani, F. (2002) Machine learning in automated text categorization. ACM Computing. Surveys, 34(1), 1–47.CrossRefGoogle Scholar
  31. 31.
    Joachims, T. (2002) Optimizing search engines using clickthrough data. Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 133–142.Google Scholar

Copyright information

© Humana Press, a part of Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • I. Burak Özyurt
    • 1
  • Gregory G. Brown
    • 1
  1. 1.Department of PsychiatryUniversity of California – San DiegoLa JollaUSA

Personalised recommendations