Abstract
Introduced is a new approach to Information Retrieval developed on the basis of Hidden Markov Models (HMMs). HMMs are shown to provide a mathematically sound framework for retrieving documenta—documents with predefined boundaries and also entities of information that are of arbitrary lengths and formats (passage retrieval). Our retrieval model is shown to encompass promising capabilities: First, the position of occurrences of indexing features can be used for indexing. Positional information is essential, for instance, when considering phrases, negation, and the proximity of features. Second, from training collections we can derive automatically optimal weights for arbitrary features. Third, a query dependent structure can be determined for every document by segmenting the documents into passages that are either relevant or irrelevant to the query. The theoretical analysis of our retrieval model is complemented by the results of preliminary experiments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
J. S. Bridle. Stochastic Models and Template Matching: some Important Relationships Between Two Apparently Different Techniques for Automatic Speech Recognition. In Institute of Acoustics , Autumn Conference, pages 452a-452h, 1984.
N. Fuhr. Probabilistic Models in Information Retrieval. The Computer Journal, 35(3):243–255, 1992.
U. Hahn. Topic Parsing: Accounting for Macro Structures in Full-Text Analysis. Information Processing N Management, 26(1):135–170, 1990.
Y. He, N.Y. Chen, and A. Kundu. Handwritten Word Recognition Using HMM with Adaptive Length Viterbi Algorithm. In International Conference on Acoustics, Speech, and Signal Processing, pages III-153-III-156, 1992.
M. A. Hearst and C. Plaunt. Subtopic Structuring for Full-Length Access. In ACM SIGIR Conference on R&D in Information Retrieval, pages 59–68, 1993.
X. D. Huang, Y. Ariki, and M. A. Jack. Hidden Markov Models for Speech Recognition. Edinburgh University Press, Edinburgh, 1990.
J. J. Hull. A Hidden Markov Model for Language Syntax in Text Recognition. In 11th IAPR International Conference on Pattern Recognition. Vol.I1,pages 124–127, 1992.
R. Kuhlen and M. Hess. Passagen-Retrieval-auch eine Möglichkeit der automatischen Verknüpfung in Hypertexten. In Information Retrieval’93, pages 100–115, Konstanz, Deutschland, October, 1993. Hochschulverband für Informationswissenschaft (HI).
J. Kupiec. Hidden Markov Estimation for Unrestricted Stochastic Context-Free Grammars. In International Conference on Acoustics, Speech, and Signal Processing, pages I-177–180, 1992.
J. Kupiec. Robust Part-of-Speech Tagging Using a Hidden Markov Model. Computer Speech and Language, 6:225–242,1992.
L. W. Levinson, L. R. Rabiner, and M. M. Sondhi. An Introduction to the Application of the Theory of Probabilistic Functions of Markov Process to Automatic Speech Recognition. The Bell System Technical Journal, 62(4):1035–1075, 1983.
Y. Linde, A. Buzo, and R. M. Gray. An Algorithm for Vector Quantizer Design. IEEE Transactions on Communication, 28(1):84–95, 1980.
A. Moffat, R. Sacks-Davis, R. Wilkinson, and J. Zobel. Retrieval of Partial Documents. In TREC-2 Proceedings, 1993.
J. O’Connor. Retrieval of Answer-Sentences and Answer-Figures from Papers by Text Searching. Information Processing &Management, 11(5/7):155–164, 1975.
J. O’Connor. Answer Passage Retrieval by Text Searching. Journal of the ASIS, 31(4):227–239, 1980.
M. F. Porter. An Algorithm for Suffix Stripping. Program,14(3):130–137, 1980.
S. E. Robertson. The Probability Ranking Principle in IR. Journal of Documentation, 33(4):294–304, 1977.
R. C. Rose and D. B. Paul. A Hidden Markov Model Based Keyword Recognition System. In International Conference on Acoustics, Speech, and Signal Processing, pages 129–132, 1990.
G. Salton, J. Allan, and C. Buckley. Approaches to Passage Retrieval in Full Text Information Systems. In ACM SIGIR Conference on R&D in Information Retrieval, pages 49–58, 1993.
G. Salton and C. Buckley. Term Weighting Approaches in Automatic Text Retrieval. Information Processing FJ Management, 24(5):513–523, 1988.
P. Schäuble and U. Glavitsch. A Probabilistic Hypermedia Retrieval Model Based on Hidden Markov Models. Workshop Intelligent Access to Information Systems, Darmstadt, Nov. 1990.
C. Stanfill and D. Waltz. Statistical Methods, Artificial Intelligence, and Information Retrieval. In P. S. Jacobs, editor, Text-Based Intelligent Systems: Current Research and Practice in Information Extraction and Retrieval, pages 215–226. Lawrence Erlbaum Associates, 1992.
C. J. van Rijsbergen. Information Retrieval. Butterworths, London, second edition, 1979.
L. D. Wilcox and M. A. Bush. HMM-Based Wordspotting for Voice Editing and Indexing. In European Conference on Speech Communication and Technology (EUROSPEECH), pages 25–28, 1991.
P. Willet and A. M. Robertson. Searching for Historical Word-Forms in a Database of 17th-Century English Text Using Spelling-Correction Methods. In N. Belkin, P. Ingwersen, and A. M. Pejtersen, editors, ACM SIGIR Conference on R&D in Information Retrieval, pages 256–265, 1992.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer-Verlag London Limited
About this paper
Cite this paper
Mittendorf, E., Schäuble, P. (1994). Document and Passage Retrieval Based on Hidden Markov Models. In: Croft, B.W., van Rijsbergen, C.J. (eds) SIGIR ’94. Springer, London. https://doi.org/10.1007/978-1-4471-2099-5_33
Download citation
DOI: https://doi.org/10.1007/978-1-4471-2099-5_33
Publisher Name: Springer, London
Print ISBN: 978-3-540-19889-5
Online ISBN: 978-1-4471-2099-5
eBook Packages: Springer Book Archive