Abstract
This is an investigation of information retrieval performance of Turkish search engines with respect to precision, normalized recall, coverage and novelty ratios. We defined seventeen query topics for Arabul, Arama, Netbul and Superonline. These queries were carefully selected to assess the capability of a search engine for handling broad or narrow topic subjects, exclusion of particular information, identifying and indexing Turkish characters, retrieval of hub/authoritative pages, stemming of Turkish words, correct interpretation of Boolean operators. We classified each document in a retrieval output as being “relevant” or “nonrelevant” to calculate precision and normalized recall ratios at various cut-off points for each pair of query topic and search engine. We found the coverage and novelty ratios for each search engine. We also tested how search engines handle meta-tags and dead links. Arama appears to be the best Turkish search engine in terms of average precision and normalized recall ratios, and the coverage of Turkish sites. Turkish characters (and stemming as well) still cause bottlenecks for Turkish search engines. Superonline and Netbul make use of the indexing information in metatag fields to improve retrieval results.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
M. Kobayashi and K. Takeda. Information retrieval on the web. ACM Computing Surveys, 32(2):144–172, June 2000.
J. Jansen. Using an intelligent agent to enhance search engine performance. First Monday, 1996. http://www..rstmonday.dk/issues/issue2 3/jansen/index.html.
W. Mettrop and P. Nieuwenhuysen. Internet search engines: Fluctuations in document accessibility. Journal of Documentation, 57:623–651, 2001.
W.B. Croft and H. Turtle. A retrieval model for incorporating hypertext links. In Proceedings of ACM Hypertext Conference, pages 213–224, New Orleans, LA, November 1989.
V.N. Gudivada, V.V. Raghavan, W.I. Grosky, and R. Kasanagottu. Information retrieval on the world wide web. IEEE Internet Computing, 1(5):58–68, 1997.
H. Chu and M. Rosenthal. Search engines for the world wide web: A comparative study and evaluation methodology. In Steve Hardin, editor, Proceedings of the 59th ASIS Annual Meeting, pages 127–135, Baltimore, Maryland, October 1996.
H.V. Lerghton and J.V. Srivastava. First 20precision among WWW search services. Journal of the American Society for Information Science, 50:870–881, 1999.
C. Oppenheim, A. Morris, and C. McKnight. The evaluation of WWW search engines. Journal of Documentation, 56:190–211, 2000.
J. Savoy and J. Picard. Retrieval effectiveness on the web. Information Processing and Management, 37:543–569, 2001.
M. Gordon and P. Pathak. Finding information on the WWW: The retrieval effectiveness of search engines. Information Processing and Management, 35:141–180, 1999.
J. S. Deogun, H. Sever, and V. V. Raghavan. Structural abstractions of hypertext documents for web-based retrieval. In Roland R. Wagner, editor, Proceedings of Ninth International Workshop on Database and Expert Systems Applications, (in conjunction with DEXA’98), pages 385–390, Vienna, Austria, August 1998.
M.E. Kücük, B. Olgun, and H. Sever. Application of metadata concepts to discovery of internet resources. In Tatyana Yakhno, editor, Advances in Information Systems (ADVIS’00), volume 1909, pages 304–313. Springer Verlag, Berlin, GR, October 2000.
J.M. Kleinberg. Authoritative source in a hyperlinked environment. In Proceedings of the 9th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 668–677, 1998.
S. C. Clarke and P. Willet. Estimating the recall performance of web search engines. Aslib Proceedings, 49(7):184–189, July/August 1997.
D. Hawking, N. Craswell, P. Thislewaite, and D. Harman. Results and challenges in web search evaluation. In D. Harman, editor, Proceedings of the 8th Text REtrieval Conference (TREC-8), Gaithersburg, Maryland, November 1999.
Y.Y. Yao. Measuring retrieval effectiveness based on user preference of documents. Journal of the American Society for Information Science, 46:133–145, 1995.
W.B. Cooper. Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. American Documentation, 19:30–41, 1968.
S. Lawrence and C.L. Giles. Searching the world wide web. Science, 280(5360):98–100, 3 April 1998. http://www.neci.nec.com/lawrence/science98.html.
R.R. Korfhage. Information Storage and Retireval. Wiley, New York, NY, 1997.
B. Jansen, A. Spink, J. Bateman, and T. Saracevic. Real life information retrieval: A study of user queries on the web. SIGIR Forum, 32(1):5–17, 1998.
C. Silverstein, M. Henziger, H. Marais, and M. Moricz. Analysis of a very large web search engine query log. SIGIR Forum, 33(1):6–12, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bitirim, Y., Tonta, Y., Sever, H. (2002). Information Retrieval Effectiveness of Turkish Search Engines. In: Yakhno, T. (eds) Advances in Information Systems. ADVIS 2002. Lecture Notes in Computer Science, vol 2457. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36077-8_9
Download citation
DOI: https://doi.org/10.1007/3-540-36077-8_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00009-9
Online ISBN: 978-3-540-36077-3
eBook Packages: Springer Book Archive