A Prototype of an Intelligent Search Engine Using Machine Learning Based Training for Learning to Rank

  • Piyush RaiEmail author
  • Shrimai Prabhumoye
  • Pranay Khattri
  • Love Rose Singh Sandhu
  • S. Sowmya Kamath
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 27)


Learning to Rank is a concept that focuses on the application of supervised or semi-supervised machine learning techniques to develop a ranking model based on training data. In this paper, we present a learning based search engine that uses supervised machine learning techniques like selection based and review based algorithms to construct a ranking model. Information retrieval techniques are used to retrieve the relevant URLs by crawling the Web in a Breadth-First manner, which are then used as training data for the supervised and review based machine learning techniques to train the crawler. We used the Gradient Descent Algorithm to compare the two techniques and for result analysis.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Croft, W.B., Metzler, D., Strohman, T.: Search Engines -Information Retrieval in Practice. Pearson Education (2009)Google Scholar
  2. 2.
    Radlinski, F., Joachims, T.: Query Chains: Learning to Rank from Implicit Feedback. In: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pp. 239–248 (2005)Google Scholar
  3. 3.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison Wesley (1999)Google Scholar
  4. 4.
    Kemp, C., Ramamohanarao, K.: Long-term learning for web search engines. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) PKDD 2002. LNCS (LNAI), vol. 2431, pp. 263–274. Springer, Heidelberg (2002)Google Scholar
  5. 5.
    Caruana, R., Baluja, S., Mitchell, T.: Using the future to “sort out” the present: Rankprop and multitask learning for medical risk evaluation. In: Advances in Neural Information Processing System, pp. 959–965 (1996)Google Scholar
  6. 6.
  7. 7.
    Joachims, T., Granka, L., Pan, B., Hembrooke, H., Gay, G.: Accurately interpreting clickthrough data asimplicit feedback. In: Annual ACM Conference on Research and Development in Information Retrieval, pp. 154–161 (2005)Google Scholar
  8. 8.
    Tan, Q., Chai, X., Ng, W., Lee, D.-L.: Applying co-training to clickthrough data for search engine adaptation. In: Lee, Y., Li, J., Whang, K.-Y., Lee, D. (eds.) DASFAA 2004. LNCS, vol. 2973, pp. 519–532. Springer, Heidelberg (2004)Google Scholar
  9. 9.
    Carterette, B., Jones, R.: Evaluating search enginesby modeling the relationship between relevance andclicks. In: Advances in Neural Information ProcessingSystems, vol. 20, pp. 217–224 (2008)Google Scholar
  10. 10.
    Craswell, N., Zoeter, O., Taylor, M., Ramsey, B.: Anexperimental comparison of click position-bias models. In: Proceedings of the International Conference on Web Search and Web DataMining, pp. 87–94 (2008)Google Scholar
  11. 11.
    Dupret, G., Piwowarski, B.: User browsing model to predict search engine click data from past observations. In: Proceedings of the 31st Annual International Conference on Research and Development in Information Retrieval (2008)Google Scholar
  12. 12.
    Richardson, M., Dominowska, E., Ragno, R.: Predicting clicks: estimating the click-through rate for new ads. In: Proceedings of the 16th International Conference on World Wide Web, pp. 521–530 (2007)Google Scholar
  13. 13.
    Zhou, D., Bolelli, L., Li, J., Giles, C.L., Zha, H.: Learning user clicks in web search. In: International Joint Conference on Artificial Intelligence (2007)Google Scholar
  14. 14.
    Ponte, J.M., Croft, W.B.: A language modeling approach to information retrieval. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 275–281 (1998)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Piyush Rai
    • 1
    Email author
  • Shrimai Prabhumoye
    • 1
  • Pranay Khattri
    • 1
  • Love Rose Singh Sandhu
    • 1
  • S. Sowmya Kamath
    • 1
  1. 1.Department of Information TechnologyNational Institute of Technology KarnatakaSurathkalIndia

Personalised recommendations