LSTM-Based Neural Network Model for Semantic Search

  • Xiaoyu Guo
  • Jing MaEmail author
  • Xiaofeng Li
Conference paper
Part of the Springer Proceedings in Business and Economics book series (SPBE)


To improve web search quality and serve a better search experience for users, it is important to capture semantic information from user query which contains user’s intention in web search. Long Short-Term Memory (LSTM), a significant network in deep learning has made tremendous achievements in capturing semantic information and predicting the semantic relatedness of two sentences. In this study, considering the similarity between predicting the relatedness of sentence pair task and semantic search, we provide a novel channel to process semantic search task: see semantic search as an atypical predicting the relatedness of sentence pair task. Furthermore, we propose an LSTM-Based Neural Network Model which is suitable for predicting the semantic relatedness between user query and potential documents. The proposed LSTM-Based Neural Network Model is trained by Home Depot dataset. Results show that our model outperforms than other models.


LSTM Deep learning Semantic search RNN 


  1. 1.
    O. Vinyals, A. Toshev, S. Bengio et al., Show and Tell: A Neural Image Caption Generator [J] (2014)Google Scholar
  2. 2.
    I. Sutskever, O. Vinyals, Q.V. Le, Sequence to Sequence Learning with Neural Networks [J] (2014)Google Scholar
  3. 3.
    P.S. Huang, X. He, J. Gao et al., Learning deep structured semantic models for web search using clickthrough data [C], in Proceedings of the 22nd ACM International Conference on Conference on Information and Knowledge Management (ACM, 2013)Google Scholar
  4. 4.
    H. Nassif, M. Mohtarami, J. Glass, Learning semantic relatedness in community question answering using neural models [C], in Proceedings of the 1st Workshop on Representation Learning for NLP (2016), pp. 137–147Google Scholar
  5. 5.
    T. Mikolov, Statistical Language Models Based on Neural Networks [J] (Presentation at Google, Mountain View, 2012), p. 80Google Scholar
  6. 6.
    R. Pascanu, T Mikolov, Y. Bengio, On the difficulty of training recurrent neural networks [C], in International Conference on Machine Learning (2013), pp. 1310–1318Google Scholar
  7. 7.
    S. Hochreiter, J. Schmidhuber, Long short-term memory[J]. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  8. 8.
    H. Palangi, L. Deng, Y. Shen et al., Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval [J]. IEEE/ACM Trans. Audio Speech Lang. Process (TASLP) 24(4), 694–707 (2016)CrossRefGoogle Scholar
  9. 9.
    A. Sordoni, Y. Bengio, H. Vahabi et al., A hierarchical recurrent encoder-decoder for generative context-aware query suggestion[C], in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (ACM, 2015), pp. 553–562Google Scholar
  10. 10.
    K.S. Tai, R. Socher, C.D. Manning, Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks [J] (2015). arXiv:1503.00075, 2015
  11. 11.
    B. Krause, L. Lu, I. Murray et al., Multiplicative LSTM for Sequence Modelling [J] (2016) arXiv:1609.07959
  12. 12.
    H.S. Joshi, Finding Similar Questions in Large-scale Community QA Forums [D]. Massachusetts Institute of Technology (2016)Google Scholar
  13. 13.
  14. 14.
    J. Pennington, R. Socher, C. Manning, Glove: global vectors for word representation [C], in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014), pp. 1532–1543Google Scholar
  15. 15.
    J. Duchi, E. Hazan, Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization[J]. J. Mach. Learn. Res. 2121–2159 (2011)Google Scholar
  16. 16.
    A.E. Hoerl, R.W. Kennard, Ridge regression: biased estimation for nonorthogonal problems [J]. Technometrics 12(1), 55–67 (1970)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.College of Economics and ManagementNanjing University of Aeronautics and AstronauticsNanjing, Jiangning DistrictChina

Personalised recommendations