Abstract
Enterprise Search is a continuously evolving and important field, which is seeing a resurgence driven by artificial intelligence. Still, there is no objective, generally accepted way to compare various enterprise search systems. SQuAD is becoming popular for measuring algorithmic reading comprehension (MRC) but is ineffective for quantifying effectiveness of enterprise search in business-use situations. In this paper we modify the SQuAD scoring methodology to propose a scoring system for enterprise search systems that aligns with the real world expectations of users. Further, we use a search system based on Calibrated Quantum Mesh (CQM) to underscore the relevance of this metric.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hawking, D.: Challenges in enterprise search. In: Proceedings of the 15th Australasian Database Conference (ADC 2004), vol. 27, pp. 15–24. Australian Computer Society, Inc., Darlinghurst (2004)
Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web (1999)
Guha, R., McCool, R., Miller, E.: Semantic search. In: Proceedings of the 12th International Conference on World Wide Web (WWW 2003), pp. 700–709. ACM, New York (2003)
Voigt, C.A., Gordon, D.B., Mayo, S.L.: Trading accuracy for speed: a quantitative comparison of search algorithms in protein sequence design. J. Mol. Biol. 299(3), 789–803 (2000). Edited by J Thornton
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. CoRR, abs/1606.05250 (2016)
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD. CoRR, abs/1806.03822 (2018)
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad Rajpurkar - Google Scholar (2019). Accessed 21 May 2019
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: The Stanford Question Answering Dataset (2019). Accessed 21 May 2019
Kulkarni, R., Kulkarni, H., Balar, K., Krishna, P.: Cognitive natural language search using calibrated quantum mesh. In: 2018 IEEE 17th International Conference on Cognitive Informatics Cognitive Computing (ICCI*CC), pp. 174–178, July 2018
Viswanath, S., Yates, M., Burt, J., Yazell, J., Kuhr, R., Strum, B., Krishna, P., Balar, K., Kulkarni, R., Kulkarni, H., Fennell, J.: An intelligent machine for document preparation. In: AICHE Annual Meeting, October 2018
Han, K.H., Park, J.W.: Process-centered knowledge model and enterprise ontology for the development of knowledge management system. Expert Syst. Appl. 36(4), 7441–7447 (2009)
Redfern, D.M.: Natural language meta-search system and method. VI 20 2000. US Patent 6,078,914
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kulkarni, H., Gupta, H., Balar, K., Krishna, P. (2020). New Metric Based on SQuAD for Evaluating Accuracy of Enterprise Search Algorithms. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Advances in Information and Communication. FICC 2020. Advances in Intelligent Systems and Computing, vol 1130. Springer, Cham. https://doi.org/10.1007/978-3-030-39442-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-39442-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39441-7
Online ISBN: 978-3-030-39442-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)