Skip to main content
SpringerLink
Log in

Advanced Similarity Measures Using Word Embeddings and Siamese Networks in CBR

  • Conference paper
  • First Online:
Intelligent Systems and Applications (IntelliSys 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1038))

Included in the following conference series:

Abstract

Automatic fuzzy text processing, context extraction and disambiguation are three challenging research areas with high relevance to complex business domains. Business knowledge can be found in plain text message exchanges, emails, support tickets, internal chat messengers and other volatile means, making the decoding of text-based domain knowledge a challenging task. Traditional natural language processing approaches focus on a comprehensive representation of business knowledge and any relevant mappings. However, such approaches can be highly complex, not cost-effective and of high maintenance, especially in environments that experience frequent changes. This work applies LSTM Siamese Networks to measure text similarities in ambiguous domains. We implement the Manhattan LSTM (MaLSTM) Siamese neural network for semi-automatic knowledge acquisition of business knowledge and decoding of domain-relevant features that enable building similarity measures. Our aim is to minimize the effort from human experts while extracting domain knowledge from rich text, containing context-free abbreviations, grammatically incorrect text and mixed language.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brynjolfsson, E., McElheran, K.: Data in action: data-driven decision making in U.S. manufacturing. Center for Economic Studies (CES), January 2016

    Google Scholar 

  2. Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 1(7), 39–59 (1994)

    Google Scholar 

  3. Bengio, Y.: Deep learning of representations: looking forward. In: Proceedings of the 1st International Conference on Statistical Language and Speech Processing, SLSP 2013, pp. 1–37. Springer, Tarragona (2013)

    Google Scholar 

  4. Bengio, Y., LeCun, Y.: Scaling learning algorithms towards, AI. In: Bottou, L., Chapelle, O., DeCoste, D., Weston, J. (eds.) Large Scale Kernel Machines, vol. 34, pp. 321–360. MIT Press, Cambridge (2007)

    Google Scholar 

  5. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013). https://doi.org/10.1109/TPAMI.2013.50

    Article  Google Scholar 

  6. Ashley, K.: Modeling Legal Argument, Reasoning with Cases and Hypotheticals. MIT-Press, Cambridge (1990)

    Google Scholar 

  7. Brüninghaus, S., Ashley, K.D.: How machine learning can be beneficial for textual case-based reasoning. In: Proceedings of the AAAI 1998/ICML 1998 Workshop on Learning for Text Categorization (AAAI Technical Report WS-98-05), Madison, WI, pp. 71–74 (1998)

    Google Scholar 

  8. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: NIPS 2013 Proceedings of the 26th International Conference on Neural Information Processing Systems, vol. 2 (2013)

    Google Scholar 

  9. Bach, K., Althoff, K.D., Newo, R., Stahl, A.: A case-based reasoning approach for providing machine diagnosis from service reports. In: Ram, A., Wiratunga, N. (eds.) Case-Based Reasoning Research and Development, ICCBR. Lecture Notes in Computer Science, vol. 6880. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  10. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)

    Article  Google Scholar 

  11. Stram, R., Reuss, P., Althoff, K.D.: Weighted one mode projection of a bipartite graph as a local similarity measure. In: Aha, D., Lieber, J. (eds.) Case-Based Reasoning Research and Development, ICCBR. Lecture Notes in Computer Science, vol. 10339. Springer, Cham (2017)

    Chapter  Google Scholar 

  12. Reuss, P., Witzke, C., Althoff, K.D.: Dependency modeling for knowledge maintenance in distributed CBR systems. In: Aha, D., Lieber, J. (eds.) Case-Based Reasoning Research and Development, ICCBR. Lecture Notes in Computer Science, vol. 10339. Springer, Cham (2017)

    Chapter  Google Scholar 

  13. Reuss, P., et al.: FEATURE-TAK - framework for extraction, analysis, and transformation of unstructured textual aircraft knowledge. In: Goel, A., Díaz-Agudo, M., Roth-Berghofer, T. (eds.) Case-Based Reasoning Research and Development, ICCBR. Lecture Notes in Computer Science, vol. 9969. Springer, Cham (2016)

    Chapter  Google Scholar 

  14. Kapetanakis, S., Petridis, M., Knight, B., Ma, J., Bacon, L.: A case based reasoning approach for the monitoring of business workflows. In: Proceedings of the 18th International Conference on Case-Based Reasoning, ICCBR 2010, Alessandria, Italy. Lecture Notes in Artificial Intelligence LNAI, vol. 6176, pp. 390–405 (2010)

    Chapter  Google Scholar 

  15. Kapetanakis, S., Petridis, M.: Evaluating a case-based reasoning architecture for the intelligent monitoring of business workflows. In: Montani, S., Jain, L.C. (eds.) Successful Case-based Reasoning Applications 2. Studies in Computational Intelligence, vol. 494, pp. 43–54. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  16. Sizov, G., Öztürk, P., Štyrák, J.: Acquisition and reuse of reasoning knowledge from textual cases for automated analysis. In: Lamontagne, L., Plaza, E. (eds.) Case-Based Reasoning Research and Development, ICCBR 2014. Lecture Notes in Computer Science, vol. 8765. Springer (2014)

    Google Scholar 

  17. Khoo, C.S.G.: Automatic identification of causal relations in text and their use for improving precision in information retrieval. Ph.D. thesis, The University of Arizona (1995)

    Google Scholar 

  18. Bengio, Y., Ducharme, R., Vincent, P., Janvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)

    MATH  Google Scholar 

  19. Amin, K., Kapetanakis, S., Althoff, KD., Dengel, A., Petridis M.: Dynamic process workflow routing using deep learning. In: AI-2018 Thirty-Eighth SGAI International Conference on Artificial Intelligence, Cambridge (2018)

    Google Scholar 

  20. Amin, K., Lancaster, G., Kapetanakis, S., Althoff, KD., Dengel, A., Petridis, M.: Case-based reasoning in natural language processing: word2vec VS fastText. In: 23rd UK Symposium on Case-Based Reasoning, Cambridge (2018)

    Google Scholar 

  21. Martin, K., Wiratunga, N., Sani, S., Massie, S., Clos, J.: A convolutional Siamese network for developing similarity knowledge in the SelfBACK dataset. In: Sanchez-Ruiz, A.A., Kofod-Petersen, A. (eds.) Proceedings of the 25th International Case-Based Reasoning Conference (ICCBR 2017): Case-Based Reasoning and Deep Learning Workshop (CBRDL 2017), ICCBR, 26–28 June 2017, Trondheim, Norway, pp. 85–94 (2017)

    Google Scholar 

  22. Mueller, J., Thyagarajan, A.: Siamese recurrent architectures for learning sentence similarity. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI 2016) (2016)

    Google Scholar 

  23. Amin, K., Kapetanakis, S., Althoff, K.D., Dengel, A., Petridis, M.: Answering with cases: a CBR approach to deep learning. In: Cox, M., Funk, P., Begum, S. (eds.) Case-Based Reasoning Research and Development, ICCBR. Lecture Notes in Computer Science, vol. 11156. Springer, Cham (2018)

    Chapter  Google Scholar 

  24. Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. DBLP Computer Science Bibliography (2017)

    Google Scholar 

  25. Bromley, J., et al.: Signature verification using a “siamese” time delay neural network. In: NIPS 1993 Proceedings of the 6th International Conference on Neural Information Processing Systems, pp. 737–744 (1993)

    Article  Google Scholar 

  26. Zeiler, M.D.: ADADELTA: an adaptive learning rate method (2012). arXiv:1212.5701

  27. Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: ICML, pp. 1310–1318 (2013)

    Google Scholar 

  28. Goel, A.K., Diaz-Agudo, B.: What’s hot in case-based reasoning. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI 2017) (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Smart Data and Knowledge Services, German Research Center for Artificial Intelligence, Trippstadter Straße 122, 67663, Kaiserslautern, Germany

    Kareem Amin, Klaus-Dieter Althoff & Andreas Dengel

  2. Institute of Computer Science, Intelligent Information Systems Lab, University of Hildesheim, Hildesheim, Germany

    Klaus-Dieter Althoff

  3. Kaiserslautern University, P.O. Box 3049, 67663, Kaiserslautern, Germany

    Kareem Amin & Andreas Dengel

  4. School of Computing Engineering and Mathematics, University of Brighton, Brighton, UK

    Stelios Kapetanakis

  5. Department of Computing, University of Middlesex, London, UK

    Miltos Petridis

  6. School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol, UK

    George Lancaster

Authors
  1. Kareem Amin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George Lancaster
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stelios Kapetanakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Klaus-Dieter Althoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andreas Dengel
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Miltos Petridis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kareem Amin .

Editor information

Editors and Affiliations

  1. School of Computing, Computer Science Research Institute, Ulster University, Newtownabbey, UK

    Yaxin Bi

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Rahul Bhatia

  3. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Amin, K., Lancaster, G., Kapetanakis, S., Althoff, KD., Dengel, A., Petridis, M. (2020). Advanced Similarity Measures Using Word Embeddings and Siamese Networks in CBR. In: Bi, Y., Bhatia, R., Kapoor, S. (eds) Intelligent Systems and Applications. IntelliSys 2019. Advances in Intelligent Systems and Computing, vol 1038. Springer, Cham. https://doi.org/10.1007/978-3-030-29513-4_32

Download citation

Publish with us

Policies and ethics

Search

Navigation