Advertisement

Automatic ontology construction from text: a review from shallow to deep learning trend

  • Fatima N. Al-AswadiEmail author
  • Huah Yong Chan
  • Keng Hoon Gan
Article
  • 58 Downloads

Abstract

The explosive growth of textual data on the web coupled with the increase on demand for ontologies to promote the semantic web, have made the automatic ontology construction from the text a very promising research area. Ontology learning (OL) from text is a process that aims to (semi-) automatically extract and represent the knowledge from text in machine-readable form. Ontology is considered one of the main cornerstones of representing the knowledge in a more meaningful way on the semantic web. Usage of ontologies has proven to be beneficial and efficient in different applications (e.g. information retrieval, information extraction, and question answering). Nevertheless, manually construction of ontologies is time-consuming as well extremely laborious and costly process. In recent years, many approaches and systems that try to automate the construction of ontologies have been developed. This paper reviews various approaches, systems, and challenges of automatic ontology construction from the text. In addition, it also discusses ways the ontology construction process could be enhanced in the future by presenting techniques from shallow learning to deep learning (DL).

Keywords

Concept classification Deep learning Ontology construction Ontology learning Semantic relation 

Notes

References

  1. Abney S (1997) Part-of-speech tagging and partial parsing. In: Young S, Bloothooft G (eds) Corpus-based methods in language and speech processing, vol. 2. Springer, Netherlands, pp 118–136CrossRefGoogle Scholar
  2. Albukhitan S, Helmy T, Alnazer A (2017) Arabic ontology learning using deep learning. Paper presented at the Proceedings of the international conference on web intelligence, Leipzig, GermanyGoogle Scholar
  3. Arel I, Rose DC, Karnowski TP (2010) Deep machine learning—a new frontier in artificial intelligence research [research frontier]. IEEE Comput Intell Mag 5:13–18.  https://doi.org/10.1109/mci.2010.938364 CrossRefGoogle Scholar
  4. Arguello Casteleiro M, Maseda Fernandez D, Demetriou G, Warren R, Fernandez-Prieto MJ, Des Diz J, Nenadic G, Keane J, Robert S (2017) A case study on sepsis using PubMed and deep learning for ontology learning. In: Informatics for health: connected citizen-led wellness and population health, vol 235. pp 516–520.  https://doi.org/10.3233/978-1-61499-753-5-516
  5. Baldi P (2012) Autoencoders, unsupervised learning, and deep architectures. Paper presented at the proceedings of ICML workshop on unsupervised and transfer learning, Bellevue, Washington, USAGoogle Scholar
  6. Basegmez E (2014) The next generation neural networks: deep learning and spiking neural networks. In: Advanced seminar in technical university of Munich, Munchen, 2014, pp 1–40Google Scholar
  7. Bengio Y, LeCun Y (2007) Scaling learning algorithms towards AI. In: Large-scale kernel machines, p 34Google Scholar
  8. Boytcheva S (2002) Overview of inductive logic programming (ILP) systems. Cybern Inf Technol 1:27–36Google Scholar
  9. Budanitsky A (1999) Lexical semantic relatedness and its application in natural language processing. University of TorontoGoogle Scholar
  10. Buitelaar P, Cimiano P, Magnini B (2005) Ontology learning from text: an overview. In: Ontology learning from text: methods, evaluation and applications, 123:3–12Google Scholar
  11. Chen X-W, Lin X (2014) Big data deep learning: challenges and perspectives. IEEE Access 2:514–525CrossRefGoogle Scholar
  12. Chen Y, Li W, Liu Y, Zheng D, Zhao T (2010) Exploring deep belief network for chinese relation extraction. In: Proceedings of the joint conference on Chinese language processing (CLP’10), pp 28–29Google Scholar
  13. Chicco D, Sadowski P, Baldi P (2014) Deep autoencoder neural networks for gene ontology annotation predictions. Paper presented at the Proceedings of the 5th ACM conference on bioinformatics, computational biology, and health informatics, Newport Beach, CaliforniaGoogle Scholar
  14. Cimiano P, Völker J (2005) Text2Onto. In: Montoyo A, Muńoz R, Métais E (eds) Natural language processing and information systems. Proceedings of 10th international conference on applications of natural language to information systems, NLDB 2005, Alicante, Spain, June 15–17, 2005. Springer, Berlin, pp 227–238.  https://doi.org/10.1007/11428817_21 CrossRefGoogle Scholar
  15. Cimiano P, Hotho A, Staab S (2005) Learning concept hierarchies from text corpora using formal concept analysis. J Artif Intell Res 24:305–339CrossRefGoogle Scholar
  16. Cohen WW (2005) Stacked sequential learning. Carnegie-Mellon Univ Pittsburgh PA School of Computer ScienceGoogle Scholar
  17. Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th international conference on machine learning. ACM, pp 160–167Google Scholar
  18. Deng L (2012) Three classes of deep learning architectures and their applications: a tutorial survey APSIPA transactions on signal and information processingGoogle Scholar
  19. Deng L (2014) A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Transactions on signal and information processing 3:1–29.  https://doi.org/10.1017/atsip.2013.9
  20. Deng L, Yu D (2011) Deep convex net: A scalable architecture for speech pattern classification. In: Annual conference of the international speech communication association, 2011. Interspeech, pp 2285–2288Google Scholar
  21. Deng L, Yu D (2014) Deep learning. Signal Process 7:3–4Google Scholar
  22. Drymonas E, Zervanou K, Petrakis EGM (2010) Unsupervised ontology acquisition from plain texts: the OntoGain system. In: Natural language processing and information systems. Springer, Berlin, pp 277–287CrossRefGoogle Scholar
  23. El-Kilany A, Tazi NE, Ezzat E (2017) Building relation extraction templates via unsupervised learning. Paper presented at the Proceedings of the 21st international database engineering and applications symposium, Bristol, United KingdomGoogle Scholar
  24. Etzioni O, Banko M, Soderland S, Weld DS (2008) Open information extraction from the web. Commun ACM 51:68–74.  https://doi.org/10.1145/1409360.1409378 CrossRefGoogle Scholar
  25. Faure D, Nédellec C (1998) Asium: Learning subcategorization frames and restrictions of selection. In: Kodratoff Y (ed) Text mining workshop, 10th European conference on machine learning (ECML 98), Chemnitz, Germany.Google Scholar
  26. Faure D, Poibeau T (2000a) First experiments of using semantic knowledge learned by ASIUM for information extraction task using INTEX. In: Ontology learning ECAI-2000 workshop, pp 7–12Google Scholar
  27. Faure D, Poibeau T (2000b) First experiments of using semantic knowledge learned by ASIUM for information extraction task using INTEX. In: Proceedings of the ECAI workshop on ontology learningGoogle Scholar
  28. Fischer A, Igel C (2012) An introduction to restricted Boltzmann machines. In: Iberoamerican congress on pattern recognition. Springer, pp 14–36Google Scholar
  29. Fleischhacker D, Völker J (2011) Inductive learning of disjointness axioms. In: On the move to meaningful internet systems: OTM 2011. Springer, Berlin, pp 680–697CrossRefGoogle Scholar
  30. Gamallo P, Gonzalez M, Agustini A, Lopes G, De Lima VS (2002) Mapping syntactic dependencies onto semantic relations. In: Proceedings of the ECAI workshop on machine learning and natural language processing for ontology engineering, pp 15–22Google Scholar
  31. Gillani Andleeb S (2015) From text mining to knowledge mining: An integrated framework of concept extraction and categorization for domain ontology. Doctoral dissertation, Budapesti Corvinus EgyetemGoogle Scholar
  32. Gómez-Pérez A, Manzano-Macho D (2003) A survey of ontology learning methods and techniques. Deliverable 1.5, Onto Web consortiumGoogle Scholar
  33. Grefenstette E, Blunsom P, de Freitas N, Hermann KM (2014) A deep architecture for semantic parsing. arXiv preprint arXiv:14047296
  34. Hahn U, Marko KG (2002) Ontology and lexicon evolution by text understanding. In: Proceedings of the ECAI 2002 workshop on machine learning and natural language processing for ontology engineering (OLT 2002), Lyon, FranceGoogle Scholar
  35. Hahn U, Romacker M (2001) The SYNDIKATE text knowledge base generator. Paper presented at the Proceedings of the first international conference on Human language technology research, San DiegoGoogle Scholar
  36. Hassan A, Mahmood A (2018) Convolutional recurrent deep learning model for sentence classification. IEEE Access 6:13949–13957.  https://doi.org/10.1109/ACCESS.2018.2814818 CrossRefGoogle Scholar
  37. Herrera RG (2014) Knowledge management systems based on ontology learning. Universidad de Granada, GranadaGoogle Scholar
  38. Hinton GE (2009) Deep belief networks. Scholarpedia 4:5947CrossRefGoogle Scholar
  39. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507MathSciNetCrossRefGoogle Scholar
  40. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554MathSciNetCrossRefGoogle Scholar
  41. Hu S, Zuo Y, Wang L, Liu P (2016) A review about building hidden layer methods of deep learning. J Adv Inf Technol 7:13–22.  https://doi.org/10.12720/jait.7.1.13-22 CrossRefGoogle Scholar
  42. Huang FJ, Boureau Y-L, LeCun Y (2007) Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8Google Scholar
  43. Jiang X, Tan A-H (2005) Mining ontological knowledge from domain-specific text documents. In: Fifth IEEE international conference on data mining. IEEEGoogle Scholar
  44. Jiang X, Tan AH (2010) CRCTOL: a semantic-based domain ontology learning system. J Am Soc Inf Sci Technol 61:150–168CrossRefGoogle Scholar
  45. Kietz J-U, Maedche A, Volz R (2000) A method for semi-automatic ontology acquisition from a corporate intranet. In: EKAW-2000 workshop “Ontologies and Text”, Juan-Les-Pins, FranceGoogle Scholar
  46. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:14085882
  47. Klein D, Manning CD (2003) Accurate unlexicalized parsing. Paper presented at the Proceedings of the 41st Annual meeting on association for computational linguistics—volume 1, Sapporo, JapanGoogle Scholar
  48. Kuang Z, Yu J, Li Z, Zhang B, Fan J (2018) Integrating multi-level deep learning and concept ontology for large-scale visual recognition. Pattern Recogn 78:198–214.  https://doi.org/10.1016/j.patcog.2018.01.027 CrossRefGoogle Scholar
  49. Le QV (2015) A tutorial on deep learning part 2: autoencoders, convolutional neural networks and recurrent neural networks. Google Brain 1–20Google Scholar
  50. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436CrossRefGoogle Scholar
  51. Lin D, Pantel P (2001) DIRT- discovery of inference rules from text. Paper presented at the proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, San Francisco, CaliforniaGoogle Scholar
  52. Liu Q, Xu K, Zhang L, Wang H, Yu Y, Pan Y (2008) Catriple: extracting triples from wikipedia categories. In: Asian semantic web conference. Springer, pp 330–344Google Scholar
  53. Maedche A, Staab S (2000) Discovering conceptual relations from text. In: Horn W (ed) Proceedings of the 14th European conference on artificial intelligence, Berlin, Germany. ECAI'00. IOS Press, pp 321–325Google Scholar
  54. Maedche A, Volz R (2001) The text-to-onto ontology extraction and maintenance environment. In: Proceedings of the ICDM-workshop on integrating data mining and knowledge management, San Jose, CaliforniaGoogle Scholar
  55. Maimon O, Browarnik A (2015) Ontology learning from text: why the ontology learning layer cake is not viable. Int J Signs Semiot Syst 4:1–14.  https://doi.org/10.4018/ijsss.2015070101 CrossRefGoogle Scholar
  56. Mathews KA, Kumar PS (2017) Extracting ontological knowledge from textual descriptions through grammar-based transformation. Paper presented at the proceedings of the knowledge capture conference, Austin, TXGoogle Scholar
  57. Mishra S, Jain S (2015) A study of various approaches and tools on ontology. In: 2015 IEEE international conference on computational intelligence and communication technology (CICT), pp 57–61.  https://doi.org/10.1109/cict.2015.43
  58. Mo D (2012) A survey on deep learning: one small step toward AI Dept Computer Science, Univ of New Mexico, USAGoogle Scholar
  59. Morin E (1999) Automatic acquisition of semantic relations between terms from technical corpora. In: Proceedings of the fifth international congress on terminology and knowledge engineering-TKE’99Google Scholar
  60. Mudhsh BAD, Al-Takhayinh AH, Al-Dala’ien OA (2015) Immediate constituent analysis (ICA). Int J Sci Res Publ 5(6)Google Scholar
  61. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2:1CrossRefGoogle Scholar
  62. Nakaya N, Kurematsu M, Yamaguchi T (2002) A domain ontology development environment using a MRD and text corpus. In: Proceedings of the fifth joint conference on knowledge-based software engineering frontiers in artificial intelligence and applications, pp 242–251Google Scholar
  63. Neelakantan AR (2017) Knowledge representation and reasoning with deep neural networks. Doctoral Dissertation, University of Massachusetts, AmherstGoogle Scholar
  64. Nivre J (2004) Incrementality in deterministic dependency parsing. Paper presented at the proceedings of the workshop on incremental parsing: bringing engineering and cognition together, Barcelona, SpainGoogle Scholar
  65. Oliveira A, Pereira FC, Cardoso A (2001) Automatic reading and learning from text. In: Proceedings of the international symposium on artificial intelligence (ISAI)Google Scholar
  66. Park J, Cho W, Rho S (2010) Evaluating ontology extraction tools using a comprehensive evaluation framework. Data Knowl Eng 69:1043–1061.  https://doi.org/10.1016/j.datak.2010.07.002 CrossRefGoogle Scholar
  67. Pereira FC, Oliveira A, Cardoso A (2000) Extracting concept maps with clouds. In: Proceedings of the Argentine symposium of artificial intelligence (ASAI)Google Scholar
  68. Petrucci G, Ghidini C, Rospocher M (2016) Ontology learning in the deep. In: Knowledge engineering and knowledge management. Springer, Cham, pp 480-495Google Scholar
  69. Sabou M, Wroe C, Goble C, Mishne G (2005) Learning domain ontologies for Web service descriptions: an experiment in bioinformatics. Paper presented at the Proceedings of the 14th international conference on World Wide Web, Chiba, JapanGoogle Scholar
  70. Salakhutdinov R, Hinton G (2007) Semantic hashing. RBM 500:500Google Scholar
  71. Sánchez D, Moreno A (2008) Learning non-taxonomic relationships from web documents for domain ontology construction. Data Knowl Eng 64:600–623CrossRefGoogle Scholar
  72. Sarikaya R, Hinton GE, Deoras A (2014) Application of deep belief networks for natural language understanding. IEEE/ACM Trans Audio Speech Lang Process 22:778–784CrossRefGoogle Scholar
  73. Sekiuchi R, Aoki C, Kurematsu M, Yamaguchi T (1998) DODDLE: a domain ontology rapid development environment. In: Lee H-Y, Motoda H (eds) PRICAI’98: topics in artificial intelligence. Proceedings of 5th Pacific Rim international conference on artificial intelligence Singapore. Springer, Berlin, pp 194–204.  https://doi.org/10.1007/bfb0095269 CrossRefGoogle Scholar
  74. Shamsfard M, Barforoush AA (2003) The state of the art in ontology learning: a framework for comparison. Knowl Eng Rev 18:293–316CrossRefGoogle Scholar
  75. Shamsfard M, Barforoush AA (2004) Learning ontologies from natural language texts. Int J Hum Comput Stud 60:17–63CrossRefGoogle Scholar
  76. Sombatsrisomboon R, Matsuo Y, Ishizuka M (2003) Acquisition of hypernyms and hyponyms from the WWW. In: Proceedings of the 2nd international workshop on active miningGoogle Scholar
  77. Specia L, Motta E (2006) A hybrid approach for relation extraction aimed at the semantic web. In: International conference on flexible query answering systems. Springer, pp 564–576Google Scholar
  78. Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. Paper presented at the proceedings of the 16th international conference on World Wide Web, Banff, Alberta, CanadaGoogle Scholar
  79. Völker J, Hitzler P, Cimiano P (2007) Acquisition of OWL DL axioms from lexical resources. In: The semantic web: research and applications. Springer, Berlin, pp 670–685Google Scholar
  80. Wang H (2015) Semantic deep learning. University of Oregon, OregonGoogle Scholar
  81. Wang J, Liu J, Kong L (2018) Ontology construction based on deep learning. In: Advances in computer science and ubiquitous computing. Springer, Singapore, pp 505–510Google Scholar
  82. Wong W, Liu W, Bennamoun M (2007) Tree-Traversing Ant Algorithm for term clustering based on featureless similarities. Data Min Knowl Disc 15:349–381.  https://doi.org/10.1007/s10618-007-0073-y MathSciNetCrossRefzbMATHGoogle Scholar
  83. Wong W, Liu W, Bennamoun M (2012) Ontology learning from text: a look back and into the future. ACM Comput Surv (CSUR) 44:20CrossRefGoogle Scholar
  84. Zelle JM, Mooney RJ (1993) Learning semantic grammars with constructive inductive logic programming. In: AAAI, pp 817–822Google Scholar
  85. Zhang J, Liu J, Wang X (2016) Simultaneous entities and relationship extraction from unstructured text. Int J Database Theory Appl 9:151–160CrossRefGoogle Scholar
  86. Zhong B, Liu J, Du Y, Liaozheng Y, Pu J (2016) Extracting attributes of named entity from unstructured text with deep belief network. Int J Database Theory Appl 9:187–196CrossRefGoogle Scholar
  87. Zhou L (2007) Ontology learning: state of the art and open issues. Inf Technol Manag 8:241–252CrossRefGoogle Scholar
  88. Zouaq A (2011) An overview of shallow and deep natural language processing for ontology learning. In: Wong W, Liu W, Bennamoun M (eds) Ontology learning and knowledge discovery using the web: challenges and recent advances, vol 2. Information Science Reference (IGI Global), USA, pp 16–37CrossRefGoogle Scholar
  89. Zouaq A, Gasevic D, Hatala M (2011b) Towards open ontology learning and filtering. Inf Syst 36:1064–1081.  https://doi.org/10.1016/j.is.2011.03.005 CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.School of Computer SciencesUniversiti Sains MalaysiaGelugorMalaysia
  2. 2.Faculty of Computer Sciences and EngineeringHodeidah UniversityHodeidahYemen

Personalised recommendations