Abstract
Keyword and feature extraction is a fundamental problem in data mining and document processing. A majority of applications directly depend on the quality and speed of keyword and feature extraction pre-processing results. In the current paper we present novel algorithms for feature extraction and change detection in unstructured data, primarily in textual and sequential data. Our approach is based on ideas from image processing and especially on the Helmholtz Principle from the Gestalt Theory of human perception. The improvements due to the novel feature extraction technique are demonstrated on several key applications: classification for strengthening document security and storage optimization, automatic summarization and segmentation for problems of information overload. The developed algorithms and applications are the result of research collaboration between Cardiff University School of Mathematics and HP Laboratories.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balinsky, A., Balinsky, H. and Simske, S.J.: On Helmholtz’s principle for document processing. In: 10 ACM Symposium on Document Engineering (DocEng2010), Manchester, 21–24 September 2010. http://doi.acm.org/10.1145/1860559.1860624
Desolneux, A., Moisan, L., Morel, J.-M.: From Gestalt Theory to Image Analysis: A Probabilistic Approach. Series in Interdisciplinary Applied Mathematics, vol. 34. Springer, New York (2008)
Balinsky, A., Balinsky, H., Simske, S.J: Keyword determination based on a weight of meaningfulness. US Patent 8,375,022, 12 Feb 2013
Balinsky, A., Balinsky, H., Simske, S.J.: Document sentences as a small world. IEEE SMC 2011, 9–12 (2011). doi:10.1109/ICSMC.2011.6084065
Dadachev, B., Balinsky, A., Balinsky, H., Forman, G.: Automatic text and data stream segmentation using weighted feature extraction. In: Proceedings of the 3rd IMA Conference on Mathematics in Defence, October 2013
The Porter Stemming Algorithm: Official home page of the Porter stemming algorithm. http://tartarus.org/martin/PorterStemmer/index.html
Acknowledgments
We would like to thank EPSRC, Hewlett-Packard and Cardiff University for funding several PhD projects and allowing us to develop our research collaboration. We also want to thank our colleagues from Hewlett-Packard Laboratories for helping us to transform mathematical ideas into real products and services.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Balinsky, A., Balinsky, H., Simske, S. (2016). Detecting Unusual Behaviour and Mining Unstructured Data. In: Aston, P., Mulholland, A., Tant, K. (eds) UK Success Stories in Industrial Mathematics. Springer, Cham. https://doi.org/10.1007/978-3-319-25454-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-25454-8_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25452-4
Online ISBN: 978-3-319-25454-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)