Abstract
The existing database system data quantity is huge, many of which are repeated data. Using the traditional approach for detecting approximately duplicate records to find similar duplicate records in the database will involve very large time complexity and space complexity, unable to obtain very good results. This chapter presents a method based on improved genetic neural network approach for detecting approximately duplicate records, using genetic algorithm to optimize the network’s initial weights; and then using the BP algorithm to train the detection data to obtain network model. The experimental results show that this method can effectively solve the huge amount of approximately duplicate record data detection problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Elmagarmid AK, Ipeirotis PG, Verykios VS (2007) Duplicate record detection: a survey. IEEE Trans Knowl Data Eng 19(1):1–6
Huang L, Jin H, Yuan P (2008) Duplicate records cleansing with length filtering and dynamic weighting. Fourth Int Conf Semant Knowl Grid 8:95–100
Hernandez M, Stolfo S (1995) The merge purge problem for large databases, vol 5. ACM Press, New York, pp 127–130
Monge AE, Elkan CR (1997) An efficient domain-independent algorithm for detecting approximately duplicate database records. Proceedings of workshop on research issues on data mining and knowledge discovery, vol 7. Tucson, pp 23–29
Gravano L, Ipeirotis PG (2001) Using Q-grams in DBMS for approximate string processing. IEEE Data Eng Bull 24(4):28–34
Lee ML, Lu H, Ling TW et a1 (1999) Cleansing data for mining and warehousing. Proceedings of the 10th international conference on database and expert systems applications, vol 5. Florence, pp 751–756
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning, vol 1. Addison-Wesley, MA, pp 1–3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this paper
Cite this paper
Liu, X., Xu, L. (2013). Detecting Approximately Duplicate Records in Database. In: Zhong, Z. (eds) Proceedings of the International Conference on Information Engineering and Applications (IEA) 2012. Lecture Notes in Electrical Engineering, vol 220. Springer, London. https://doi.org/10.1007/978-1-4471-4844-9_45
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4844-9_45
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4843-2
Online ISBN: 978-1-4471-4844-9
eBook Packages: EngineeringEngineering (R0)