Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Data Profiling

  • Theodore Johnson
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_601

Synonyms

Database profiling

Definition

Data profiling refers to the activity of creating small but informative summaries of a database [1]. These summaries range from simple statistics such as the number of records in a table and the number of distinct values of a field, to more complex statistics such as the distribution of n-grams in the field text, to structural properties such as keys and functional dependencies. Database profiles are useful for database exploration, detection of data quality problems [2], and for schema matching in data integration [1]. Database exploration helps a user identify important database properties, whether it is data of interest or data quality problems. Schema matching addresses the critical question, “do two fields or sets of fields or tables represent the same information?” Answers to these questions are very useful for designing data integration scripts.

Historical Background

Databases which support a complex organization tend to be quite complex...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Evoke Software. Data profiling and mapping, the essential first step in data migration and integration projects. Available at: http://www.evokesoftware.com/pdf/wtpprDPM.pdf (2000).
  2. 2.
    Dasu T, Johnson T, Muthukrishnan S, Shkapenyuk V. Mining database structure; or, how to build a data quality browser. In: Proceedings of the ACM SIGMOD International Conference on Management of data; 2002. p. 240–51.Google Scholar
  3. 3.
    Dasu T, Johnson T. Exploratory data mining and data cleaning. New York: Wiley Interscience; 2003.CrossRefzbMATHGoogle Scholar
  4. 4.
    Kang J, Naughton JF. On schema matching with opaque column names and data values. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2003. p. 205–16.Google Scholar
  5. 5.
    Broder A. On the resemblance and containment of documents. In: Proceedings of the IEEE Conference on Compression and Comparison of Sequences; 1997. p. 21–9.Google Scholar
  6. 6.
    Dasu T, Johnson T, Marathe A. Database exploration using database dynamics. IEEE Data Eng Bull. 2006;29(2):43–59.Google Scholar
  7. 7.
    Gravano L, Ipeirotis PG, Jagadish HV, Koudas N, Muthukrishnan S, Srivastava D. Approximate String Joins in a Database (Almost) for Free. In: Proceedings of the 27th International Conference on Very Large Data Bases; 2001. p. 491–500.Google Scholar
  8. 8.
    Huhtala Y, Karkkainen J, Porkka P, Toivonen H. TANE: an efficient algorithm for discovering functional and approximate dependencies. Comp J. 1999;42(2):100–11.CrossRefzbMATHGoogle Scholar
  9. 9.
    Shen W, DeRose P, Vu L, Doan AH, Ramakrishnan R. Source-aware entity matching: a compositional approach. In: Proceedings of the 23rd International Conference on Data Engineering. p. 196–205.Google Scholar
  10. 10.
    IBM Websphere Information Integration. Available at: http://ibm.ascential.com
  11. 11.
    Informatica Data Explorer. Available at: http://www.informatica.com/products_services/data_explorer

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.AT&T Labs – ResearchFlorham ParkUSA

Section editors and affiliations

  • Venkatesh Ganti
    • 1
  1. 1.Microsoft ResearchMicrosoft CorporationRedmondUSA