Advertisement

IIRS: A Novel Framework of Identifying Commodity Entities on E-commerce Big Data

  • Qiqing Fang
  • Yamin Hu
  • Shujun Lv
  • Lejiang Guo
  • Lei Xiao
  • Yahui Hu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9098)

Abstract

Identification of the same commodity entities is a major challenge in the heterogeneous multi-source e-commerce of big data. This paper introduces a framework based on Map-Reduce, called IIRS, which is made up of data index, data integration, entity recognition and data sorting. IIRS aims to form the unified model and high efficient commodity information with building an index model based on commodity’s attribute/value and constructing a global model map to record commodity’s attribute and value, identify the commodity entities in different e-commerce with measuring the similarity of the commodity’s identity, and then output the same identity commodity sets and their associated properties organized in the inverted index list. Through an extensive experimental study on real e-commerce dataset on Hadoop, IIRS significantly demonstrates its feasibility, accuracy, and high efficiency.

Keywords

Big data E-commerce Entity identification Map reduce Normalization identification 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Herndndez, M.A., Stolfo, S.J.: The merge/purge problem for large databases. SIGMOD 24(2), 127–138 (1995)CrossRefGoogle Scholar
  2. 2.
    Arasu, A., Kaushik, R.: A grammar-based entity representation framework for data cleaning. In: SIGMOD, pp. 233–244 (2009)Google Scholar
  3. 3.
    Wenfei, F., Xibei, J., Jianzhong, L., et al.: Reasoning about record matching rules. VLDB 2(1), 407–418 (2009)Google Scholar
  4. 4.
    Chaudhuri, S., Ganti, V., Motwani, R.: Robust identification of fuzzy duplicates. In: ICDE 2005, pp. 865–876 (2005)Google Scholar
  5. 5.
    Chen, Z., Kalashnikov, D.V., Mehrotra, S.: Adaptive graphical approach to entity resolution. In: Proc. of the 7th ACM IEEE-CS Joint Conf. on Digital Libraries, New York, pp. 204–213 (2007)Google Scholar
  6. 6.
    Singla, P., Domingos, P.: Entity resolution with markovlogic. In: ICDM, pp. 572–582 (2006)Google Scholar
  7. 7.
    Augsten, N., Bohlen, M., Dyreson, C., et aI.: Approximate joins for data—centric XML. In: ICDE 2008, pp. 814–823 (2008)Google Scholar
  8. 8.
    Li, W., Rong, Z., Chaofeng, S., et al.: A Product Normalization Method for E-Commerce. Chinese Journal of Compters 37(2), 312–325 (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Qiqing Fang
    • 1
  • Yamin Hu
    • 1
  • Shujun Lv
    • 1
  • Lejiang Guo
    • 1
  • Lei Xiao
    • 1
  • Yahui Hu
    • 1
    • 2
  1. 1.Air Force Early Warning AcademyWuhanChina
  2. 2.School of ComputerWuhan UniversityWuhanChina

Personalised recommendations