Text Mining of Internet Content: The Bridge Connecting Product Research with Customers in the Digital Era

Abstract

Primary and secondary market research usually deal with analysis of available data on existing products and customers’ preferences for features in possible new products. This analysis helps a manufacturer to identify nuggets of opportunity in defining and positioning of new products in global markets. Considering the fact that the number of Internet users and quantum of textual data available on the Internet are increasing exponentially, we can say that Internet is probably the largest data repository that manufacturer’s cannot ignore, in order to better understand customers’ opinions about products. This emphasizes the importance of web mining to locate and process relevant information from billions of documents available online. Its nature of being unstructured and dynamic, an online document adds more challenges to web mining. This paper focuses on application of web content analysis, a type of web mining in business intelligence for product review. We provide an overview of techniques used to solve the problem and challenges involved in the same.