Finding and Extracting Academic Information from Conference Web Pages
This paper proposes a method for finding and extracting academic information from conference Web pages. The main contributions include: (1) A lightweight topic crawling method based on search engine is used to crawl academic conference Web pages. (2) An new vision-based page segmentation algorithm is proposed to improve the result of classical VIPS algorithm by introducing complete tree. This algorithm can divide Web pages into text blocks. (3) Using bayesian network classifier, all text blocks are classified as 10 categories according to its vision features, key-word features and text content features. The initial classification results have 75 % precision and 67 % recall. (4) The context information of text blocks are employed to repair and refine initial classification results, which are improved to 96 % precision and 98 % recall. Finally, academic information is easily extracted from the classified text blocks. Experimental results on real-world datasets show that our method is effective and efficient for finding and extracting academic information from conference Web pages.
KeywordsTopic crawler Web information extraction Page segmentation
- 1.Tang, J., Zhang, J., Yao, L., Li, J., et al.: ArnetMiner: extraction and mining of academic social networks. Presented at the Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA (2008)Google Scholar
- 4.Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. Presented at the Proceedings of the Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, California, United States (1998)Google Scholar
- 6.Cai, D., Yu, S., Wen, J.-R., Ma, W.-Y.: VIPS: a vision-based page segmentation algorithm. Microsoft Technical Report (2003)Google Scholar
- 8.Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)Google Scholar