Skip to main content
SpringerLink
Log in

Hybrid Theme Crawler Based on Links and Semantics

  • Conference paper
  • First Online:
The 8th International Conference on Computer Engineering and Networks (CENet2018) (CENet2018 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 905))

Included in the following conference series:

  • 767 Accesses

Abstract

Common theme crawler generally analyses the page content or link structure, without solving the problem of computational complexity and easy “myopia”, resulting in the page of recall and precision is not high. This paper introduces a mixed theme decision strategy, which fully considers the text content and link structure of the page. By introducing knowledge map database and entity database, the computational complexity is simplified and the judgment accuracy is increased. The experiment shows that the rate of inspection and precision is greatly improved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Yang, X.U., Wang, W.Y.: Research on Subject relevance algorithm of theme crawler. Mod. Comput. (2016)

    Google Scholar 

  2. Qiu, L., Lou, Y., Chang, M.: Research on theme crawler based on shark-search and PageRank algorithm. In: International Conference on Cloud Computing and Intelligence Systems, pp. 268–271. IEEE (2016)

    Google Scholar 

  3. Zhang, Y.F., Zheng, S.H.: Heritrix based theme crawler design. J. Chang. Univ. Technol. (2016)

    Google Scholar 

  4. Qiu, L., Lou, Y.S., Chang, M.: An improved shark-search algorithm for theme crawler. Microcomput. Appl. (2017)

    Google Scholar 

  5. Kumar, N., Singh, M.: Framework for distributed semantic web crawler. In: International Conference on Computational Intelligence and Communication Networks, pp. 1403–1407. IEEE (2016)

    Google Scholar 

  6. Wu, T., Liang, Y., Wu, C., Piao, S.F., Ma, D.Y., Zhao, G.Z., Han, X.S.: A chinese topic crawler focused on customer development. Procedia CIRP 56, 476–480 (2016)

    Article  Google Scholar 

  7. Wu, L., Wang, Y.B.: The research of the topic crawler algorithm based on semantic similarity aggregation. J. Commun. Univ. China (Sci. Technol.) (2018)

    Google Scholar 

  8. Yuan, Y.W., Lu, P.J.: Design of a topic crawler for college bidding announcement. Softw. Guid. (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Kang Zhao, Yang Yang, Zhipeng Gao & Long Cheng

Authors
  1. Kang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhipeng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Long Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Yang .

Editor information

Editors and Affiliations

  1. School of Computing, Edinburgh Napier University, Edinburgh, UK

    Qi Liu

  2. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Mustafa Mısır

  3. School of Computer, Fudan University, Shanghai, China

    Xin Wang

  4. College of Information Science and Technology/College of Cyber Security, Jinan University, Guangzhou, China

    Weiping Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, K., Yang, Y., Gao, Z., Cheng, L. (2020). Hybrid Theme Crawler Based on Links and Semantics. In: Liu, Q., Mısır, M., Wang, X., Liu, W. (eds) The 8th International Conference on Computer Engineering and Networks (CENet2018). CENet2018 2018. Advances in Intelligent Systems and Computing, vol 905. Springer, Cham. https://doi.org/10.1007/978-3-030-14680-1_74

Download citation

Publish with us

Policies and ethics

Search

Navigation