Using Dark Web Crawler to Uncover Suspicious and Malicious Websites

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 782)


It has been recognized that most of the Internet is not accessible through regular search engines and web browsers. This part of the web is known as dark web, and the surface is about 400 to 500 times larger than the size of the web that we know [1]. The goal of this project is to design a dark web crawler that can uncover any suspicious and malicious websites from the TOR (The Onion Router) network. The proposed system will create a database of suspicious and malicious websites by scraping relative linking attributes that may be contained within TOR network web pages. The proposed database automatically updates itself and it will archive previous versions of TOR sites while saving available working links. This will give law enforcement authorities the ability to search both the current TOR database and previous versions of the Database to detect suspicious and malicious websites.


Dark web crawler TOR Suspicious Malicious websites 


  1. 1.
    Staff, C.: The Deep Web Is The 99% of The Internet You Don’t See (2016). Curiosity. Accessed 01 Mar 2017
  2. 2.
    Balduzzi, M.: Ciancaglini, V.: Cybercrime in the Deep Web. Report by Black Hat EU, Amsterdam (2015)Google Scholar
  3. 3.
    Finklea, K.: Dark Web. Report Published by Congressional Research Service (2017)Google Scholar
  4. 4.
    Finklea, K.: Cybercrime: Conceptual Issues for Congress and U.S. Law Enforcement. Report Published by Congressional Research Service (2015)Google Scholar
  5. 5.
    Poulsen, K.: FBI Admits It Controlled Tor Servers Behind Mass Malware Attack. Published in (2013)Google Scholar
  6. 6.
    Aliens, C.: Anonymous Hacks Freedom Hosting II, Bringing Down Almost 20% of Active Darknet Sites. Report published by DeepDot.Web (2017)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Information TechnologyKwantlen Polytechnic UniversitySurreyCanada

Personalised recommendations