Advertisement

Python Scrapers for Scraping Cryptomarkets on Tor

  • Yubao WuEmail author
  • Fengpan Zhao
  • Xucan Chen
  • Pavel Skums
  • Eric L. Sevigny
  • David Maimon
  • Marie Ouellet
  • Monica Haavisto Swahn
  • Sheryl M. Strasser
  • Mohammad Javad Feizollahi
  • Youfang Zhang
  • Gunjan Sekhon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11611)

Abstract

Cryptomarkets are commercial websites on the web that operate via darknet, a portion of the Internet that limits the ability to trace users’ identity. Cryptomarkets have facilitated illicit product trading and transformed the methods used for illicit product transactions. The survellience and understanding of cryptomarkets is critical for law enforcement and public health. In this paper, we design and implement Python scrapers for scraping cryptomarkets. The design of the scraper system is described with details and the source code of the scrapers is shared with the public.

Keywords

Scraper Cryptomarket Tor Darknet MySQL 

References

  1. 1.
    Martin, J.: Drugs on the Dark Net: How Cryptomarkets are Transforming the Global Trade in Illicit Drugs (2014)Google Scholar
  2. 2.
    Aldridge, J., Décary-Hétu, D.: Not an ‘Ebay for Drugs’: the Cryptomarket ‘Silk Road’ as aparadigm shifting criminal innovation. Available at SSRN 2436643 (2014)Google Scholar
  3. 3.
    Christin, N.: Traveling the silk road: a measurement analysis of a large anonymous online marketplace. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 213–224. ACM (2013)Google Scholar
  4. 4.
    EMCDDA: Europol: DarkNet markets ecosystem – lifetimes and reasons for closure of over 100 global darknet markets offering drugs, sorted by date (2018)Google Scholar
  5. 5.
    European Monitoring Centre for Drugs and Drug Addiction and Europol: Drugs and the DarkNet: perspectives for enforcement, research and policy (2017)Google Scholar
  6. 6.
    DarkNet Market Archives (2013–2015). https://www.gwern.net/DNM-archives. Accessed 12 Feb 2019
  7. 7.
    Lawrence, H., Hughes, A., Tonic, R., Zou, C.: D-miner: a framework for mining, searching, visualizing, and alerting on darknet events. In: 2017 IEEE Conference on Communications and Network Security (CNS), pp. 1–9. IEEE (2017)Google Scholar
  8. 8.
    Hayes, D., Cappa, F., Cardon, J.: A framework for more effective dark web marketplace investigations. Information 9(8), 186 (2018)CrossRefGoogle Scholar
  9. 9.
    Tor Relay Configurator. https://tor-relay.co/. Accessed 15 Feb 2019
  10. 10.
    Tor Good Bad ISPs. https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISPs. Accessed 15 Feb 2019
  11. 11.
  12. 12.
    Tor Manual. https://www.torproject.org/docs/tor-manual.html.en. Accessed 15 Feb 2019
  13. 13.
    Tor Relay Search. https://metrics.torproject.org/rs.html. Accessed 15 Feb 2019
  14. 14.
    Check Tor Connection. https://check.torproject.org. Accessed 15 Feb 2019

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yubao Wu
    • 1
    Email author
  • Fengpan Zhao
    • 1
  • Xucan Chen
    • 1
  • Pavel Skums
    • 1
  • Eric L. Sevigny
    • 2
  • David Maimon
    • 2
  • Marie Ouellet
    • 2
  • Monica Haavisto Swahn
    • 3
  • Sheryl M. Strasser
    • 3
  • Mohammad Javad Feizollahi
    • 4
  • Youfang Zhang
    • 4
  • Gunjan Sekhon
    • 4
  1. 1.Department of Computer ScienceGeorgia State UniversityAtlantaUSA
  2. 2.Department of Criminal Justice and CriminologyGeorgia State UniversityAtlantaUSA
  3. 3.School of Public HealthGeorgia State UniversityAtlantaUSA
  4. 4.Institute for InsightGeorgia State UniversityAtlantaUSA

Personalised recommendations