Technology to the Rescue: A Software-Based Approach to Tackle Extreme Speech

  • Jamil AmmarEmail author
  • Songhua Xu


This chapter discusses how a variety of software technologies can be leveraged tackle extreme speech on social media. The chapter first overviews Google’s strategy to direct extremist searches towards anti-radicalization websites and comments on its efficacy. The chapter then proceeds to propose two promising technical approaches that can empower private companies and relevant federal agencies to reduce religious extremists’ presence on and impact through social media in a systematic and economically viable way. The first approach leverages content analysis and multimedia mining algorithms to automatically detect content produced by extremists. Using the approach, immediately after a user uploads a document, picture, or video onto a social media platform, a backend engine equipped with a series of state-of-the-art computational content analysis algorithms will be deployed to analyze the types of topics latent in the uploaded material, which will enable social media platforms to discover materials intentionally mislabeled and mis-described by extremists in their attempt to circumvent the traditional keyword or text matching based detection mechanism. Leveraging the same backend content filtering engine, social media platforms can also automatically aggregate multiple files separately updated by one or multiple users independently according to the content similarity between these files. By detecting duplicate or nearly duplicate content on social media through the aforementioned algorithmic engine for automatic content analysis, computers can comprehensively and systematically aggregate otherwise isolated user behaviors associated with individual copies of essentially the same material to produce a consolidated view of content request and consumption for more effective content screening and surveillance. The second approach algorithmically examines collective content consumption behaviors on social media to detect documents, pictures, and videos posted by religious extremists. This approach will detect content posted by religious extremists by observing and analyzing the information propagation pathways and consumption patterns on social media through an automatic and algorithmic approach. Compared with the current self-reporting-based practice, the computational approach can react more efficiently to detect content of concern even when the targeted audience chooses not to cooperate with the social media platform. Additionally, this approach can also save a significant amount of expert labeling effort to train the automatic algorithmic content detector. Overall, the approach will be able to automatically determine the true nature of the content carried by the concerned video regardless of any deceitful labels, titles, or description text the video’s producer or distributor may purposefully associate the video with. The new approach will be able to automatically and systematically detect all videos carrying religiously inciting content in a way that is also much more efficient and comprehensive than the current user self-reporting-based practice. Lastly, the chapter concludes by briefly overviewing three categories of promising software technologies, including natural language processing, machine learning, and social network analysis technologies, which can be tremendously powerful for tackling religious extreme speech on social media.


Software technologies Anti-radicalization websites Content analysis Multimedia mining algorithms Extremists Extreme speech 

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.Rutgers School of Law–NewarkNewarkUSA
  2. 2.New Jersey Institute of TechnologyKearnyUSA

Personalised recommendations