Needles in Haystacks: Law, Capability, Ethics, and Proportionality in Big Data Intelligence-Gathering

  • Julian Richards


As we move resolutely into a Big Data age, it appears that a sense of ‘panoptic panic’, boosted by the Snowden revelations, about the expanding capabilities of our intelligence agencies is growing amongst sections of the public. Law enforcers and security officials will often speak about the need to amass large amounts of data to find needles in haystacks, but civil libertarians and cyber-utopianists warn of a slide towards Orwellian powers of mass surveillance. Big Data implies an epistemological shift in data analysis, and opens up new opportunities for security agencies to become more proactive and pre-emptive. This chapter argues that, while the risks of such a move must be recognised, we must also be able to deliver security responsibly.


Big Data Intelligence agencies Snowden revelations Global surveillance Ethics of intelligence Law enforcement Intelligence-led policing Intelligence-gathering Risk society 


  1. Barlow, J.P. (1996, February 8). A declaration of the independence of Cyberspace. Davos. Retrieved October 22, 2015, from
  2. BBC. (2015, January 18). Nick Clegg defends opposition to ‘snoopers’ charter. Retrieved July 16, 2015, from
  3. Beck, U. (2002). The terrorist threat: World risk society revisited. Theory, Culture and Society, 19(4), 39–55.CrossRefGoogle Scholar
  4. Butler, D. (2013). When Google got flu wrong. Nature, 494(7436), 155–156.CrossRefGoogle Scholar
  5. CNN. (2002). Bush State of the Union address. Retrieved July 17, 2015, from
  6. Deleuze, G. (1992). Postscript on the societies of control. October, 59, 3–7.Google Scholar
  7. Huppert, J. (2012, December 11). Communications Data Bill cannot proceed. Liberal Democrat Voice. Retrieved July 17, 2015, from
  8. Kerr, I., & Earle, J. (2013). Prediction, preemption, presumption: How Big Data threatens big picture privacy. Stanford Law Review Online, 66(65).Google Scholar
  9. Lyon, D. (2014, July–December). Surveillance, Snowden, and Big Data: Capacities, consequences, critique. Big Data and Society.Google Scholar
  10. Omand, D. (2012). Securing the state. London: C. Hurst and Co.Google Scholar
  11. Richards, J. (2012). A guide to national security: Threats, responses and strategies. Oxford: Oxford University Press.Google Scholar
  12. The Guardian. (2014, July 15). Academics: UK ‘Drip’ data law changes are ‘serious expansion of surveillance. Retrieved July 17, 2015, from
  13. Zwitter, A. (2014, July–December). Big Data ethics. Big Data and Society. Google Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s) 2016

Authors and Affiliations

  • Julian Richards
    • 1
  1. 1.Centre for Intelligence and Security Studies (BUCSIS)University of BuckinghamBuckinghamUK

Personalised recommendations