Abstract
A key component for turning open source data and information into open source intelligence occurs during the analysis and interpretation stages. In addition, verification and validation stages can turn this OSINT into validated OSINT, which has a higher degree of credibility. Due to the wide range of data types that can be extracted from open information sources, the types of data analysis that can be performed on this data is specific to the type of data that we have. This chapter presents a set of analysis processes that can be used when encountering specific types of data regardless of what that data is concerning. These methods will assist an open source investigator in getting the most from their data as well as preparing it for further analysis using visualisation and visual analytics techniques for exploration and presentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
We note that within open police data a degree of geo-masking must take place in order to protect victims, but it is then important that investigators and any other users of the data are aware of the limitations of such a dataset.
References
Abbott C (n.d.) RC(C) evaluation system. Open briefing. http://www.openbriefing.org/intelligenceunit/intelligencemethod/rccsystem/
Andrews S, Yates S, Akhgar B, Fortune D (2013) The ATHENA project: using formal concept analysis to facilitate the actions of crisis responders. In: Akghar B, Yates S (eds) Strategic intelligence management. Elsvier, Oxford, pp 167–180
Bajak A (2015) How Federica Fragapane visualized organized crime in northern Italy. In: Storybench. http://www.storybench.org/how-federica-fragapane-visualized-organized-crime-in-northern-italy/
Bermingham A, Conway M, McInerney L, O’Hare N, Smeaton AF (2009) Combining social network analysis and sentiment analysis to explore the potential for online radicalisation. In: International conference on advances in social network analysis and mining, 2009. ASONAM’09, Athens, pp 231–236
Best Jr RA, Cumming A (2007) Open source intelligence (OSINT): issues for congress, vol 5, Dec 2007
Burn-Murdoch J (2013) Social media analytics: are we nearly there yet? In: The guardian. http://www.theguardian.com/news/datablog/2013/jun/10/social-media-analytics-sentiment-analysis
Carter B (2013) The F.B.I. criticizes the news media after several mistaken reports of an arrest. In: The New York Times. http://www.nytimes.com/2013/04/18/business/media/fbi-criticizes-false-reports-of-a-bombing-arrest.html
Chen H, Chung W, Xu JJ, Wang G, Qin Y, Chau M (2004) Crime data mining: a general framework and some examples. Computer 37(4):50–56
College of Policing (2015) Intelligence report. In: Authorised professional practice. https://www.app.college.police.uk/app-content/intelligence-management/intelligence-report/
Department of the Army (2012) Open source intelligence. http://fas.org/irp/doddir/army/atp2-22-9.pdf
Donovan J (2016) The Twitris sentiment analysis tool by Cognovi Labs predicted the Brexit hours earlier than polls. In: TechCrunch. https://techcrunch.com/2016/06/29/the-twitris-sentiment-analysis-tool-by-cognovi-labs-predicted-the-brexit-hours-earlier-than-polls/
Fortunato S (2010) Community detection in graphs. Phys Rep 486(3):75–174
Gan Q, Zhu M, Li M, Liang T, Cao Y, Zhou B (2014) Document visualization: an overview of current research. Wiley Interdisc Rev Comput Stat 6(1):19–36
Ganter B, Wille R (1999) Formal concept analysis: mathematical foundations. Springer Science & Business Media
Görg C, Kang YA, Liu Z, Stasko JT (2013) Visual analytics support for intelligence analysis. IEEE Comput 46(7):30–38
Gunnell D, Hillier J, Blakeborough L (2016) Social network analysis of an urban street gang using police intelligence data. https://www.gov.uk/government/publications/social-network-analysis-of-an-urban-street-gang-using-police-intelligence-data
Home Office (2016) Social network analysis: “How to guide.” https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/491572/socnet_howto.pdf
Hribar G, Podbregar I, Ivanuša T (2014) OSINT: a “Grey Zone”? Int J Intell CounterIntell 27(3):529–549
Isah H, Neagu D, Trundle P (2015) Bipartite network model for inferring hidden ties in crime data. In: 2015 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM). IEEE, Aug 2015, pp 994–1001
Krebs V (2002a) Uncloaking terrorist networks. First Monday 7(4). http://pear.accc.uic.edu/ojs/index.php/fm/article/view/941/863
Krebs VE (2002b) Mapping networks of terrorist cells. Connections 24(3):43–52
Krebs V (2013) Social network analysis: an introduction. In: OrgNet
Lu Y, Luo X, Polgar M, Cao Y (2010) Social network analysis of a criminal hacker community. J Comput Inf Syst 51(2):31–41
Malm A, Bichler G (2013) Using friends for money: the positional importance of money-launderers in organized crime. Trends Organized Crime 16(4):365–381
Malm AE, Kinney JB, Pollard NR (2008) Social network and distance correlates of criminal associates involved in illicit drug production. Secur J 21(1):77–94
Manning CD, Schütze H (1999) Foundations of statistical natural language processing, vol 999. MIT press, Cambridge
Mastrobuoni G, Patacchini E (2012) Organized crime networks: an application of network analysis techniques to the American mafia. Rev Network Econ 11(3):1–43
McCarthy T, Ratcliffe J (2005) Garbage in, garbage out: geocoding accuracy and spatial analysis of crime. In: Geographic information systems and crime analysis, IGI Global
NATO (2001) NATO open source intelligence handbook
NATO (2002) Exploitation of intelligence on the internet. http://www.oss.net/dynamaster/file_archive/030201/1c0160cde7302e1c718edb08884ca7d7/Intelligence Exploitation of the Internet FINAL 18NOV02.pdf
Noubours S, Pritzkau A, Schade U (2013) NLP as an essential ingredient of effective OSINT frameworks. In: Military communications and information systems conference (MCC), Oct 2013. IEEE, pp 1–7
Pallaris C (2008) Open source intelligence: a strategic enabler of national security. CSS Analyses Secur Policy 3(32):1–3
Palmer C (2013) Police tap social media in wake of London attack. In: ITNews. http://www.itnews.com.au/news/police-tap-social-media-in-wake-of-london-attack-344319
Poelmans J, Elzinga P, Dedene G, Viaene S, Kuznetsov SO (2011) A concept discovery approach for fighting human trafficking and forced prostitution. In: International conference on conceptual structures. Springer, Berlin, July 2011, pp 201–214
Procter R, Vis F, Voss A (2011) Reading the riots: investigating London’s summer of disorder. In: The guardian. http://www.theguardian.com/uk/interactive/2011/dec/07/london-riots-twitter
Rauscher J, Swiezinski L, Riedl M, Biemann C (2013) Exploring cities in crime: significant concordance and co-occurrence in quantitative literary analysis. In: Proceedings of the computational linguistics for literature workshop at NAACL-HLT, June 2013
Salton G, McGill MJ (1986) Introduction to modern information retrieval
Schaurer F, Störger J (2013) Intelligencer guide to the study of intelligence. The evolution of open source intelligence (OSINT). J US Intell Stud 19:53–56
Tayebi MA, Bakker L, Glasser U, Dabbaghian V (2011) Locating central actors in co-offending networks. In: 2011 International Conference on advances in social networks analysis and mining (ASONAM), July 2011. IEEE, pp 171–179
Tompson L, Johnson S, Ashby M, Perkins C, Edwards P (2015) UK open source crime data: accuracy and possibilities for research. Cartogr Geogr Inf Sci 42(2):97–111
Xu JJ, Chen H (2004) Fighting organized crimes: using shortest-path algorithms to identify associations in criminal networks. Decis Support Syst 38(3):473–487
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this chapter
Cite this chapter
Gibson, H., Ramwell, S., Day, T. (2016). Analysis, Interpretation and Validation of Open Source Data. In: Akhgar, B., Bayerl, P., Sampson, F. (eds) Open Source Intelligence Investigation. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-47671-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-47671-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47670-4
Online ISBN: 978-3-319-47671-1
eBook Packages: Computer ScienceComputer Science (R0)