In Search of Public Agenda with Text Mining: An Exploratory Study of Agenda Setting Dynamics Between the Traditional Media and Wikipedia

  • Philip T. Y. LeeEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11154)


When Downs [1] proposed his famous issue-attention cycle in 1972, he thought that the mass media would report news and information that arouses people’s interests. This thought, however, is prone to challenges. With the prevalence of the Internet and, perhaps more importantly, the concept of Web 2.0, Wikipedia becomes another major source of information for the public. Given that Wikipedia allows anyone to edit the content, the details about a particular event or issue on pages of Wikipedia can be considered as a quasi-public agenda. Understanding this quasi-public agenda may help us evaluate different models of policy cycles, including the Downs’ famous issue-attention cycle. My study aims to assess the agenda setting dynamics among 5 major news outlets in the UK, as the traditional mass media, and Wikipedia, as a form of participatory journalism. By agenda, it refers to the choices of frames and sentiment. Using text mining techniques, my study assesses the choices of frames and sentiment adopted by the articles of the news outlets and the Wikipedia pages concerning with the issue Brexit. The timeline of the study is between the date when the Wikipedia page “Brexit” emerged and the date of the Brexit referendum. The study also explores the possible relationship between these agendas. Frame analysis of the news articles will be conducted through automatic text classification, whereas the frames on the Wikipedia pages will be analyzed through both text classification and clustering. Lexicon-based approach will be used for sentiment analysis. The relationship between the agendas will be explored through Granger-causality tests.


Agenda setting Media agenda Participatory journalism Wikipedia Text mining 


  1. 1.
    Downs, A.: Up and down with ecology: the issue-attention cycle. Polit. Am. Econ. Policy Mak. 48 (1996)Google Scholar
  2. 2.
    Baumgartner, F.R., Jones, B.D.: Agendas and instability in American politics. University of Chicago Press, Chicago (2010)Google Scholar
  3. 3.
    Howlett, M.: Issue-attention and punctuated equilibria models reconsidered: an empirical examination of the dynamics of agenda-setting in Canada. Can. J. Polit Sci. 30(01), 3–29 (1997)Google Scholar
  4. 4.
  5. 5.
    Rowinski, P.: Mind the gap: the language of prejudice and the press omissions that led a people to the precipice (2016).
  6. 6.
  7. 7.
  8. 8.
    Bowman, S., Willis, C.: We Media: How Audiences are Shaping the Future of News and Information. The Media Center at The American Press Institute, Arlington (2003)Google Scholar
  9. 9.
    Lih, A. (2004). Wikipedia as participatory journalism: reliable sources? Metrics for evaluating collaborative media as a news resource. NatureGoogle Scholar
  10. 10.
    Entman, R.M.: Punctuating the homogeneity of institutionalized news: abusing prisoners at Abu Ghraib versus killing civilians at Fallujah. Polit. Commun. 23, 215–224 (2006)Google Scholar
  11. 11.
    McCombs, M.E., Shaw, D.L.: The agenda-setting function of mass media. Public Opin. Q. 36, 176–187 (1972)Google Scholar
  12. 12.
    Baker, C.E.: Advertising and a Democratic Press. Oxford University Press, New York (1994)Google Scholar
  13. 13.
    Bennett, W.L.: News: The Politics of Illusion, 7th edn. Pearson, New York (2007)Google Scholar
  14. 14.
    Anderson, C.W.: Deliberative, agonistic, and algorithmic audiences: journalism’s vision of its public in an age of audience transparency. Int. J. Commun. 5, 529–547 (2011)Google Scholar
  15. 15.
    Harcup, T., O’neill, D.: What is news? Galtung and Ruge revisited. Journal. Stud. 2(2), 261–280 (2001)Google Scholar
  16. 16.
    Lasica, J.D.: Blogs and journalism need each other. Nieman Rep. 57(3), 70–74 (2003)Google Scholar
  17. 17.
    Gillmor, D.: Moving toward participatory journalism. Nieman Rep. 57(3), 79–80 (2003)Google Scholar
  18. 18.
    Lodge, M., Hood, C.: Pavlovian policy responses to media feeding frenzies? Dangerous dogs regulation in comparative perspective. J. Conting. Crisis Manag. 10(1), 1–13 (2002)Google Scholar
  19. 19.
    Altaweel, M., Bone, C.: Applying content analysis for investigating the reporting of water issues. Comput. Environ. Urban Syst. 36(6), 599–613 (2012)Google Scholar
  20. 20.
    Talamini, E., Caldarelli, C.E., Wubben, E.F., Dewes, H.: The composition and impact of stakeholders’ agendas on US ethanol production. Energy Policy 50, 647–658 (2012)Google Scholar
  21. 21.
    Talamini, E., Dewes, H.: The macro-environment for liquid biofuels in Brazilian science and public policies. Sci. Public Policy 39, 13–29 (2012)Google Scholar
  22. 22.
    Talamini, E., Wubben, E.F., Dewes, H.: The macro-environment for liquid biofuels in german science, mass media and government. Rev. Eur. Stud. 5(2), 33 (2013)Google Scholar
  23. 23.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, no. 14, pp. 281–297, June 1967Google Scholar
  24. 24.
    Quinn, K.M., Monroe, B.L., Colaresi, M., Crespin, M.H., Radev, D.R.: How to analyze political attention with minimal assumptions and costs. Am. J. Polit. Sci. 54(1), 209–228 (2010)Google Scholar
  25. 25.
    Boussalis, C., Coan, T.G.: Text-mining the signals of climate change doubt. Glob. Environ. Change 36, 89–100 (2016)Google Scholar
  26. 26.
    Grimmer, J.: A Bayesian hierarchical topic model for political texts: measuring expressed agendas in senate press releases. Polit. Anal. 18(1), 1–35 (2010)Google Scholar
  27. 27.
    Silva, M.J., Carvalho, P., Sarmento, L., de Oliveira, E., Magalhaes, P.: The design of OPTIMISM, an opinion mining system for Portuguese politics. In: New Trends in Artificial Intelligence: Proceedings of EPIA, pp. 12–15 (2009)Google Scholar
  28. 28.
    Thomas, M., Pang, B., Lee, L.: Get out the vote: determining support or opposition from congressional floor-debate transcripts. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pp. 327–335. Association for Computational Linguistics, July 2006Google Scholar
  29. 29.
    Taddy, M.: Measuring political sentiment on twitter: factor optimal design for multinomial inverse regression. Technometrics 55(4), 415–425 (2013)MathSciNetGoogle Scholar
  30. 30.
    Malouf, R., Mullen, T.: Taking sides: user classification for informal online political discourse. Internet Res. 18(2), 177–190 (2008)Google Scholar
  31. 31.
    Godbole, N., Srinivasaiah, M., Skiena, S.: Large-scale sentiment analysis for news and blogs. ICWSM 7(21), 219–222 (2007)Google Scholar
  32. 32.
    Balahur, A., et al.: Sentiment analysis in the news. arXiv preprint arXiv:1309.6202 (2013)
  33. 33.
    Dang-Xuan, L., Stieglitz, S., Wladarsch, J., Neuberger, C.: An investigation of influentials and the role of sentiment in political communication on twitter during election periods. Inf. Commun. Soc. 16(5), 795–825 (2013)Google Scholar
  34. 34.
    Park, S.J., Lim, Y.S., Sams, S., Nam, S.M., Park, H.W.: Networked politics on Cyworld: the text and sentiment of Korean political profiles. Soc. Sci. Comput. Rev. 29(3), 288–299 (2011)Google Scholar
  35. 35.
    Tumasjan, A., Sprenger, T.O., Sandner, P.G., Welpe, I.M.: Predicting elections with twitter: what 140 characters reveal about political sentiment. ICWSM 10, 178–185 (2010)Google Scholar
  36. 36.
    Lim, J.: A cross-lagged analysis of agenda setting among online news media. Journal. Mass Commun. Q. 83(2), 298–312 (2006)MathSciNetGoogle Scholar
  37. 37.
    Dotson, D.M., Jacobson, S.K., Kaid, L.L., Carlton, J.S.: Media coverage of climate change in Chile: a content analysis of conservative and liberal newspapers. Environ. Commun. J. Nat. Cult. 6(1), 64–81 (2012)Google Scholar
  38. 38.
    Meraz, S.: Is there an elite hold? Traditional media to social media agenda setting influence in blog networks. J. Comput. Mediat. Commun. 14(3), 682–707 (2009)Google Scholar
  39. 39.
    Wu, Y., Atkin, D., Lau, T.Y., Lin, C., Mou, Y.: Agenda setting and micro-blog use: an analysis of the relationship between Sina Weibo and newspaper agendas in China. J. Soc. Media Soc. 2(2), 8–25 (2013)Google Scholar
  40. 40.
    Ali, S.R., Fahmy, S.: Gatekeeping and citizen journalism: the use of social media during the recent uprisings in Iran, Egypt, and Libya. Media War Confl. 6(1), 55–69 (2013)Google Scholar
  41. 41.
    Messner, M., Garrison, B.: Study shows some blogs affect traditional news media agendas. Newsp. Res. J. 32(3), 112–126 (2011)Google Scholar
  42. 42.
    Grimmer, J., Stewart, B.M.: Text as data: the promise and pitfalls of automatic content analysis methods for political texts. Polit. Anal. 21(3), 267–297 (2013)Google Scholar
  43. 43.
    Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980)Google Scholar
  44. 44.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2008)Google Scholar
  45. 45.
    Granger, C.W.: Investigating causal relations by econometric models and cross-spectral methods. Econom. J. Econom. Soc. 424–438 (1969)Google Scholar
  46. 46.
    Lee, S.Y.L., Gholami, R., Tong, T.Y.: Time series analysis in the assessment of ICT impact at the aggregate level–lessons and implications for the new economy. Inf. Manag. 42(7), 1009–1022 (2005)Google Scholar
  47. 47.
    Luo, X., Zhang, J., Duan, W.: Social media and firm equity value. Inf. Syst. Res. 24(1), 146–163 (2013)Google Scholar
  48. 48.
    Dutta, A.: Telecommunications and economic activity: an analysis of Granger causality. J. Manag. Inf. Syst. 17(4), 71–95 (2001)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.The University of Hong KongLung Fu ShanHong Kong SAR

Personalised recommendations