Abstract
Critical thinking and reasoning are essential for making informed judgments. This can be especially important for the intelligence community and associated government agencies. In this paper, we explore methods to evaluate critical thinking and reasoning. We focus on a public opinion/argument-based website - Yourview. We introduce the Yourview platform and present the annotated Yourview dataset that was created following a pilot period that focused on collecting public opinion on a range of topics. We then propose a method to classify arguments and their components related to the comments in the Yourview dataset. We assess the influence of components of argumentation as the basis for critical thinking and subsequently score and visualize these relations. Building on this, we predict critical thinking scores for what makes a good argument using a multilayer perceptron (MLP). The results of these models help enhance reasoning and establishment of knowledge from persuasive texts.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Youyou, W., Kosinski, M., Stillwell, D.: Computer-based personality judgments are more accurate than those made by humans. Proc. Natl. Acad. Sci. 112(4), 1036–1040 (2015)
Movshovitz-Attias, D., Movshovitz-Attias, Y., Steenkiste, P., Faloutsos, C.: Analysis of the reputation system and user contributions on a question answering website: stackoverflow. In: Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 886–893. ACM (2013)
Stab, C., Gurevych, I.: Annotating argument components and relations in persuasive essays, pp. 1501–1510 (2014)
Attali, Y., Burstein, J.: Automated Essay Scoring with E-rater® v. 2.0. ETS Research Report Series, vol. 2004, no. 2 (2004)
Feng, V.W., Lin, Z., Hirst, G., Holdings, S.P.: The impact of deep hierarchical discourse structures in the evaluation of text coherence. In: COLING, pp. 940–949 (2014)
Persing, I., Ng, V.L.: End-to-end argumentation mining in student essays. In: Proceedings of Human Language Technologies: The 2016 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1384–1394 (2016)
Habernal, I., Gurevych, I.: Which argument is more convincing? Analyzing and predicting convincingness of web arguments using bidirectional LSTM. In: ACL, no. 1 (2016)
Abrami, P.C., Bernard, R.M., Borokhovski, E., Waddington, D.I., Wade, C.A., Persson, T.: Strategies for teaching students to think critically: a meta-analysis. Rev. Educ. Res. 85(2), 275–314 (2015)
van Gelder, T.: Cultivating deliberation for democracy. J. Public Delib. 8(1) (2012)
Moens, M.-F., Boiy, E., Palau, R.M., Reed, C.: Automatic detection of arguments in legal texts. In: Proceedings of the 11th International Conference on Artificial Intelligence and Law, pp. 225–230. ACM (2007)
Rooney, N., Wang, H., Browne, F.: Applying kernel methods to argumentation mining. In: FLAIRS Conference (2012)
Mochales, R., Moens, M.-F.: Study on the structure of argumentation in case law. In: Proceedings of the 2008 Conference on Legal Knowledge and Information Systems, pp. 11–20 (2008)
Lin, Z., Kan, M.-Y., Ng, H.T.: Recognizing implicit discourse relations in the Penn Discourse Treebank. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1, pp. 343–351. Association for Computational Linguistics (2009)
Louis, A., Joshi, A., Prasad, R., Nenkova, A.: Using entity features to classify implicit discourse relations. In: Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Association for Computational Linguistics, pp. 59–62 (2010)
Peldszus, A., Stede, M.: Joint prediction in MST-style discourse parsing for argumentation mining. In: EMNLP, vol. 2015, pp. 938–948 (2015)
Zesch, T., Wojatzki, M., Scholten-Akoun, D.: Task-independent features for automated essay grading. In: BEA@ NAACL-HLT, pp. 224–232 (2015)
Persing, I., Ng, V.: Modeling prompt adherence in student essays. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: Volume 1: Long Papers, pp. 1534–1543 (2014)
Stab, C., Gurevych, I.: Recognizing insufficiently supported arguments in argumentative essays. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, vol. 1, pp. 980–990 (2017)
Habernal, I., Gurevych, I.: What makes a convincing argument? Empirical analysis and detecting attributes of convincingness in web argumentation. In: EMNLP, pp. 1214–1223 (2016)
Wachsmuth, H., Naderi, N., Hou, Y., Bilu, Y., Prabhakaran, V., Thijm, T.A., Hirst, G., Stein, B.: Computational argumentation quality assessment in natural language. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 1, pp. 176–187 (2017)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(Jul), 2121–2159 (2011)
Zeiler, M.D.: Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Mahana, M., Johns, M., Apte, A.: Automated essay grading using machine learning. Machine Learning Session, Stanford University (2012)
Acknowledgements
The authors would like to thank the SWARM team and especially Tim van Gelder for providing access to the Yourview data set. This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research projects Activity (IARPA), under Contract [2017-16122000002]. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Kang, K., Sinnott, R.O. (2018). Improving Online Argumentation Through Deep Learning. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2018. ICCSA 2018. Lecture Notes in Computer Science(), vol 10960. Springer, Cham. https://doi.org/10.1007/978-3-319-95162-1_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-95162-1_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95161-4
Online ISBN: 978-3-319-95162-1
eBook Packages: Computer ScienceComputer Science (R0)