A scientific writing pedagogy and mixed methods assessment for engineering education using open-coding and multi-dimensional scaling
Abstract
Assessments of new pedagogical practices usually rely on instructor oriented surveys and questionnaires to measure student perceptions of teaching methods; however, fixed response categories in structured questionnaires might bias results. This paper demonstrates a mixed methods approach using open and multi-dimensional scaling (MDS) for a student-oriented exploratory analysis and visualization of perceptions of teaching methods. A scientific writing and research methodology course is a required course for first year PhD students in software engineering and geo-infomatics at Wuhan University, China. PhD students attending this course came from countries whose first language is not English, and from a variety of software engineering and geo-infomatics domains. The problem therefore, was to elicit un-mediated perceptions of course assignment, reduce and generalize the resulting data for interpretation. A graphical visualization of themes emerging in student responses to two open-ended questions about an assignment provided a basis for inferring student interests and needs. In this course, an assessment of a task oriented problem-solving experience was implemented through a mixed method strategy incorporating qualitative methods, exploratory data mining techniques, and cartographic visualization. The visualization shows that participating students generally perceived the exercise as challenging, helped them understand journal requirements, and develop ways to survey texts to extract information. The results also suggested that this consensus breaks down in terms of each participant’s own goals, domain, and research interests. Unstructured questions, open coding, and MDS visualization might also prove to be helpful in the process of devising and assessing other student centered pedagogies.
Keywords
Scientific writing Engineering education Assessment Multi-dimensional scaling Open codingNotes
Acknowledgements
Funding was provided by The International Society for Photogrammetry and Remote Sensing (ISPRS): Education and Capacity Building Initiatives 2018 (Grant No. TC I).
References
- Al Bahadly, I. (2006). Team learning and multiple assessments in engineering courses. In Proceedings of the 5th WSEAS international conference on electronics, hardware, wireless and optical communications, Madrid, Spain, February 15–17, pp. 193–198.Google Scholar
- Bitchener, J., & Basturkmen, H. (2006). Perceptions of the difficulties of postgraduate l2 thesis students writing the discussion section. Journal of English for Academic Purposes, 5(1), 4–18.CrossRefGoogle Scholar
- Chang, Y.-S. (2013). Student technological creativity using online problem-solving activities. International Journal of Technology and Design Education, 23(3), 803–816.CrossRefGoogle Scholar
- Commandeur, J. J. F., & Heiser, W. J. (1993). Mathematical derivations in the proximity scaling (PROXSCAL) of symmetric data matrices. Leiden: Department of Data Theory, University of Leiden.Google Scholar
- Conrad, S. (2017). A comparison of practitioner and student writing in civil engineering. Journal of Engineering Education, 106(2), 191–217.CrossRefGoogle Scholar
- Coxon, A. P. M. (2005). Integrating qualitative and quantitative data: What does the user need? Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 6(2), Art. 40, http://nbn-resolving.de/urn:nbn:de:0114-fqs0502402.
- Creswell, J. W. (1994). Pearson educational research planning conducting and evaluating quantitative and qualitative research person new international edition plus my education lab without eText. International Journal for Numerical and Analytical Methods in Geomechanics, 18(1), 1–24.CrossRefGoogle Scholar
- Criscuolo, C., & Martin, R. (2004). An emerging knowledge-based economy in China? Indicators from OECD databases, OECD Science, Technology and Industry Working Papers, 2004/4, OECD Publishing. https://doi.org/10.1787/256502026705.
- Dal, M. (2013). Teaching electric drives control course: Incorporation of active learning into the classroom. IEEE Transactions on Education, 56(4), 459–469.CrossRefGoogle Scholar
- Denzin, N., & Lincoln, Y. (2003). The landscape of qualitative research (p. 133). London: Sage.Google Scholar
- Freire, P. (1970). Adult literacy process as cultural action for freedom. Harvard Educational Review, 40(2), 205–225.CrossRefGoogle Scholar
- Gweon, G., et al. (2017). Towards effective group work assessment: Even what you don’t see can bias you. International Journal of Technology and Design Education, 27, 1–16.CrossRefGoogle Scholar
- Hartley, J. (2014). Some thoughts on Likert-type scales. International Journal of Clinical and Health Psychology, 14(1), 83–86.CrossRefGoogle Scholar
- Hess, N., & Ghawi, M. (1997). English for academic purposes: Teacher development in a demanding arena. English for Specific Purposes, 16(1), 15–26.CrossRefGoogle Scholar
- Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3–20.CrossRefGoogle Scholar
- Kruskal, J. B. (1964). Nonmetric multidimensional scaling: A numerical method. Psychometrika, 29(2), 115–129.CrossRefGoogle Scholar
- Kuebel, C. R., Koops, L. H., & Bond, V. L. (2018). Cultivating teachers of general music methods: The graduate years. Journal of Music Teacher Education, 28(1), 10–23.CrossRefGoogle Scholar
- Limniou, M., & Smith, M. (2014). The role of feedback in e-assessments for engineering education. Education and Information Technologies, 19(1), 209–225.CrossRefGoogle Scholar
- Liu, F., Wang, L., & Qian, Y. (2017). Analysis of MOOCs courses dropout rate based on students' studying behaviors. Advances in Social Science Education and Humanities Research, 83, 139–144.Google Scholar
- Shepard, R. N. (1962a). The analysis of proximities: Multidimensional scaling with an unknown distance function. I. Psychometrika, 27(2), 125–140.CrossRefGoogle Scholar
- Shepard, R. N. (1962b). The analysis of proximities: Multidimensional scaling with an unknown distance function. II. Psychometrika, 27(3), 219–246.CrossRefGoogle Scholar
- Singh, M., & Fu, D. (2008). Flowery inductive rhetoric meets creative deductive arguments becoming transnational researcher-writers. International Journal of Asia-Pacific Studies, 4(1), 121–137.Google Scholar
- Tajino, A. (1997). Learner difficulty: What is it, and how well do we understand it? The Teacher Trainer, 11(2), 12–14.Google Scholar
- Tao, J., et al. (2015). Extending engineering specialty course concepts in electrical engineering education. International Journal of Electrical Engineering Education, 52(1), 39–51.CrossRefGoogle Scholar
- Tao, J., et al. (2016). Integrated pedagogy for specialty courses in chinese engineering education. International Journal of Engineering Education, 32(5B), 2284–2293.Google Scholar
- Tomasek, T. (2009). Critical reading: Using reading prompts to promote active engagement with text. International Journal of Teaching and Learning in Higher Education, 21(1), 127–132.Google Scholar
- Torregosa, M., Ynalvez, M. A., Schiffman, R., & Morin, K. (2014). English language proficiency, academic networks, and academic performance of Mexican–American baccalaureate nursing students. Nursing Education Perspectives, 36, 140502140546007. https://doi.org/10.5480/13-1136.1.Google Scholar
- Weaver, M. G., Samoshin, A. V., & Lewis, R. B. (2016). Developing students’ critical thinking, problem solving, and analysis skills in an inquiry-based synthetic organic laboratory course. Journal of Chemical Education, 93(5), 847–851.CrossRefGoogle Scholar
- Young, R. F., & Miller, E. R. (2004). Learning as changing participation: Discourse roles in ESL writing conferences. Modern Language Journal, 88(4), 519–535.CrossRefGoogle Scholar