Skip to main content
SpringerLink
Log in

Theme Analyses for Open-Ended Survey Responses in Education Research on Summer Melt Phenomenon

  • Conference paper
  • First Online:
Advances in Quantitative Ethnography (ICQE 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1112))

Included in the following conference series:

Abstract

Summer melt is a phenomenon when college-intending students fail to enroll in the fall after high school graduation. Previous research on summer melt utilized surveys, typically consisting of Likert scale questions and open-ended response questions. Open-ended responses can elicit more information from students, but they have not been fully analyzed due to the cost, time, and complexity of theme extraction with manual coding. In the present study, we applied the topic modeling approach to extract topics and relevant themes, and evaluated model performance by comparing model-generated topics and categories with the human-identified topics and themes. Results showed that the topic model allows for extracting similar topics as the survey questions that were investigated, but only extracted part of the themes classified by the human. Discussion and implications focus on potential improvements in automated topic and theme classification from open-ended survey responses.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bettinger, E.P., Long, B.T., Oreopoulos, P., Sanbonmatsu, L.: The role of application assistance and information in college decisions: results from the H&R Block FAFSA experiment. Q. J. Econ. 127, 1205–1242 (2012)

    Article  Google Scholar 

  2. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3(1), 993–1022 (2003)

    MATH  Google Scholar 

  3. Bobek, B.L., Zhao, R.: Education and career navigation. In: Camara, W., O’Connor, R., Mattern, K., Hanson, M.A. (eds.) Beyond Academics: A Holistic Framework for Enhancing Education and Workplace Success, pp. 39–51. ACT Inc, Iowa City (2015)

    Google Scholar 

  4. Castleman, B.L., Page, L.C.: A trickle or a torrent? Understanding the extent of summer melt among college-intending high school graduates. Soc. Sci. Q. 95(1), 202–220 (2014)

    Article  Google Scholar 

  5. Castleman, B.L., Page, L.C., Snowdon, A.L.: Strategic data project summer melt handbook: a guide to investigating and responding to Summer Melt. Center for Education Policy Research, Harvard University (2013)

    Google Scholar 

  6. Li, H., Zhou, Y., Bobek, B.L.: Automated identification of open-ended survey response themes in education research: a summer melt study. Poster presented at Twenty-Ninth Annual Meeting of the Society for Text & Discourse, New York, NY (2019)

    Google Scholar 

  7. Lucas, C., Nielsen, R.A., Roberts, M.E., Stewart, B.M., Storer, A., Tingley, D.: Computer-assisted text analysis for comparative politics. Polit. Anal. 23(2), 254–277 (2015)

    Article  Google Scholar 

  8. McCallum, A.K.: MALLET: a machine learning for language toolkit (2002). http://mallet.cs.umass.edu

  9. Merriam, S.B., Tisdell, E.J.: Qualitative Research: A Guide to Design and Implementation. Wiley, Hoboken (2015)

    Google Scholar 

  10. Oreopoulos, P., Dunn, R.: Information and college access: evidence from a randomized field experiment. Scand. J. Econ. 115, 3–26 (2013)

    Article  Google Scholar 

  11. Paek, P.L., Bobek, B.L.: Unpacking the factors contributing to summer melt and impacting college readiness. Paper presented at the Annual Conference for the American Research Education Association (AERA), New York, NY (2018)

    Google Scholar 

  12. Roberts, M.E., et al.: Structural topic models for open-ended survey responses. Am. J. Polit. Sci. 58(4), 1064–1082 (2014)

    Article  Google Scholar 

  13. Roberts, M.E., Stewart, B.M., Airoldi, E.M.: A model of text for experimentation in the social sciences. J. Am. Stat. Assoc. 111(515), 988–1003 (2016)

    Article  MathSciNet  Google Scholar 

  14. Shaffer, D.W.: Quantitative Ethnography. Cathcart Press, Madison (2017)

    Google Scholar 

  15. Shum, S.B.: Book review: quantitative ethnography by David Williamson Shaffer. J. Learn. Anal. 6(1), 99–101 (2019)

    Article  MathSciNet  Google Scholar 

  16. Trostel, P., Smith, M.C.: It’s Not Just the Money: The Benefits of College Education to Individuals and to Society. Lumina Foundation, Indianapolis (2015)

    Google Scholar 

  17. Vandenbroucke, G.: Lifetime benefits of an education have never been so high. Reg. Econ. 25(1), 10–11 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ACT, Inc., Iowa City, IA, 52243, USA

    Haiying Li, Joyce Zhou-Yile Schnieders & Becky L. Bobek

Authors
  1. Haiying Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joyce Zhou-Yile Schnieders
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Becky L. Bobek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiying Li .

Editor information

Editors and Affiliations

  1. University of Wisconsin–Madison, Madison, WI, USA

    Brendan Eagan

  2. University of Copenhagen, Copenhagen, Denmark

    Morten Misfeldt

  3. University of Wisconsin–Madison, Madison, WI, USA

    Amanda Siebert-Evenstone

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, H., Schnieders, J.ZY., Bobek, B.L. (2019). Theme Analyses for Open-Ended Survey Responses in Education Research on Summer Melt Phenomenon. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds) Advances in Quantitative Ethnography. ICQE 2019. Communications in Computer and Information Science, vol 1112. Springer, Cham. https://doi.org/10.1007/978-3-030-33232-7_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33232-7_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33231-0

  • Online ISBN: 978-3-030-33232-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation