Using Apache Spark for Modeling Student Behavior at Scale

Lewkow, Nicholas; Feild, Jacqueline

doi:10.1007/978-981-13-0650-1_9

Nicholas Lewkow¹⁵ &
Jacqueline Feild¹⁵

Part of the book series: Lecture Notes in Educational Technology ((LNET))

478 Accesses

Abstract

This chapter is an introduction to parallel processing with education data. As the amount of education data continues to grow, new methods for processing this data efficiently are required. This chapter gives a history of popular parallel computing frameworks and discusses problem types that are easily mapped to these frameworks. Following that, an example machine-learning problem is described and a single-threaded and parallel pipeline using Apache Spark are compared. We hope this information can be used by other practitioners looking to utilize Apache Spark to expand their models to include more students and more data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Chapter: USD 29.95; Price excludes VAT (USA)

eBook: USD 139.00; Price excludes VAT (USA)

Softcover Book: USD 179.99; Price excludes VAT (USA)

Hardcover Book: USD 179.99; Price excludes VAT (USA)

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

Ghemawat, S., Gobioff, H., & Leung, S. (2003). The Google file system. In ACM SIGOPS operating systems review (Vol. 37, No. 5). ACM.
Google Scholar
O’Malley, O. (2008). Tera2byte sort on Apache Hadoop. http://sortbenchmark.org. Cited October 31, 2017.
Pappano, L. (2012). The year of the MOOC. The New York Times. http://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html. Cited October 31, 2017.
Shah, D. (2016). By the numbers: MOOCS in 2016. In Class central. https://www.class-central.com/report/mooc-stats-2016. Cited October 31, 2017.
Xin, R. (2014). Apache spark the fastest open source engine for sorting a petabyte. In Databricks engineering blog. https://databricks.com/blog/2014/10/10/spark-petabyte-sort.html. Cited October 31, 2017.

Download references

Author information

Authors and Affiliations

McGraw-Hill Education, 281 Summer Street, Boston, MA, 02210, USA
Nicholas Lewkow & Jacqueline Feild

Authors

Nicholas Lewkow
View author publications
You can also search for this author in PubMed Google Scholar
Jacqueline Feild
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicholas Lewkow .

Editor information

Editors and Affiliations

Department of Learning Technologies, University of North Texas, Denton, TX, USA
J. Michael Spector
School of Computing and Information System, Athabasca University, Athabasca, Canada
Vivekanandan Kumar
Analytics and R&D, McGraw-Hill Education, Boston, MA, USA
Alfred Essa
Department of Engineering Science, National Cheng Kung University, Tainan, Taiwan
Yueh-Min Huang
Open University in the Netherlands, Heerlen, The Netherlands
Rob Koper
School of Computing, University of Eastern Finland, Joensuu, Finland
Richard A. W. Tortorella
Smart Learning Institute, Beijing Normal University, Beijing, China
Ting-Wen Chang
School of Educational Technology, Beijing Normal University, Beijing, China
Yanyan Li
School of Educational Technology, Beijing Normal University, Beijing, China
Zhizhen Zhang

Rights and permissions

Reprints and permissions

Copyright information

About this chapter

Cite this chapter

Lewkow, N., Feild, J. (2018). Using Apache Spark for Modeling Student Behavior at Scale. In: Spector, J., et al. Frontiers of Cyberlearning. Lecture Notes in Educational Technology. Springer, Singapore. https://doi.org/10.1007/978-981-13-0650-1_9

Download citation

DOI: https://doi.org/10.1007/978-981-13-0650-1_9
Published: 04 November 2018
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-0649-5
Online ISBN: 978-981-13-0650-1
eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics