Abstract
In a massive online course with hundreds of thousands of students, it is unfeasible to provide an accurate and fast evaluation for each submission. Currently the researchers have proposed the algorithms called peer grading for the richly-structured assignments. These algorithms can deliver fairly accurate evaluations through aggregation of peer grading results, but not improve the effectiveness of allocating submissions. Allocating submissions to peers is an important step before the process of peer grading. In this paper, being inspired from the Longest Processing Time (LPT) algorithm that is often used in the parallel system, we propose a Modified Longest Processing Time (MLPT), which can improve the allocation efficiency. The dataset used in this paper consists of two parts, one part is collected from our MOOCs platform, and the other one is manually generated as the simulation dataset. We have shown the experimental results to validate the effectiveness of MLPT based on the two type datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fonteles, A.S., Bouveret, S., Gensel, J.: Heuristics for task recommendation in spatiotemporal crowdsourcing systems. In: Proceedings of the 13th International Conference on Advances in Mobile Computing and Multimedia, pp. 1–5. ACM (2015)
Cheng, J., Teevan, J., Bernstein, M.S.: Measuring crowdsourcing effort with error-time curves. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1365–1374. ACM (2015)
Qiu, C., Squicciarini, A.C., Carminati, B., et al.: CrowdSelect: increasing accuracy of crowdsourcing tasks through behavior prediction and user selection. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 539–548. ACM (2016)
Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)
Wiki for Multiprocessor Scheduling Information. https://en.wikipedia.org/wiki/Multiprocessor_scheduling
Piech, C., Huang, J., Chen, Z., et al.: Tuned models of peer assessment in MOOCs. arXiv preprint arXiv:1307.2579 (2013)
Coffman, Jr. E.G., Sethi, R.: A generalized bound on LPT sequencing. In: Proceedings of the 1976 ACM SIGMETRICS Conference on Computer Performance Modeling Measurement and Evaluation, pp. 306–310. ACM (1976)
Alfarrarjeh, A., Emrich, T., Shahabi, C.: Scalable spatial crowdsourcing: a study of distributed algorithms. In: 2015 16th IEEE International Conference on Mobile Data Management, vol. 1, pp. 134–144. IEEE (2015)
Baneres, D., Caballé, S., Clarisó, R.: Towards a learning analytics support for intelligent tutoring systems on MOOC platforms. In: 2016 10th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS), pp. 103–110. IEEE (2016)
Gonzalez, T., Ibarra, O.H., Sahni, S.: Bounds for LPT schedules on uniform processors. SIAM J. Comput. 6(1), 155–166 (1977)
Mi, F., Yeung, D.Y.: Probabilistic graphical models for boosting cardinal and ordinal peer grading in MOOCs. In: AAAI, pp. 454–460 (2015)
Massabò, I., Paletta, G., Ruiz-Torres, A.J.: A note on longest processing time algorithms for the two uniform parallel machine makespan minimization problem. J. Sched. 19(2), 207–211 (2016)
Gardner, K., Zbarsky, S., Harchol-Balter, M., et al.: The power of d choices for redundancy. ACM SIGMETRICS Perform. Eval. Rev. 44(1), 409–410 (2016)
Feier, M.C., Lemnaru, C., Potolea, R.: Solving NP-complete problems on the CUDA architecture using genetic algorithms. In: International Symposium on Parallel and Distributed Computing, ISPDC 2011, Cluj-Napoca, Romania, pp. 278–281. DBLP, July 2011
Ul, Hassan U., Curry, E.: Efficient task assignment for spatial crowdsourcing. Expert Syst. Appl: Int. J. 58(C), 36–56 (2016)
Jung, H.J., Lease, M.: Crowdsourced task routing via matrix factorization. Eprint Arxiv (2013)
Karger, D.R., Oh, S., Shah, D.: Budget-optimal crowdsourcing using low-rank matrix approximations. In: 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 284–291. IEEE (2011)
Yan, Y., Fung, G.M., Rosales, R., et al.: Active learning from crowds. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011), pp. 1161–1168 (2011)
Tong, Y., She, J., Ding, B., et al.: Online minimum matching in real-time spatial data: experiments and analysis. Proc. VLDB Endow. (PVLDB) 9(12), 1053–1064 (2016)
Tong, Y., She, J., Ding, B., et al.: Online mobile micro-task allocation in spatial crowdsourcing. In: Proceedings of the 32nd International Conference on Data Engineering (ICDE 2016), pp. 49–60 (2016)
Tong, Y., She, J., Meng, R.: Bottleneck-aware arrangement over event-based social networks: the max-min approach. World Wide Web J. 19(6), 1151–1177 (2016)
She, J., Tong, Y., Chen, L., et al.: Conflict-aware event-participant arrangement and its variant for online setting. IEEE Trans. Knowl. Data Eng. (TKDE) 28(9), 2281–2295 (2016)
She, J., Tong, Y., Chen, L., et al.: Conflict-aware event-participant arrangement. In: Proceedings of the 31st International Conference on Data Engineering (ICDE 2015), pp. 735–746 (2015)
Acknowledgments
This work was supported in part by grant from State Key Laboratory of Software Development Environment (Funding No. SKLSDE-2015ZX-03) and NSFC (Grant No. 61532004).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Han, Y., Wu, W., Pu, Y. (2017). Task Assignment of Peer Grading in MOOCs. In: Bao, Z., Trajcevski, G., Chang, L., Hua, W. (eds) Database Systems for Advanced Applications. DASFAA 2017. Lecture Notes in Computer Science(), vol 10179. Springer, Cham. https://doi.org/10.1007/978-3-319-55705-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-55705-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55704-5
Online ISBN: 978-3-319-55705-2
eBook Packages: Computer ScienceComputer Science (R0)