What Influences Workers’ Performance?

Abstract

The inadequacy of collected data through crowdsourcing microtask jobs was addressed numerously. Different reasons were given, namely task related (ill-designed jobs like given instruction may be misunderstood), worker related (participants who do not focus enough, share their attention with a parallel activity and do not work as instructed or try to maximize their monetary benefit with minimum effort), and environment related (workers may be interrupted or use inadequate equipment) [77, 82, 83, 93]. To reduce their impact, different quality control approaches were proposed such as using gold standards, majority voting, and behavioral logging to evaluate reliability of the collected data in post-processing [17, 31]. In the context of QoE, researchers have used additional methods like content questions and consistency tests [46, 47].