Abstract
We present a web-based tool for evaluating the information architecture of a website. The tool allows the use of crowdsourcing platforms like Amazon’s MTurk as a means for recruiting test persons, and to conduct asynchronous remote navigation stress tests (cf. Instone 2000). We also report on an evaluation study which compares our tool-based crowdsourced approach to a more traditional laboratory test setting. Results of this comparison indicate that although there are interesting differences between the two testing approaches, both lead to similar test results.
Keywords
References
Amazon Mechanical Turk: Requester User Interface Guide (2008), http://s3.amazonaws.com/awsdocs/MechTurk/latest/amt-ui.pdf (last accessed on February 28, 2013)
Bolt, N., Tulathimutte, T.: Remote Research. Rosenfeld Media, New York (2010)
Bussolon, S., Del Missier, F., Russi, B.: Online card sorting: as good as the paper version. In: Proceedings of the 13th European Conference on Cognitive Ergonomics (ECCE), Zürich (2006), http://www.oat.ethz.ch/news/presentations/bussolon (last accessed on February 28, 2013)
Bruun, A., Gull, P., Hofmeister, L., Stage, J.: Let your users do the testing: a comparison of three remote asynchronous usability testing methods. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI 2009), pp. 1619–1628. ACM, New York (2009), http://doi.acm.org/10.1145/1518701.1518948 (last accessed on February 28, 2013)
Castillo, J.C., Hartson, H.R., Hix, D.: Remote usability evaluation: Can users report their own critical incidents? In: Proceedings of CHI 1998, pp. 253–254. ACM, New York (1998)
Liu, D., Lease, M., Kuipers, R., Bias, R.: Crowdsourcing Usability Testing. In: Computing Research Repository, CoRR (2007), http://arxiv.org/abs/1203.1468
Franco, S., Herbstritt, S., Johnson, E., Schumacher, S., Van Zandt, L.: Assessing Users’ Needs for Usability on the Job Search Features of WorkInTexas.com (2010), http://susieherbstritt.com/downloads/witcomux.pdf (last accessed on February 28, 2013)
Hartson, H.R., Castillo, J.C., Kelso, J., Neale, W.C.: Remote evaluation: the network as an extension of the usability laboratory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Common Ground (CHI 1996), pp. 228–235. ACM, New York (1996)
Instone, K.: Navigations Stress Test (2000), http://instone.org/navstress (last accessed on February 28, 2013)
Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with Mechanical Turk. In: Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (CHI 2008), pp. 453–456. ACM, New York (2008)
Morville, P., Rosenfeld, L.: Information Architecture for the World Wide Web. Designing Large-Scale Web Sites. O’Reilly, Sebastopol (2006)
Nielsen, J.: Discount Usability: 20 Years. Alertbox, September 14 (2009), http://www.useit.com/alertbox/discount-usability.html (last accessed on February 28, 2013)
Sauro, J.: The Methods UX Professionals Use (2011), http://www.measuringusability.com/blog/ux-methods.php (last accessed on February 28, 2013)
Toub, S.: Evaluating Information Architecture. A Practical Guide to Assessing Web Site Organization. Argus Center for Information Architecture Whitepaper (2000), http://argus-acia.com/white_papers/evaluating_ia.pdf (last accessed on February 28, 2013)
Thompson, J.: Investigating the Effectiveness of Applying the Critical Incident Technique to Remote Usability Evaluation. Master thesis, Virginia Polytechnic Institute and State University (1999), http://scholar.lib.vt.edu/theses/available/etd-121699-205449/unrestricted/thesis.pdf (last accessed on February 28, 2013)
W3C. W3C Working Draft: HTML5. A vocabulary and associated APIs for HTML and XHTML (2012), http://www.w3.org/TR/html5/the-canvas-element.html#the-canvas-element (last accessed on February 28, 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Meier, F., Bazo, A., Burghardt, M., Wolff, C. (2013). Evaluating a Web-Based Tool for Crowdsourced Navigation Stress Tests. In: Marcus, A. (eds) Design, User Experience, and Usability. Web, Mobile, and Product Design. DUXU 2013. Lecture Notes in Computer Science, vol 8015. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39253-5_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-39253-5_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39252-8
Online ISBN: 978-3-642-39253-5
eBook Packages: Computer ScienceComputer Science (R0)