Advertisement

BMC Neuroscience

, 16:P278 | Cite as

Developing and validating an isotrigon texture discrimination task using Amazon Mechanical Turk

  • John WG Seamons
  • Marconi S Barbosa
  • Jonathan D Victor
  • Dominique Coy
  • Ted Maddess
Open Access
Poster presentation
  • 255 Downloads

Keywords

Spatial Frequency Discrimination Task Human Visual System Natural Image Comparable Quality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The human visual system must employ mechanisms to minimize informational redundancy whilst maintaining that which is behaviorally relevant [1, 2]. Previous research has concentrated on two-point correlations via spatial frequency and orientation tuning. Higher-order correlations are less studied, but they may inform us about cortical functioning [3]. Isotrigon textures can be used to probe the sensitivity of the human visual system as their structure is exclusively due to 4th and higher-order spatial correlations [4]. Although artificially generated, the same features that give isotrigons salience also create salience in natural images [2]. We implemented an isotrigon discrimination task using the crowdsourcing platform Amazon Mechanical Turk (mTurk) [5]. An important secondary aim was to evaluate the suitability of mTurk for visual psychometric studies as very few exist [6].

960 HITs were uploaded to mTurk and 121 naïve subjects participated. Based on data quality, 91% of HITs were retained at a cost of $0.132 AUD per HIT. The mTurk data was compared to two supervised lab datasets. Lab and mTurk performance functions were very similar (Figure 1A) and highly correlated (Figure 1B). Bland-Altman plots were examined and the mean lab/mTurk coefficient of repeatability was 15.5%. Factor analysis was performed on the combined data and 2 principal factors were identified. Previous studies support that the number of mechanisms is less than 10 [7] and more likely 2-4 [8, 9]. The congruence between the lab and mTurk data is striking considering the unsupervised mode of delivery. In conclusion, mTurk is an underutilized platform for visual psychometric research which can produce data of comparable quality to lab samples at reduced cost and increased scale.
Figure 1

1A: Color map of median texture discrimination performance versus isotrigon texture type. Lab datasets DC (84 HITs) and M1-6 (270 HITs). mTurk datasets Live1 (480 HITs) and Live2 (480 HITs). 1B: Pearson's correlation coefficients between datasets (abbreviations as above).

References

  1. 1.
    H Barlow: Redundancy reduction revisited. Network. 2001, 12 (3): 241-253.CrossRefGoogle Scholar
  2. 2.
    G Tkacik, JS Prentice, JD Victor, V Balasubramanian: Local statistics in natural scenes predict the saliency of synthetic textures. Proc Natl Acad Sci U S A. 2010, 107 (42): 18149-18154.CrossRefGoogle Scholar
  3. 3.
    JD Victor: Isodipole Textures: A Window on Cortical Mechanisms of Form Processing. Early Vision and Beyond. Edited by: Papathomas TVC, C.; Gorea, A; Kowler, E. 1995, MIT Press, 99-107.Google Scholar
  4. 4.
    T Maddess, Y Nagai, AC James, A Ankiewcz: Binary and ternary textures containing higher-order spatial correlations. Vision Res. 2004, 44 (11): 1093-1113.CrossRefGoogle Scholar
  5. 5.
    W Mason, S Suri: Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods. 2012, 44 (1): 1-23.CrossRefGoogle Scholar
  6. 6.
    J Freeman, CM Ziemba, DJ Heeger, EP Simoncelli, JA Movshon: A functional and perceptual signature of the second visual area in primates. Nature neuroscience. 2013, 16 (7): 974-981.CrossRefGoogle Scholar
  7. 7.
    RR Taylor, T Maddess, Y Nagai: Spatial biases and computational constraints on the encoding of complex local image structure. J Vis. 2008, 8 (7): 19 11-13.Google Scholar
  8. 8.
    T Maddess, Y Nagai: Discriminating isotrigon textures [corrected]. Vision Res. 2001, 41 (28): 3837-3860.CrossRefGoogle Scholar
  9. 9.
    T Maddess, Y Nagai, JD Victor, RR Taylor: Multilevel isotrigon textures. J Opt Soc Am A Opt Image Sci Vis. 2007, 24 (2): 278-293.CrossRefGoogle Scholar

Copyright information

© Seamons et al. 2015

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  • John WG Seamons
    • 1
  • Marconi S Barbosa
    • 1
  • Jonathan D Victor
    • 2
  • Dominique Coy
    • 1
  • Ted Maddess
    • 1
  1. 1.Eccles Institute for NeuroscienceJohn Curtin School of Medical Research, ANUCanberraAustralia
  2. 2.Department of Neurology & NeuroscienceWeill Cornell Medical CollegeNew YorkUSA

Personalised recommendations