A Maximum Likelihood Method for Estimating Performance in a Rapid Serial Visual Presentation Target-Detection Task
In human-agent teams, communications are frequently limited by how quickly the human component can deliver information to the computer-based agents. Treating the human as a sensor can help relax this limitation. As an instance of this, the rapid serial visual presentation target-detection paradigm provides a fast lane for human target-detection information; however, estimating target-detection performance can be challenging when the inter-stimulus interval is short, relative to human response time variability. This difficulty stems from the uncertainty in assigning each response to the correct stimulus image. We developed a maximum likelihood method to estimate the hit rate and false alarm rate that generally outperforms classic heuristic-based approaches and our previously developed regression-based method. Simulations show that this new method provides unbiased and accurate estimates of target-detection performance across a range of true hit rate and false alarm rate values. In light of the improved estimation of hit rates and false alarm rates, this maximum likelihood method would seem the best choice for estimating human target-detection performance.
KeywordsRSVP Hit rate False-alarm rate Response time Simulation
- 2.Wang, D., et al.: Using humans as sensors: an estimation-theoretic perspective. In: Proceedings of the 13th International Symposium on Information Processing in Sensor Networks, pp. 35–46 (2014)Google Scholar
- 3.Bohannon, A.W., Waytowich, N.R., Lawhern, V.J., Sadler, B.M., Lance, B.J.: Collaborative image triage with humans and computer vision. In: IEEE Systems Man and Cybernetics, pp. 004046–004051 (2016)Google Scholar
- 4.Mathan, S., Ververs, P., Dorneich, M., Whitlow, S., Carciofini, J., Erdogmus, D., et al.: Neurotechnology for image analysis: searching for needles in haystacks efficiently. In: Augmented Cognition: Past, Present and Future (2006)Google Scholar
- 9.Marathe, A.R., Lance, B.J., Nothwang, W., Metcalfe, J.S., McDowell, K.: Confidence metrics improve human-autonomy integration. In: Proceedings of 9th ACM/IEEE International Conference on Human-Robot Interaction (2014a)Google Scholar