sdtlu: An R package for the signal detection analysis of eyewitness lineup data

Abstract

In a standard eyewitness lineup scenario, a witness observes a culprit commit a crime and is later asked to identify the culprit from a set of faces, the lineup. Signal detection theory (SDT), a powerful modeling framework for analyzing data, has recently become a common way to analyze lineup data. The goal of this paper is to introduce a new R package, sdtlu (Signal Detection Theory – LineUp), that streamlines and automates the SDT analysis of lineup data. sdtlu provides functions to process lineup data, determine the best-fitting SDT parameters, compute model-based performance measures such as area under the curve (AUC) and diagnosticity, use bootstrapping to determine uncertainty intervals around these parameters and measures, and compare parameters across two different data sets. The package incorporates closed-form solutions for both simultaneous and sequential lineups that allow for model-based analyses without Monte Carlo simulation. Show-ups are also supported. The package can estimate the base-rate of lineups that include a guilty suspect when the guilt or innocence of each suspect in the data set is unknown, as in “real-world” lineups. The package can also produce a full set of graphs, including data and model-based ROC curves and the underlying SDT model.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Notes

  1. 1.

    We recognize that suspect position can influence identification in both simultaneous and sequential lineups (e.g., Palmer, Sauer, & Holt, 2017). Whereas the sequential model naturally incorporates position effects, the baseline simultaneous model does not, and doing so introduces a host of difficulties because the witness can consider the faces in any order and can return to previously considered faces. To avoid the additional model complexity involved in incorporating position effects into the simultaneous model, we leave that change for future work.

  2. 2.

    Confidence ratings for rejections are not used because it is unclear how one should make these confidence ratings and because confidence is often fairly unsystematic for rejections in data sets.

  3. 3.

    The G2 method provides maximum likelihood estimates of parameter values, as G2 is determined by the likelihood of the data under a given parameter set in the model of interest (the "research" model) and the likelihood of the data in a "full" model in which the probability of each response category matches its proportion in the data. The latter value does not change for different parameter sets, so G2 differences across parameter sets are determined only by the likelihood of the "research" model.

  4. 4.

    Although this should technically be sigma_l, we use sigma_f to avoid confusion with sigma_1.

  5. 5.

    Trial-by-trial data are not used. The data and predictions are collapsed over suspect positions.

  6. 6.

    Note that Gronlund et al. (2009) did not rely on SDT modeling to interpret their data, so noting that their paradigm violated the model’s assumptions is not a criticism of these authors and does not undermine the original purpose of their study.

  7. 7.

    Note that the package code implements the joint, rather than conditional probabilities, for example, P(resp = susconf = i) = P(resp = susconf = itar = pres) + P(resp = susconf = itar = abs), P(resp = susconf = itar = pres) = pP(resp = susconf = i| tar = pres), and P(resp = susconf = itar = abs) = (1 − p)P(resp = susconf = i| tar = abs). For ease of exposition, the conditionals are presented here, but the results are equivalent. The same transformation holds for all equations below.

  8. 8.

    Note that the package code implements the joint, rather than conditional probabilities. See FN 7.

References

  1. Bothwell, R. K., Deffenbacher, K. A., Brigham, J. C. (1987). Correlation of eyewitness accuracy and confidence: Optimality hypothesis revisited. Journal of Applied Psychology, 72, 691–695.

    Article  Google Scholar 

  2. Brewer, N., & Burke, A. (2002). Effects of testimonial inconsistencies and eyewitness confidence on mock-juror judgments. Law and Human Behavior, 26, 353–364.

    Article  Google Scholar 

  3. Carlson, C. A., & Carlson, M. A. (2014). An evaluation of lineup presentation, weapon presence, and a distinctive feature using ROC analysis. Journal of Applied Research in Memory and Cognition, 3, 45–53.

    Article  Google Scholar 

  4. Cohen, A.L., Starns, J.J., Rotello, C.M., & Cataldo, A.M. (2020). Estimating the proportion of guilty suspects and posterior probability of guilt in lineups using signal-detection models. Cognitive Research: Principles and Implications, 5, 21. https://doi.org/10.1186/s41235-020-00219-4

  5. Colloff, M. F., Wade, K. A., Wixted, J. T., & Maylor, E. A. (2017). A signal-detection analysis of eyewitness identification across the adult lifespan. Psychology and Aging, 32, 243–258. DOI: https://doi.org/10.1037/pag0000168

    Article  PubMed  Google Scholar 

  6. Cutler, B. L., Penrod, S. D., & Dexter, H. R. (1990). Juror sensitivity to eyewitness identification evidence. Law and Human Behavior, 14, 185–191.

    Article  Google Scholar 

  7. Deffenbacher, K. A. (1980). Eyewitness accuracy and confidence: Can we infer anything about their relationship? Law and Human Behavior, 4, 243–260. https://doi.org/10.1007/BF01040617

    Article  Google Scholar 

  8. Dobolyi, D. G., & Dodson, C. S. (2013). Eyewitness confidence in simultaneous and sequential lineups: A criterion shift account for sequential mistaken identification overconfidence. Journal of Experimental Psychology: Applied, 19, 345–357.

    PubMed  Google Scholar 

  9. Gonzalez, R., Ellsworth, P. C., & Pembroke, M. (1993). Response biases in lineups and showups. Journal of Personality and Social Psychology, 64, 525–537

    Article  Google Scholar 

  10. Gronlund, S. D., Carlson, C. A., Dailey, S. B., Goodsell, C. A. (2009). Robustness of the sequential lineup advantage. Journal of Experimental Psychology: Applied, 15, 140–152. DOI: https://doi.org/10.1037/a0015082

    Article  PubMed  Google Scholar 

  11. Horry, R., Brewer, N., Weber, N., & Palmer, M. A. (2015). The effects of allowing a second sequential lineup lap on choosing and probative value. Psychology, Public Policy, and Law, 21, 121–133. doi:https://doi.org/10.1037/law0000041

    Article  Google Scholar 

  12. Horry, R., Palmer, M., & Brewer, N. (2012). Backloading in the sequential lineup prevents within-lineup criterion shifts that undermine eyewitness identification performance. Journal of Experimental Psychology: Applied, 18, 346–360.

    PubMed  Google Scholar 

  13. Levi, A. M. (2012). Much better than the sequential lineup: a 120-person lineup. Psychology, Crime & Law, 18, 631–640. DOI: https://doi.org/10.1080/1068316X.2010.526120

    Article  Google Scholar 

  14. Macmillan, N. A., & Creelman, D. C. (2005). Detection Theory: A User’s Guide. New Jersey: Lawrence Erlbaum Associates.

    Google Scholar 

  15. McClish, D. K. (1989). Analyzing a portion of the ROC curve. Medical Decision Making, 9, 190–195.

    Article  Google Scholar 

  16. Mickes, L. (2015). Receiver operating characteristic analysis and confidence–accuracy characteristic analysis in investigations of system variables and estimator variables that affect eyewitness memory. Journal of Applied Research in Memory and Cognition, 4, 93–102. DOI: https://doi.org/10.1016/j.jarmac.2015.01.003

    Article  Google Scholar 

  17. Mickes, L., Flowe, H. D., & Wixted, J. T. (2012). Receiver operating characteristic analysis of eyewitness memory: Comparing the diagnostic accuracy of simultaneous versus sequential lineups. Journal of Experimental Psychology: Applied, 18, 361–376.

    PubMed  Google Scholar 

  18. Palmer, M. A., Brewer, N., Weber, N., & Nagesh, A. (2013). The confidence–accuracy relationship for eyewitness identification decisions: Effects of exposure duration, retention interval, and divided attention. Journal of Experimental Psychology: Applied, 19, 55–71. DOI: https://doi.org/10.1037/a0031602

    Article  PubMed  Google Scholar 

  19. Palmer, M. A., Sauer, J. D., & Holt, G. A. (2017). Undermining position effects in choices from arrays, with implications for police lineups. Journal of experimental psychology: applied, 23(1), 71.

    PubMed  Google Scholar 

  20. Police Executive Research Forum. (2013). A national survey of eyewitness identification procedures in law enforcement agencies. Retrieved from https://www.ncjrs.gov/pdffiles1/nij/grants/242617.pdf

  21. Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) “pROC: an open-source package for R and S+ to analyze and compare ROC curves”. BMC Bioinformatics, 7, 77. DOI: https://doi.org/10.1186/1471-2105-12-77

    Article  Google Scholar 

  22. Rotello, C. M., & Chen, T. (2016). ROC curve analyses of eyewitness identification decisions: An analysis of the recent debate. Cognitive Research: Principles and Implications, 1:10. DOI https://doi.org/10.1186/s41235-016-0006-7

    Article  Google Scholar 

  23. Sing, T., Sander, O., Beerenwinkel, N., & Lengauer, T. (2005). ROCR: visualizing classifier performance in R. Bioinformatics, 21, 3940–3941.

    Article  Google Scholar 

  24. Starr, S. J., Metz, C. E., Lusted, L. B., & Goodenough, D. J. (1975). Visual detection and localization of radiographic images. Radiology, 116, 553–538.

    Article  Google Scholar 

  25. Swets, J. A., & Pickett, R. M. (1982). Evaluation of diagnostic systems: Methods from signal detection theory. New York: Academic Press.

    Google Scholar 

  26. Thomas, E. A. C., & Myers, J. L. (1972). Implications of latency data for threshold and nonthreshold models of signal detection. Journal of Mathematical Psychology, 9, 253–285. DOI: https://doi.org/10.1016/0022-2496(72)90018-1

    Article  Google Scholar 

  27. Tunnicliff, J. L., & Clark, S. E. (2000). Selecting foils for identification lineups: Matching suspects or descriptions? Law and Human Behavior, 24, 231–258.

    Article  Google Scholar 

  28. Wang, X.-F. (2012). Nonparametric smooth ROC curves for continuous data. R package version 0.1–2.

  29. Wells, G. L. (2014). Eyewitness identification: Probative value, criterion shifts, and policy regarding the sequential lineup. Current Directions in Psychological Science, 23, 11–16. doi:https://doi.org/10.1177/0963721413504781

    Article  Google Scholar 

  30. Wells, G. L., & Lindsay, R. C. (1980). On estimating the diagnosticity of eyewitness nonidentifications. Psychological Bulletin, 88, 776–784. doi:https://doi.org/10.1037/0033-2909.88.3.776

    Article  Google Scholar 

  31. Wells, G. L., & Olson, E. A. (2003). Eyewitness testimony. Annual Review of Psychology, 54, 277–295.

    Article  Google Scholar 

  32. Wells, G. L., Rydell, S. M., & Seelau, E. P. (1993). The selection of distractors for eyewitness lineups, Journal of Applied Psychology, 78, 835–844.

  33. Wells, G. L., & Turtle, J. W., (1986). Eyewitness identification: The importance of lineup models. Psychological Bulletin, 99, 320–329.

    Article  Google Scholar 

  34. Wells, G. L., Yang, Y., & Smalarz, L. (2015). Eyewitness identification: Bayesian information gain, base-rate effect equivalency curves, and reasonable suspicion. Law and Human Behavior, 39(2), 99.

    Article  Google Scholar 

  35. Wetmore, S. A., Neuschatz, J. S., Gronlund, S. D., Wooten, A., Goodsell, C. A., Carlson C. A. (2015). Effect of retention interval on showup and lineup performance. Journal of Applied Research in Memory and Cognition, 4, 8–14. DOI: https://doi.org/10.1016/j.jarmac.2014.07.003

    Article  Google Scholar 

  36. Wixted, J. T., Mickes, L. (2012). The field of eyewitness memory should abandon “probative value” and embrace receiver operating characteristic analysis. Perspectives on Psychological Science, 7, 275–278. doi:https://doi.org/10.1177/1745691612442906

    Article  PubMed  Google Scholar 

  37. Wixted, J. T., Mickes, L., Dunn, J. C., Clark, S. E., & Wells, W. (2016). Estimating the reliability of eyewitness identifications from police lineups. Proceedings of the National Academy of Sciences, 113, 304-309. https://doi.org/10.1073/pnas.1516814112

    Article  Google Scholar 

  38. Wixted, J. T., Vul, E., Mickes, L., & Wilson, B. M. (2018). Models of lineup memory. Cognitive Psychology, 105, 81–114.

    Article  Google Scholar 

  39. Wogalter, M. S., Malpass, R. S., & McQuiston, D. E. (2004). A national survey of U.S. police on preparation and conduct of identification lineups. Psychology, Crime & Law, 10, 69 –82. DOI: https://doi.org/10.1080/10683160410001641873

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Andrea Cataldo for her help collating the example data sets, Tanya Leise for help verifying AUC, and to the authors of the data sets for making them available.

Open Practices Statement

The R package and all data are available at https://osf.io/mfk4e.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Andrew L. Cohen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A Equations for simultaneous lineups

Let ϕ(s, μ, σ) be the density of a normal distribution with mean μ and standard deviation σ and let \( \Phi \left(s,\mu, \sigma \right)=\underset{-\infty }{\overset{s}{\int }}\upphi \left(x,\mu, \sigma \right) dx \) be the cumulative normal.

Let resp=response, with values sus=suspect, fil=filler, and rej=reject. Let conf=the response confidence level, with values 1…max confidence level. Note that, for notational convenience, 1 is the lowest confidence level here. Let tar=target, with values pres=present and abs=absent.

The following model parameters are used: p=P(target present), μt=target distribution mean, σt=target distribution standard deviation, μl =lure distribution mean, σl =lure distribution standard deviation, ci=the values of the ith response criterion, where c1 is the lowest response criterion.

The lineup size is given by l.

Also see Fig. 1.

Suspect response

The probability of a suspect response at confidence level i is given byFootnote 7

$$ P\left( resp= sus\cap conf=i\right)= pP\left( resp= sus\cap conf=i| tar= pres\right)+\left(1-p\right)P\left( resp= sus\cap conf=i| tar= abs\right), $$
(A1a)

where

$$ P\left( resp= sus\cap conf=i| tar= pres\right)=\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds $$
(A1b)

and

$$ P\left( resp= sus\cap conf=i| tar= abs\right)=\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds. $$
(A1c)

In Equation A1a finds the overall probability of observing a suspect ID at a given confidence level by taking the weighted average of the corresponding probability for target present and target absent lineups, where the weight is the proportion of target-present lineups (p). This equation is needed to get predictions for restricted data, such as data from real lineups, where the guilt status of each suspect is unknown.

Equations A1b and A1c provide the probabilities of choosing the suspect at a given confidence level for target-present and target-absent lineups individually. These equations are used for full data in which each suspect can be classified as guilty or innocent.

In Equation A1b, the first term in the integral is the probability density at a value of s for a guilty suspect (i.e., a draw from the target distribution). The second term in the integral is the probability that a given filler (i.e., a draw from the lure distribution) has a strength value below s, raised to the power of the number of fillers (l – 1) to give the joint probability that all of the fillers have a strength value below s (the model assumes that filler strengths are independent of one another and independent of the suspect strength). Multiplying the two terms gives the joint probability density that the suspect has a strength value of s and has a higher strength than all of the fillers, i.e., the suspect is selected as the lineup member whose strength value is used to make the identification decision. Integrating this equation between ci and ci+1 gives the joint probability that the suspect both has a strength value in this range and has the highest strength value in the lineup, i.e., the probability of selecting the suspect with confidence level i.

In Equation A1c has the same structure as Equation A1b, but the probability density for the suspect is based on the lure distribution to represent a target absent lineup.

Filler response

The probability of a filler response at confidence level i is given by

$$ P\left( resp= fil\cap conf=i\right)= pP\left( resp= fil\cap conf=i| tar= pres\right)+\left(1-p\right)P\left( resp= fil\cap conf=i| tar= abs\right), $$
(A2a)

where

$$ P\left( resp= fil\cap conf=i| tar= pres\right)=\left(l-1\right)\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-2}\Phi \left(s,{\mu}_t,{\sigma}_t\right) ds $$
(A2b)

and

$$ P\left( resp= fil\cap conf=i| tar= abs\right)=\left(l-1\right)\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-2}\Phi \left(s,{\mu}_l,{\sigma}_l\right) ds. $$
(A2c)

Equation A2a finds that the overall probability of selecting a filler at confidence level i by taking the weighted average of the corresponding probability for target-present and target-absent lineups, which would be needed for restricted data.

Equations A2b and A2c give the probability of selecting a filler at confidence level i for target-present and target-absent lineups individually, so these equations would be used for full data.

For Equation A2b, the first term in the integral is the probability density at strength value s for a filler F1 (i.e., a draw from the lure distribution). The second term in the integral is the probability that one of the other l – 2 fillers has a strength value below s. Exponentiating provides the joint probability that all of these other fillers all have a strength value below s. The third term in the integral is the probability that the suspect (i.e., a random draw from the target distribution) has a strength value below s. Multiplying these three terms gives the probability density that F1 has a strength value of s and this strength value is higher than the strength values for all the other fillers and the suspect. That is, F1 is selected as the lineup member whose strength value will inform the identification decision and has a strength value of s. Integrating this equation between ci and ci+1 gives the joint probability that filler F1 both has a strength value in this range and has the highest strength value in the lineup, that is, the probability of selecting filler F1 with confidence i. Finally, multiplying this value by the number of fillers (l – 1) gives the probability of selecting any of the fillers at confidence level i.

Equation A2c has the same structure as Equation A2b, except that the suspect becomes a draw from the lure distribution to represent a target-absent lineup.

No identification

The probability of rejecting the lineup, that is, not identifying any lineup member as the culprit, is given by

$$ P\left( resp= rej\right)= pP\left( resp= rej| tar= pres\right)+\left(1-p\right)P\left( resp= rej| tar= abs\right), $$
(A3a)

where

$$ P\left( resp= rej| tar= pres\right)=\underset{-\infty }{\overset{c_1}{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds+\left(l-1\right)\underset{-\infty }{\overset{c_1}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-2}\Phi \left(s,{\mu}_t,{\sigma}_t\right) ds $$
(A3b)

and

$$ P\left( resp= rej| tar= abs\right)=\underset{-\infty }{\overset{c_1}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds+\left(l-1\right)\underset{-\infty }{\overset{c_1}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-2}\Phi \left(s,{\mu}_l,{\sigma}_l\right) ds. $$
(A3c)

Equation A3a indicates that the overall probability of rejecting a lineup is the weighted average of the probability of rejection for target-present and target-absent lineups. This value is needed to fit restricted data.

Equations A3b and A3c give the probability of rejection for target-present and target-absent lineups, and so are used for full data.

In Equation A3b, the first integral has the same structure as Equation A1b and gives the probability that the suspect has the highest strength value and a strength value below c1. The second integral has the same structure as Equation A2b and gives the probability that a given filler has the highest strength value and a strength value below c1, which is multiplied by the number of fillers that could potentially have the highest strength value (l – 1). Adding these two terms gives the total probability that the lineup member with the highest strength value (whether suspect or filler) has a strength below c1, that is, the probability that the lineup will be rejected.

Equation A3c has the same structure as Equation A3b, except that the suspect is now a draw from the lure distribution to represent a target-absent lineup.

Appendix B Equations for sequential lineups

Let ϕ(s, μ, σ) be the density of a normal distribution with mean μ and standard deviation σ and let \( \Phi \left(s,\mu, \sigma \right)=\underset{-\infty }{\overset{s}{\int }}\upphi \left(x,\mu, \sigma \right) dx \) be the cumulative normal.

Let resp=response, with values sus=suspect, fil=filler, and rej=reject. Let conf=the response confidence level, with values 1…max confidence level. Note that, for notational convenience, 1 is the lowest confidence level here. Let tar=target, with values pres=present and abs=absent. Let spos=the subject position, with values=1…lineup size.

The following model parameters are used: p=P(target present), μt=target distribution mean, σt=target distribution standard deviation, μl =lure distribution mean, σl =lure distribution standard deviation, ci=the values of the ith response criterion, where c1 is the lowest response criterion.

The lineup size is given by l.

For the sequential lineups, suspect and filler responses depend on the suspect position.

Also see Fig. 1.

Suspect response

The probability of a suspect response at confidence level i is given byFootnote 8

$$ P\left( resp= sus\cap conf=i\right)= pP\left( resp= sus\cap conf=i| tar= pres\right)+\left(1-p\right)P\left( resp= sus\cap conf=i| tar= abs\right), $$
(B1a)

where

$$ P\left( resp= sus\cap conf=i| tar= pres\right)=\sum \limits_{j=1}^lP\left( spos=j\right)P\left( resp= sus\cap conf=i| spos=j\cap tar= pres\right), $$
(B1b)

where P(spos = j)is the probability that the suspect is in position j, as determined by the lineup designer and

$$ P\left( resp= sus\cap conf=i| spos=j\cap tar= pres\right)=\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{j-1}\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right) ds $$
(B1c)

and

$$ P\left( resp= sus\cap conf=i| tar= abs\right)=\sum \limits_{j=1}^lP\left( spos=j\right)P\left( resp= sus\cap conf=i| spos=j\cap tar= abs\right), $$
(B1d)

where

$$ P\left( resp= sus\cap conf=i| spos=j\cap tar= abs\right)=\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{j-1}\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds. $$
(B1e)

Equation B1a provides the overall probability of a suspect ID at confidence level i, used for restricted data, by taking the weighted average of the probability of a suspect ID at confidence level i for target-present and target-absent lineups. The weight is determined by the proportion of target-present lineups (p).

Equations B1b and B1c give the probability of a suspect ID at confidence level i for target-present lineups. Equation B1c assumes a given suspect position and Equation B1b calculates this value across the full distribution of suspect positions. In Equation B1c, the first term is the probability that the witness would reach the suspect position (j) in the lineup; that is, that all j – 1 preceding fillers would have strength values below the identification criterion c1. The second term is the probability that the witness would identify the suspect with confidence level i, found by integrating the probability density of the target distribution between the criteria defining the bounds of confidence region i. Multiplying the two terms gives the probability that the witness would reach the suspect in the lineup sequence and would identify them with confidence level i once they do so. Equation B1b takes the weighted average of the values returned by Equation B1c for each suspect position, where the weights are taken from the probability distribution of suspect positions.

Equations B1d and B1e have the same structure as Equations B1b and B1c, except that the suspect strength comes from the lure distribution instead of the target distribution to represent a target-absent lineup.

Filler response

The probability of a filler response at confidence level i is given by

$$ P\left( resp= fil\cap conf=i\right)= pP\left( resp= fil\cap conf=i| tar= pres\right)+\left(1-p\right)P\left( resp= fil\cap conf=i| tar= abs\right), $$
(B2a)

where

$$ P\left( resp= fil\cap conf=i| tar= pres\right)=\sum \limits_{j=1}^lP\left( spos=j\right)P\left( resp= fil\cap conf=i| spos=j\cap tar= pres\right), $$
(B2b)

where P(spos = j)is the probability that the suspect is in position j, as determined by the lineup designer, and

$$ P\left( resp= fil\cap conf=i| spos=j\cap tar= pres\right)=\sum \limits_{k=1}^l\left\{\begin{array}{cc}0& if\ k=j\\ {}\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{k-1}\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds& if\ k<j\\ {}\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{k-2}\Phi \left({c}_1,{\mu}_t,{\sigma}_t\right)\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds& if\ k>j\end{array}\right. $$
(B2c)

where k denotes each lineup position, and

$$ P\left( resp= fil\cap conf=i| tar= abs\right)=\sum \limits_{j=1}^lP\left( spos=j\right)P\left( resp= fil\cap conf=i| spos=j\cap tar= abs\right), $$
(B2d)

where

$$ P\left( resp= fil\cap conf=i| spos=j\cap tar= abs\right)=\sum \limits_{k=1}^l\left\{\begin{array}{cc}0& if\ k=j\\ {}\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{k-1}\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds& if\ k<j.\\ {}\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{k-2}\Phi \left({c}_1,{\mu}_l,{\sigma}_l\right)\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds& if\ k>j\end{array}\right. $$
(B2e)

Equations B2a, B2b, and B2d are directly analogous to Equations B1a, B1b, and B1d. See the explanation of those equations. Equations B2c and B2e are similar to Equations B1c and B1e, but they give the probability of selecting any of the fillers (as opposed to the single suspect) with a given confidence level.

First consider Equation B2c. The sum goes over all the positions (k) in the lineup (1 through l). For each position, the bracketed equations give the probability of selecting a filler at confidence level i for that position. This value is 0 if the suspect (and not a filler) is in that position (the first “if” statement). Recall that the suspect is in position j. The second “if” statement applies to filler positions that come before the suspect position in the lineup. For these positions, the equation is the probability of rejecting all of the k – 1 fillers that came before the filler in position k – that is, the probability that k – 1 random draws from the lure distribution would all fall below the identification criterion c1 multiplied by the probability of selecting confidence level i for the filler in position k, which is found by integrating the probability density of the lure distribution between the confidence criteria defining this confidence region. The third “if” statement applies to filler positions that come after the suspect position. The only change from the equation just discussed is that now the probability of getting “past” the faces before position k is found by multiplying the probability that each of the k – 2 preceding fillers fell below the identification criterion and the probability that the one preceding guilty suspect fell below the identification criterion.

Equation B2e is like Equation B2c, except that, because this is for a target-absent lineup, the suspect term in the third “if” statement is defined by the lure distribution.

No identification

Lineup rejections do not depend on suspect position. The probability of rejecting the lineup, that is, failing to identify any individual, is given by

$$ P\left( resp= rej\right)= pP\left( resp= rej| tar= pres\right)+\left(1-p\right)P\left( resp= rej| tar= abs\right), $$
(B3a)

where

$$ P\left( resp= rej| tar= pres\right)=\Phi \left({c}_1,{\mu}_t,{\sigma}_t\right)\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{l-1} $$
(B3b)

and

$$ P\left( resp= rej| tar= abs\right)=\Phi \left({c}_1,{\mu}_l,{\sigma}_l\right)\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{l-1}. $$
(B3c)

Equation B3a provides the overall probability of rejection across all lineups, used for fitting restricted data, by taking the weighted average of the probability of rejection for target-present and target-absent lineups. Equation B3b gives the probability that all members of a target-present lineup would fall below the identification criterion c1, obtained by multiplying the probability that a draw from the target distribution falls below this criterion by the probability that all of the l – 1 fillers also fall below this criterion. Equation B3c is the same as equation B3b, except now all lineup members are assumed to be draws from the lure distribution.

Appendix C Equations for AUC and diagnosticity

Area under the curve

Let Ta = P(suspect pick | target absent) for a given set of model parameters θ. Let \( {T}_a^{\ast } \) be the highest Ta on the ROC curve, which can be less than 1 for lineups. Let Tp(x) = P(suspect pick | target present) when Ta = x. Then,

$$ AUC=\underset{0}{\overset{T_a^{\ast }}{\int }}{T}_p(x) dx. $$
(C1)

Analytic solutions do not typically exist for AUC. Here, AUC is computed numerically using iterative quadrature.

Diagnosticity

Let ϕ(s, μ, σ) be the density of a normal distribution with mean μ and standard deviation σ and let \( \Phi \left(s,\mu, \sigma \right)=\underset{-\infty }{\overset{s}{\int }}\upphi \left(x,\mu, \sigma \right) dx \) be the cumulative normal. Let μt=target distribution mean, σt=target distribution standard deviation, μl =lure distribution mean, σl =lure distribution standard deviation, ci=the values of the ith response criterion, where c1 is the lowest response criterion. Let the lineup size be given by l. Also see Fig. 1.

Simultaneous lineups

First, consider diagnosticity for simultaneous lineups. Overall diagnosticity, collapsing over response threshold, is given by

$$ diagnosticity=\frac{\underset{c_1}{\overset{\infty }{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds}{\underset{c_1}{\overset{\infty }{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds} $$
(C2)

and diagnosticity at confidence level i is given by

$$ diagnosticity=\frac{\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds}{\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right)\Phi {\left(s,{\mu}_l,{\sigma}_l\right)}^{l-1} ds}. $$
(C3)

Sequential lineups

Next, consider diagnosticity for sequential lineups. Overall diagnosticity, collapsing over confidence, is given by

$$ diagnosticity=\frac{\sum \limits_{j=1}^lP\left( spos=j\right)\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{j-1}\underset{c_1}{\overset{\infty }{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right) ds}{\sum \limits_{j=1}^lP\left( spos=j\right)\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{j-1}\underset{c_1}{\overset{\infty }{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds}, $$
(C4)

where P(spos=j) is the probability of a suspect at position j.

The diagnosticity at confidence level i is given by

$$ diagnosticity=\frac{\sum \limits_{j=1}^lP\left( spos=j\right)\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{j-1}\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_t,{\sigma}_t\right) ds}{\sum \limits_{j=1}^lP\left( spos=j\right)\Phi {\left({c}_1,{\mu}_l,{\sigma}_l\right)}^{j-1}\underset{c_i}{\overset{c_{i+1}}{\int }}\upphi \left(s,{\mu}_l,{\sigma}_l\right) ds}. $$
(C5)

Appendix D Full sdtlu_compare_2 figure

Fig. 16
figure16

Figure generated from sdtlu_compare_2 with the Palmer et al. (2013) short and long delay simultaneous lineup data. To remove whitespace, the figure has been reformatted

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cohen, A.L., Starns, J.J. & Rotello, C.M. sdtlu: An R package for the signal detection analysis of eyewitness lineup data. Behav Res 53, 278–300 (2021). https://doi.org/10.3758/s13428-020-01402-7

Download citation

Keywords

  • Eyewitness lineups
  • Computational modeling
  • Signal detection
  • R package