Performance Testing and Reporting
The evaluation of system performance and assessment of overall system reliability are important to the implementation and use of all biometric systems. Almost all biometric systems are ‘black boxes’ to their users, and it is only through appropriate testing that a system’s accuracy and performance can be determined. However, this testing process is not always straightforward, as there are many factors that influence performance, and published error rates often provide only approximate guidance. Issues related to the misunderstanding of test results are frequently the reason why some biometric implementations have failed to meet expectations. Properly conducted testing can provide crucial information for the planning of a system’s functionality, and the introduction of controls to maximize results.
Discuss the evaluation process (Sect. 5.1).
Describe the components of a test plan (Sect. 5.2).
Introduce the different types of evaluation (Sect. 5.2.1).
Present the fundamental techniques for assessment, including establishing ground truth and data size (Sect. 5.3.1).
Discuss issues related to the test set (Sect. 5.3).
Present the elements of proper reporting of test results (Sect. 5.4).
KeywordsGround Truth Performance Test Testing Process Test Plan Biometric System
Unable to display preview. Download preview PDF.
- 1.Performix biometric research and analysis software. http://www.biometix.com/performix.htm(2007)
- 2.Cappelli, R., Erol, A., Maio, D., Maltoni, D.: Synthetic Fingerprint-image generation. Pattern Recognition, 2000. Proceedings. 15th International Conference on 3 (2000)Google Scholar
- 3.ISO: Information technology –biometric performance testing and reporting – part 1: Principles and framework (ISO/IEC 19795-1:2006) (2006)Google Scholar
- 6.Mansfield, A., Wayman, J.: Best practices in testing and reporting performance of biometric devices. NPL Report CMSC 14(02) (2002)Google Scholar
- 7.Mansfield, T., Kelly, G., Chandler, D., Kane, J.: Biometric product testing final report. Contract 92, 4009,309 (2001)Google Scholar
- 9.Phillips, P.J., Scruggs, W.T., O’Toole, A.J., Flynn, P.J., Bowyer, K.W., Schott, C.L., Sharpe, M.: FRVT 2006 and ice 2006 large-scale results. Tech. Rep. NISTIR 7408, National Institute of Standards and Technology (????)Google Scholar
- 10.Reynolds, D.A., Doddington, G.R., Przybocki, M.A., Martin, A.F.: The NIST speaker recognition evaluation -overview methodology, systems, results, perspective. Speech Commun. 31(2-3), 225–254 (2000)Google Scholar
- 11.Schuckers, M.: Estimation and sample size calculations for correlated binary error rates of biometric identification rates.Proceedings of the American Statistical Association: Biometrics Section (2003)Google Scholar