Acuity VEP: improved with machine learning
Acuity-VEP approaches basically all use the information obtained across a number of check sizes (or spatial frequencies) to derive a measure of acuity. Amplitude is always used, sometimes combined with phase or a noise measure. In our approach, we employ steady-state brief-onset low-contrast checkerboard stimulation and obtain amplitude and significance for six different check sizes, yielding 12 numbers. The rule-based “heuristic algorithm” (Bach et al. in Br J Ophthalmol 92:396–403, 2008. https://doi.org/10.1136/bjo.2007.130245) is successful in over 95% with a limit of agreement (LoA) of ± 0.3LogMAR between behavioral and objective acuity for 109 cases. We here aimed to test whether machine learning techniques with this relatively small dataset could achieve a similar LoA.
Given recent advances in machine learning (ML), we applied a wide class of ML algorithms to this dataset. This was done within the “caret” framework of R using altogether 89 methods, of which rule-based and multiple regression approaches performed best. For cross-validation, using a jackknife (leave-one-out) approach, we predicted each case based on an ML model having been trained on all remaining 108 cases.
The ML approach predicted visual acuity well across many different types of ML algorithms. Using amplitude values only (discarding the p values) improved the outcome. Nearly half of the tested ML algorithms achieved an LoA better than the heuristic algorithm; several “Random Forest”- or “multiple regression”-type algorithms achieved an LoA of below ± 0.3. In the cases where the heuristic approach failed, acuity was predicted successfully. We then applied the ML model trained with the Bach et al.  dataset to a new dataset from 2018 (78 cases) and found both for the heuristic algorithm and for the ML approach an LoA of ± 0.259, a nearly one-line improvement.
The ML approach appears to be a useful alternative to rule-based analysis of acuity-VEP data. The achieved accuracy is comparable or better (in no case the ML-based acuity differed more than ± 0.29 LogMAR from behavioral acuity), and testability is higher, nearly 100%. Possible pitfalls are examined.
KeywordsVisual acuity VEP Machine learning Objective assessment
We are grateful to Jessica Knötzele who gave permission to use a dataset that she had acquired as part of a different study. We also thank our anonymous reviewers, one of whom suggested the assessment depicted in Fig. 5.
Compliance with Ethical Standards
Conflict of interest
Michael Bach serves as consultant for Diagnosys, aiding this company to implement the heuristic approach . Autor Sven Heinrich declares that he has no conflict of interest.
Statement of human rights
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Statement on the welfare of animals
This article does not contain any studies with animals performed by any of the authors.
Informed consent was obtained from all individual participants included in the study.
- 6.R Development Core Team (2014) R: A Language and Environment for Statistical Computing. http://www.R-project.org. Accessed 18 Aug 2014
- 7.RStudio Team (2015) (2015). RStudio: Integrated Development for R. http://www.rstudio.com/. Accessed 21 Oct 2018
- 8.Kuhn M (2018) caret (Classification and Regression Training) R package that contains misc functions for training and plotting classification and regression models: topepo/caret. https://github.com/topepo/caret. Accessed 21 Oct 2018