Artificial Retina: A Future Cellular-Resolution Brain-Machine Interface

  • Dante G. MuratoreEmail author
  • E. J. Chichilnisky
Part of the The Frontiers Collection book series (FRONTCOLL)


Brain-machine interfaces (BMIs) of the future will be used to treat diverse neurological disorders and augment human capabilities. However, to realize this futuristic promise will require a major leap forward in how electronic devices interact with the nervous system. Current BMIs provide coarse communication with the target neural circuitry, because they fail to respect its cellular and cell-type specificity. Instead, they indiscriminately activate or record many cells at the same time and provide only partial restoration of lost abilities. A future BMI that may pave the path forward is an artificial retina—a device that can restore vision to people blinded by retinal degeneration. Because the retina is relatively well understood and easily accessible, it is an ideal neural circuit in which to develop a BMI that can approach or exceed the performance of the biological circuitry. This chapter summarizes the basic neuroscience of vision, identifies the requirements for an effective retinal interface, and describes some of the necessary circuits and systems. Based on these ideas and the lessons from first-generation retinal prostheses, a novel neuroengineering approach is proposed: the first BMI that will interact with neural circuitry at cellular and cell-type resolution.



The authors would like to thank Marty Breidenbach, Ruwan Silva, Stephen Weinreich, Matthias Kuhl, Daniel Palanker, Nishal Shah and the Stanford Artificial Retina Group [54] for useful discussions and comments, and Peter Li, Chris Sekirnjak, and Nora Brackbill for figure contributions. DM, EJC and the Stanford Artificial Retina Project are funded by the Wu Tsai Neurosciences Institute. EJC is funded by NEI grant EY021271 and a Stein Innovation Award from Research to Prevent Blindness.


  1. 1.
    M.W. Slutzky, Brain-machine interfaces: powerful tools for clinical treatment and neuroscientific investigations. Neuroscientist 25, 139–154 (2019)CrossRefGoogle Scholar
  2. 2.
    M.A. Lebedev, M.A.L. Nicolelis, Brain-machine interfaces: from basic science to neuroprostheses and neurorehabilitation. Physiol. Rev. 97, 767–837 (2017)CrossRefGoogle Scholar
  3. 3.
    A.H. Marblestone, B.M. Zamft, Y.G. Maguire, M.G. Shapiro, T.R. Cybulski, J.I. Glaser, D. Amodei, P.B. Stranges, R. Kalhor, D.A. Dalrymple, D. Seo, E. Alon, M.M. Maharbiz, J.M. Carmena, J.M. Rabaey, E.S. Boyden, G.M. Church, K.P. Kording, Physical principles for scalable neural recording. Front. Comput. Neurosci. 7, 137 (2013)CrossRefGoogle Scholar
  4. 4.
    G.W. Fraser, S.M. Chase, A. Whitford, A.B. Schwartz, Control of a brain–computer interface without spike sorting. J. Neural Eng. 6, 055004 (2009)ADSCrossRefGoogle Scholar
  5. 5.
    C.A. Chestek, V. Gilja, P. Nuyujukian, J.D. Foster, J.M. Fan, M.T. Kaufman, M.M. Churchland, Z. Rivera-Alvidrez, J.P. Cunningham, S.I. Ryu, K.V. Shenoy, Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. J. Neural Eng. 8, 045005 (2011)ADSCrossRefGoogle Scholar
  6. 6.
    B.P. Christie, D.M. Tat, Z.T. Irwin, V. Gilja, P. Nuyujukian, J.D. Foster, S.I. Ryu, K.V. Shenoy, D.E. Thompson, C.A. Chestek, Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. J. Neural Eng. 12, 016009 (2015)ADSCrossRefGoogle Scholar
  7. 7.
    J. Li, Z. Li, Sums of spike waveform features for motor decoding. Front. Neurosci. 11, 406 (2017)CrossRefGoogle Scholar
  8. 8.
    E.M. Trautmann, S.D. Stavisky, S. Lahiri, K.C. Ames, M.T. Kaufman, D.J. O’Shea, S. Vyas, X. Sun, S.I. Ryu, S. Ganguli, K.V. Shenoy, Accurate estimation of neural population dynamics without spike sorting. Neuron 103, 292–308.e4 (2019)CrossRefGoogle Scholar
  9. 9.
    N. Even-Chen, D.G. Muratore, S.D. Stavisky, L.R. Hochberg, J.M. Henderson, B. Murmann, K.V. Shenoy, Motor intracortical interface design opportunities for an order of magnitude power saving. Nat. Biomed. Eng. (2020) (In Press)Google Scholar
  10. 10.
    G.A. Goetz, D.V. Palanker, Electronic approaches to restoration of sight. Rep. Prog. Phys. 79, 096701 (2016)ADSCrossRefGoogle Scholar
  11. 11.
    E.R. Kandel, J.H. Schwartz, T.M. Jessell, Principles of Neural Science (McGraw-Hill, New York, 2012)Google Scholar
  12. 12.
    C.E. Schoonover, Portraits of the Mind: Visualizing the Brain from Antiquity to the 21st Century. Abrams (2010)Google Scholar
  13. 13.
    L.E. Grosberg, K. Ganesan, G.A. Goetz, S.S. Madugula, N. Bhaskhar, V. Fan, P. Li, P. Hottowy, W. Dabrowski, A. Sher, A.M. Litke, S. Mitra, E.J. Chichilnisky, Activation of ganglion cells and axon bundles using epiretinal electrical stimulation. J. Neurophysiol. 118, 1457–1471 (2017)CrossRefGoogle Scholar
  14. 14.
    A.L. Yarbus, Eye Movements and Vision (Plenum, New York, 1967)CrossRefGoogle Scholar
  15. 15.
    R.W. Rodieck, The First Steps in Seeing (Sinauer, Sunderland, 1998)Google Scholar
  16. 16.
    B.A. Wandell, Foundations of Vision (Sinauer, Sunderland, 1995)Google Scholar
  17. 17.
    B. Roska, M. Meister, The retina dissects the visual scene into distinct features, in The New Visual Neurosciences (2014), pp. 163–182Google Scholar
  18. 18.
    T. Gollisch, M. Meister, Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65, 150–164 (2010)CrossRefGoogle Scholar
  19. 19.
    M. Beyeler, A. Rokem, G.M. Boynton, I. Fine, Learning to see again: biological constraints on cortical plasticity and the implications for sight restoration technologies. J. Neural Eng. 14, 051003 (2017)ADSCrossRefGoogle Scholar
  20. 20.
    P. Dayan, L.F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, 2001)Google Scholar
  21. 21.
    H.G. Rey, C. Pedreira, R. Quian Quiroga, Past, present and future of spike sorting techniques. Brain Res. Bull. 119, 106–117 (2015)CrossRefGoogle Scholar
  22. 22.
    J.E. Chung, J.F. Magland, A.H. Barnett, V.M. Tolosa, A.C. Tooker, K.Y. Lee, K.G. Shah, S.H. Felix, L.M. Frank, L.F. Greengard, A fully automated approach to spike sorting. Neuron 95, 1381–1394.e6 (2017)CrossRefGoogle Scholar
  23. 23.
    D. Carlson, L. Carin, Continuing progress of spike sorting in the era of big data. Curr. Opin. Neurobiol. 55, 90–96 (2019)CrossRefGoogle Scholar
  24. 24.
    J.F. Fohlmeister, P.A. Coleman, R.F. Miller, Modeling the repetitive firing of retinal ganglion cells. Brain Res. 510, 343–345 (1990)CrossRefGoogle Scholar
  25. 25.
    D. Boinagrov, J. Loudin, D. Palanker, Strength-duration relationship for extracellular neural stimulation: numerical and analytical models. J. Neurophysiol. 104, 2236–2248 (2010)CrossRefGoogle Scholar
  26. 26.
    G.J. Chader, J. Weiland, M.S. Humayun, Artificial vision: needs, functioning, and testing of a retinal electronic prosthesis. Prog. Brain Res. 175, 317–332 (2009)CrossRefGoogle Scholar
  27. 27.
    E. Zrenner, Fighting blindness with microelectronics. Sci. Transl. Med. 5, 210ps16 (2013)CrossRefGoogle Scholar
  28. 28.
    A.T. Chuang, C.E. Margo, P.B. Greenberg, Retinal implants: a systematic review. Br. J. Ophthalmol. 98, 852–856 (2014)CrossRefGoogle Scholar
  29. 29.
    Y.H.-L. Luo, L. da Cruz, A review and update on the current status of retinal prostheses (bionic eye). Br. Med. Bull. 109, 31–44 (2014)CrossRefGoogle Scholar
  30. 30.
    K. Grifantini, Aiding the Eye, Watching the Brain: James Weiland, IEEE Fellow, explores the unique challenges of retinal prostheses. IEEE Pulse 8, 39–41 (2017)CrossRefGoogle Scholar
  31. 31.
    M.S. Humayun, E. de Juan, G. Dagnelie Jr., The bionic eye: a quarter century of retinal prosthesis research and development. Ophthalmology 123, S89–S97 (2016)CrossRefGoogle Scholar
  32. 32.
    D. Nanduri, M.S. Humayun, R.J. Greenberg, M.J. McMahon, J.D. Weiland, Retinal prosthesis phosphene shape analysis, in 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2008), pp. 1785–1788Google Scholar
  33. 33.
    C. de Balthasar, S. Patel, A. Roy, R. Freda, S. Greenwald, A. Horsager, M. Mahadevappa, D. Yanai, M.J. McMahon, M.S. Humayun, R.J. Greenberg, J.D. Weiland, I. Fine, Factors affecting perceptual thresholds in epiretinal prostheses. Invest. Ophthalmol. Vis. Sci. 49, 2303–2314 (2008)CrossRefGoogle Scholar
  34. 34.
    A. Caspi, J.D. Dorn, K.H. McClure, M.S. Humayun, R.J. Greenberg, M.J. McMahon, Feasibility study of a retinal prosthesis: spatial vision with a 16-electrode implant. Arch. Ophthalmol. 127, 398–401 (2009)CrossRefGoogle Scholar
  35. 35.
    R.J. Jensen, J.F. Rizzo 3rd, O.R. Ziv, A. Grumet, J. Wyatt, Thresholds for activation of rabbit retinal ganglion cells with an ultrafine, extracellular microelectrode. Invest. Ophthalmol. Vis. Sci. 44, 3533–3543 (2003)CrossRefGoogle Scholar
  36. 36.
    A. Butterwick, A. Vankov, P. Huie, Y. Freyvert, D. Palanker, Tissue damage by pulsed electrical stimulation. IEEE Trans. Biomed. Eng. 54, 2261–2267 (2007)CrossRefGoogle Scholar
  37. 37.
    S.F. Cogan, Neural stimulation and recording electrodes. Annu. Rev. Biomed. Eng. 10, 275–309 (2008)CrossRefGoogle Scholar
  38. 38.
    D.R. Merrill, M. Bikson, J.G.R. Jefferys, Electrical stimulation of excitable tissue: design of efficacious and safe protocols. J. Neurosci. Methods 141, 171–198 (2005)CrossRefGoogle Scholar
  39. 39.
    L.S. Robblee, T.L. Rose, Electrochemical guidelines for selection of protocols and electrode materials for neural stimulation, in Neural Prostheses: Fundamental Studies (1990), pp. 25–66Google Scholar
  40. 40.
    A. Horsager, S.H. Greenwald, J.D. Weiland, M.S. Humayun, R.J. Greenberg, M.J. McMahon, G.M. Boynton, I. Fine, Predicting visual sensitivity in retinal prosthesis patients. Invest. Ophthalmol. Vis. Sci. 50, 1483–1491 (2009)CrossRefGoogle Scholar
  41. 41.
    J.-H. Jung, D. Aloni, Y. Yitzhaky, E. Peli, Active confocal imaging for visual prostheses. Vis. Res. 111, 182–196 (2015)CrossRefGoogle Scholar
  42. 42.
    E. Zrenner, K.U. Bartz-Schmidt, H. Benav, D. Besch, A. Bruckmann, V.-P. Gabel, F. Gekeler, U. Greppmaier, A. Harscher, S. Kibbel, J. Koch, A. Kusnyerik, T. Peters, K. Stingl, H. Sachs, A. Stett, P. Szurman, B. Wilhelm, R. Wilke, Subretinal electronic chips allow blind patients to read letters and combine them to words. Proc. R. Soc. B Biol. Sci. 278, 1489–1497 (2011)CrossRefGoogle Scholar
  43. 43.
    H. Lorach, G. Goetz, R. Smith, X. Lei, Y. Mandel, T. Kamins, K. Mathieson, P. Huie, J. Harris, A. Sher, D. Palanker, Photovoltaic restoration of sight with high visual acuity. Nat. Med. 21, 476–482 (2015)CrossRefGoogle Scholar
  44. 44.
    L.H. Jepson, P. Hottowy, G.A. Weiner, W. Dabrowski, A.M. Litke, E.J. Chichilnisky, High-fidelity reproduction of spatiotemporal visual signals for retinal prosthesis. Neuron 83, 87–92 (2014)CrossRefGoogle Scholar
  45. 45.
    C. Sekirnjak, P. Hottowy, A. Sher, W. Dabrowski, A.M. Litke, E.J. Chichilnisky, High-resolution electrical stimulation of primate retina for epiretinal implant design. J. Neurosci. 28, 4446–4456 (2008)CrossRefGoogle Scholar
  46. 46.
    L.H. Jepson, P. Hottowy, K. Mathieson, D.E. Gunning, W. Dabrowski, A.M. Litke, E.J. Chichilnisky, Focal electrical stimulation of major ganglion cell types in the primate retina for the design of visual prostheses. J. Neurosci. 33, 7194–7205 (2013)CrossRefGoogle Scholar
  47. 47.
    A.M. Litke, N. Bezayiff, E.J. Chichilnisky, W. Cunningham, W. Dabrowski, A.A. Grillo, M. Grivich, P. Grybos, P. Hottowy, S. Kachiguine, R.S. Kalmar, K. Mathieson, D. Petrusca, M. Rahman, A. Sher, What does the eye tell the brain?: Development of a system for the large-scale recording of retinal output activity. IEEE Trans. Nucl. Sci. 51, 1434–1440 (2004)ADSCrossRefGoogle Scholar
  48. 48.
    E.J. Chichilnisky, R.S. Kalmar, Functional asymmetries in ON and OFF ganglion cells of primate retina. J. Neurosci. 22, 2737–2747 (2002)CrossRefGoogle Scholar
  49. 49.
    E.S. Frechette, A. Sher, M.I. Grivich, D. Petrusca, A.M. Litke, E.J. Chichilnisky, Fidelity of the ensemble code for visual motion in primate retina. J. Neurophysiol. 94, 119–135 (2005)CrossRefGoogle Scholar
  50. 50.
    G.D. Field, J.L. Gauthier, A. Sher, M. Greschner, T.A. Machado, L.H. Jepson, J. Shlens, D.E. Gunning, K. Mathieson, W. Dabrowski, L. Paninski, A.M. Litke, E.J. Chichilnisky, Functional connectivity in the retina at the resolution of photoreceptors. Nature 467, 673–677 (2010)ADSCrossRefGoogle Scholar
  51. 51.
    V.H. Fan, L.E. Grosberg, S.S. Madugula, P. Hottowy, W. Dabrowski, A. Sher, A.M. Litke, E.J. Chichilnisky, Epiretinal stimulation with local returns enhances selectivity at cellular resolution. J. Neural Eng. 16, 025001 (2019)ADSCrossRefGoogle Scholar
  52. 52.
    L.H. Jepson, P. Hottowy, K. Mathieson, D.E. Gunning, W. Dąbrowski, A.M. Litke, E.J. Chichilnisky, Spatially patterned electrical stimulation to enhance resolution of retinal prostheses. J. Neurosci. 34, 4871–4881 (2014)CrossRefGoogle Scholar
  53. 53.
    C.E. Rhoades, N.P. Shah, M.B. Manookin, N. Brackbill, A. Kling, G. Goetz, A. Sher, A.M. Litke, E.J. Chichilnisky, Unusual physiological properties of smooth monostratified ganglion cell types in primate retina. Neuron 103, 658–672.e6 (2019)CrossRefGoogle Scholar
  54. 54.
    Stanford Artificial Retina Project.
  55. 55.
    E. Richard, G.A. Goetz, E.J. Chichilnisky, Recognizing retinal ganglion cells in the dark, in Advances in Neural Information Processing Systems (2015), pp. 2476–2484Google Scholar
  56. 56.
    V. Karkare, S. Gibson, D. Marković, A 75-µW, 16-channel neural spike-sorting processor with unsupervised clustering. IEEE J. Solid-State Circuits 48, 2230–2238 (2013)ADSCrossRefGoogle Scholar
  57. 57.
    M. Pagin, M. Ortmanns, A neural data lossless compression scheme based on spatial and temporal prediction, in 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS) (2017), pp. 1–4Google Scholar
  58. 58.
    C. Aprile, K. Ture, L. Baldassarre, M. Shoaran, G. Yilmaz, F. Maloberti, C. Dehollain, Y. Leblebici, V. Cevher, Adaptive learning-based compressive sampling for low-power wireless implants. IEEE Trans. Circuits Syst. I Regul. Pap. 65, 3929–3941 (2018)CrossRefGoogle Scholar
  59. 59.
    T. Wu, W. Zhao, E. Keefer, Z. Yang, Deep compressive autoencoder for action potential compression in large-scale neural recording. J. Neural Eng. 15, 066019 (2018)ADSCrossRefGoogle Scholar
  60. 60.
    T. Okazawa, I. Akita, A time-domain analog spatial compressed sensing encoder for multi-channel neural recording. Sensors 18, 184 (2018). Scholar
  61. 61.
    V. Majidzadeh, A. Schmid, Y. Leblebici, A 16-channel, 359 μW, parallel neural recording system using Walsh-Hadamard coding, in Proceedings of the IEEE 2013 Custom Integrated Circuits Conference (2013), pp. 1–4Google Scholar
  62. 62.
    D. Tsai, R. Yuste, K.L. Shepard, Statistically reconstructed multiplexing for very dense, high-channel-count acquisition systems. IEEE Trans. Biomed. Circuits Syst. 12, 13–23 (2018)CrossRefGoogle Scholar
  63. 63.
    M. Sharma, A.T. Gardner, H.J. Strathman, D.J. Warren, J. Silver, R.M. Walker, Acquisition of neural action potentials using rapid multiplexing directly at the electrodes. Micromachines (2018). Scholar
  64. 64.
    D.G. Muratore, P. Tandon, M. Wootters, E.J. Chichilnisky, S. Mitra, B. Murmann, A data-compressive wired-or readout for massively parallel neural recording. IEEE Trans. Biomed. Circuits Syst. 13, 1128–1140 (2019)Google Scholar
  65. 65.
    N.P. Shah, S. Madugula, L. Grosberg, G. Mena, P. Tandon, P. Hottowy, A. Sher, A. Litke, S. Mitra, E.J. Chichilnisky, Optimization of electrical stimulation for a high-fidelity artificial retina, in 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER) (2019)Google Scholar
  66. 66.
    A. Zhou, B.C. Johnson, R. Muller, Toward true closed-loop neuromodulation: artifact-free recording during stimulation. Curr. Opin. Neurobiol. 50, 119–127 (2018)CrossRefGoogle Scholar
  67. 67.
    N.P. Shah, S. Madugula, P. Hottowy, A. Sher, A. Litke, L. Paninski, E.J. Chichilnisky, Efficient characterization of electrically evoked responses for neural interfaces, in Neural Information Processing Systems (NeurIPS) (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA
  2. 2.Wu Tsai Neurosciences InstituteStanford UniversityStanfordUSA
  3. 3.Department of NeurosurgeryStanford UniversityStanfordUSA
  4. 4.Department of OphthalmologyStanford UniversityStanfordUSA

Personalised recommendations