An Affordable Bio-Sensing and Activity Tagging Platform for HCI Research

  • SiddharthEmail author
  • Aashish Patel
  • Tzyy-Ping Jung
  • Terrence J. Sejnowski
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10284)


We present a novel multi-modal bio-sensing platform capable of integrating multiple data streams for use in real-time applications. The system is composed of a central compute module and a companion headset. The compute node collects, time-stamps and transmits the data while also providing an interface for a wide range of sensors including electroencephalogram, photoplethysmogram, electrocardiogram, and eye gaze among others. The companion headset contains the gaze tracking cameras. By integrating many of the measurements systems into an accessible package, we are able to explore previously unanswerable questions ranging from open-environment interactions to emotional-response studies. Though some of the integrated sensors are designed from the ground-up to fit into a compact form factor, we validate the accuracy of the sensors and find that they perform similarly to, and in some cases better than, alternatives.


Bio-sensing Multi-modal bio-sensing Emotion studies Brain-computer interfaces 


  1. 1.
    LaFleur, K., et al.: Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface. J. Neural Eng. 10(4), 046003 (2013)CrossRefGoogle Scholar
  2. 2.
    Bell, C.J., et al.: Control of a humanoid robot by a noninvasive brain–computer interface in humans. J. Neural Eng. 5(2), 214 (2008)CrossRefGoogle Scholar
  3. 3.
    Carlson, T., del Millan, R.J.: Brain-controlled wheelchairs: a robotic architecture. IEEE Robot. Autom. Mag. 20(1), 65–73 (2013)CrossRefGoogle Scholar
  4. 4.
    Kassner, M., Patera, W., Bulling, A.: Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction, April 2014. CoRR abs 1405.0006 (2014)Google Scholar
  5. 5.
    Makeig, S., et al.: Independent component analysis of electroencephalographic data. In: Advances in Neural Information Processing Systems, pp. 145–151 (1996)Google Scholar
  6. 6.
    Makeig, S., et al.: Blind separation of auditory event-related brain responses into independent components. Proc. Natl. Acad. Sci. 94(20), 10979–10984 (1997)CrossRefGoogle Scholar
  7. 7.
    Poh, M.-Z., Swenson, N.C., Picard, R.W.: Motion-tolerant magnetic earring sensor and wireless earpiece for wearable photoplethysmography. IEEE Trans. Inf. Technol. Biomed. 14(3), 786–794 (2010)CrossRefGoogle Scholar
  8. 8.
    Van der Wall, E.E., Van Gilst, W.H.: Neurocardiology: close interaction between heart and brain. Neth. Heart J. 21(2), 51–52 (2013)CrossRefGoogle Scholar
  9. 9.
    Patterson, J.A.C., McIlwraith, D.C., Yang, G.-Z.: A flexible, low noise reflective PPG sensor platform for ear-worn heart rate monitoring. In: Sixth International Workshop on Wearable and Implantable Body Sensor Networks, BSN 2009. IEEE (2009)Google Scholar
  10. 10.
    Samuels, M.A.: The brain–heart connection. Circulation 116(1), 77–84 (2007)CrossRefGoogle Scholar
  11. 11.
    Morgante, J.D., Zolfaghari, R., Johnson, S.P.: A critical test of temporal and spatial accuracy of the Tobii T60XL eye tracker. Infancy 17(1), 9–32 (2012)CrossRefGoogle Scholar
  12. 12.
    Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010)CrossRefGoogle Scholar
  13. 13.
    Notch Motion Tracking System.
  14. 14.
  15. 15.
    Da He, D., Winokur, E.S., Sodini, C.G.: A continuous, wearable, and wireless heart monitor using head ballistocardiogram (BCG) and head electrocardiogram (ECG). In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC (2011)Google Scholar
  16. 16.
    He, D.D.: A wearable heart monitor at the ear using ballistocardiogram (BCG) and electrocardiogram (ECG) with a nanowatt ECG heartbeat detection circuit. Dissertation Massachusetts Institute of Technology (2013)Google Scholar
  17. 17.
    Vaughan, T.M., Wolpaw, J.R., Donchin, E.: EEG-based communication: prospects and problems. IEEE Trans. Rehabil. Eng. 4(4), 425–430 (1996)CrossRefGoogle Scholar
  18. 18.
    Widrow, B., et al.: Adaptive noise cancelling: principles and applications. Proc. IEEE 63(12), 1692–1716 (1975)CrossRefGoogle Scholar
  19. 19.
    Redmon, J., et al.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  20. 20.
    Everingham, M., et al.: The Pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  21. 21.
    Bell, A.J., Sejnowski, T.J.: An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7(6), 1129–1159 (1995)CrossRefGoogle Scholar
  22. 22.
    Kothe, C.A., Makeig, S.: BCILAB: a platform for brain–computer interface development. J. Neural Eng. 10(5), 056014 (2013)CrossRefGoogle Scholar
  23. 23.
    Kothe, C.: Lab streaming layer (LSL). Accessed 2015
  24. 24.
    Hsu, S.-H., et al.: Online recursive independent component analysis for real-time source separation of high-density EEG. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE (2014)Google Scholar
  25. 25.
    Matlab Signal Processing Toolbox.
  26. 26.
    Devillez, H., Nathalie, G., Guérin-Dugué, A.: An eye fixation–related potentials analysis of the P300 potential for fixations onto a target object when exploring natural scenes. J. Vis. 15(13), 20 (2015)CrossRefGoogle Scholar
  27. 27.
    Kamienkowski, J.E., et al.: Fixation-related potentials in visual search: a combined EEG and eye tracking study fixation-related potentials in visual search. J. Vis. 12(7), 4 (2012)CrossRefGoogle Scholar
  28. 28.
    Acqualagna, L., Blankertz, B.: Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clin. Neurophysiol. 124(5), 901–908 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Siddharth
    • 1
    • 2
    Email author
  • Aashish Patel
    • 1
  • Tzyy-Ping Jung
    • 2
  • Terrence J. Sejnowski
    • 2
    • 3
  1. 1.Department of Electrical and Computer EngineeringUniversity of CaliforniaSan DiegoUSA
  2. 2.Institute for Neural ComputationUniversity of CaliforniaSan DiegoUSA
  3. 3.The Computational Neurobiology LaboratorySalk InstituteLa JollaUSA

Personalised recommendations