Skip to main content
SpringerLink
Log in

Experiential Media Systems – The Biofeedback Project

  • Chapter
  • First Online:
Multimedia Content Analysis

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

Experiential media systems refer to real-time, physically grounded multimedia systems in which the user is both the producer and consumer of meaning. These systems require embodied interaction on part of the user to gain new knowledge. In this chapter, we have presented our efforts to develop a real-time, multimodal biofeedback system for stroke patients. It is a highly specialized experiential media system where the knowledge that is imparted refers to a functional task — the ability to reach and grasp an object. There are several key ideas in this chapter: we show how to derive critical motion features using a biomechanical model for the reaching functional task. Then we determine the formal progression of the feedback and its relationship to action. We show how to map movement parameters into auditory and visual parameters in real-time. We develop novel validation metrics for spatial accuracy, opening, flow, and consistency. Our real-world experiments with unimpaired subjects show that we are able to communicate key aspects of motion through feedback. Importantly, they demonstrate that the messages encoded in the feedback can be parsed by the unimpaired subjects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. G. D. Abowd, E. D. Mynatt and T. Rodden (2002). The human experience [of ubiquitous computing]. IEEE Pervasive Computing 1(1): 48–57.

    Article  Google Scholar 

  2. . V. Basmajian (1989). Biofeedback : principles and practice for clinicians. Williams & Wilkins 0683003569 Baltimore.

    Google Scholar 

  3. R. A. Brooks (1991). Intelligence Without Reason, International Joint Conference on Articial Intelligence, pp. 569–595, Aug. 1991, Sydney, Australia.

    Google Scholar 

  4. R. A. Brooks (1991). Intelligence without representation. Artificial Intelligence 47(1–3): 139–159.

    Article  Google Scholar 

  5. R. A. Brooks, M. Coen, D. Dang, J. Debonet, J. Kramer, T. Lozano-Perez, J. Mellor, P. Pook, C. Stauffer, L. Stein, M. Torrance and M. Wessler (1997). The Intelligent Room Project, Proceedings of the Second International Cognitive Technology Conference (CT'97), Aug. 1997, Aizu, Japan.

    Google Scholar 

  6. Y. Chen, H. Huang, W. Xu, R. Wallis, H. Sundaram, T. Rikakis, J. He, T. Ingalls and L. Olson (2006). The Design Of A Real-Time, Multimodal Biofeedback System For Stroke Patient Rehabilitation, SIG ACM Multimedia, Oct. 2006, Santa Barbara, CA.

    Book  Google Scholar 

  7. Y. Chen, H. Huang, W. Xu, R. I. Wallis, H. Sundaram, T. Rikakis, T. Ingalls, L. Olson and J. He (2006). The design of a real-time, multimodal biofeedback system for stroke patient rehabilitation, Proc. of the 14th annual ACM international conference on Multimedia, 763–772, Oct. 2006, Santa Barbara, CA, USA.

    Google Scholar 

  8. Y. Chen, W. Xu, H. Sundaram, T. Rikakis and S.-M. Liu (2007). Media Adaptation Framework in Biofeedback System for Stroke Patient Rehabilitation, Proceedings of the 15th annual ACM international conference on Multimedia, ACM Press, Sep. 2007, Augsburg, Germany.

    Google Scholar 

  9. M. C. Cirstea, A. B. Mitnitski, A. G. Feldman and M. F. Levin (2003). Interjoint coordination dynamics during reaching in stroke. Experimental Brain Research 151(3): 289–300.

    Article  Google Scholar 

  10. M. L. Dombovy (2004). Understanding stroke recovery and rehabilitation: current and emerging approaches. Current Neurology and Neuroscience Reports 2004 4(1): 31–35.

    Google Scholar 

  11. P. Dourish (2001). Where the action is : the foundations of embodied interaction. MIT Press 0262041960 (alk. paper) Cambridge, Mass. ; London.

    Google Scholar 

  12. E. Dursun, N. Dursun and D. Alican (2004). Effects of biofeedback treatment on gait in children with cerebral palsy. Disability and Rehabilitation 26(2): 116–120.

    Article  Google Scholar 

  13. J. Gallichio and P. Kluding (2004). Virtual Reality in Stroke Rehabilitation: Review of the Emerging Research. Physical Therapy Reviews 9(4): 207–212.

    Article  Google Scholar 

  14. C. Ghez, T. Rikakis, R. L. Dubois and P. Cook (2000). An Auditory display system for aiding interjoint coordination, Proc. International Conference on Auditory Display, Apr. 2000, Atlanta, GA.

    Google Scholar 

  15. J. Gray (2003). What next ? : A dozen information-technology research goals. Journal of the ACM 50(1): 41–57.

    Article  Google Scholar 

  16. G. E. Gresham, P. W. Duncan and W. B. E. A. Stason (1996). Post-Stroke Rehabilitation/Clinical Practive Guideline. Aspen Publishers, Inc. 30-010-00 Gaithersburg, Maryland.

    Google Scholar 

  17. D. J. Grout and C. V. Palisca (2001). A history of western music. Norton 0393975274 New York.

    Google Scholar 

  18. H. Woldag, G. Waldmann, G. Heuschkel and H. Hummelsheim (2003). Is the repetitive training of complex hand and arm movements beneficial for motor recovery in stroke patients? Clinical Rehabilitation 2003 Nov 17(7): 723–730.

    Article  Google Scholar 

  19. X. He, W.-Y. Ma, O. King, M. Li and H. Zhang (2003). Learning and Inferring a Semantic Space from User's Relevance Feedback for Image Retrieval. IEEE Transactions on Circuits and Systems for Video Technology.

    Google Scholar 

  20. E. R. Hilgard and G. H. Bower (1975). Recent developments. Theories of learning(eds). Englewood Cliffs, N.J.,, Prentice-Hall: 550–605.

    Google Scholar 

  21. M. Holden and T. Dyar (2002). Virtual environment traing: a new tool for neurorehabilitation. Neurology Report 26(2): 62–72.

    Google Scholar 

  22. M. Holden, E. Todorov, J. Callahan and E. Bizzi (1999). Virtual environment training imporves motor performance in two patients with stroke: case report. Neurology Report 23(2): 57–67.

    Google Scholar 

  23. J. Hollan, E. Hutchins, D. Kirsh and A. Sutcliffe (2000). Distributed cognition: toward a new foundation for human-computer interaction research On the effective use and reuse of HCI knowledge. ACM Transaction Computing-Human Interaction 7(2): 174–196.

    Article  Google Scholar 

  24. E. Hutchins (1995). Cognition in the wild. MIT Press 0262082314 Cambridge, Mass.

    Google Scholar 

  25. H. Ishii and B. Ullmer (1997). Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press, 234–241,

    Google Scholar 

  26. H. Ishii, C. Wisneski, S. Brave, A. Dahley, M. Gorbet, B. Ullmer and P. Yarin (1998). ambientROOM: integrating ambient media with architectural space, CHI 98 conference summary on Human factors in computing systems, ACM Press, 173–174,

    Google Scholar 

  27. D. Jack, R. Boian, A. S. Merians, M. Tremaine, G. C. Burdea, S. V. Adamovich, M. Recce and H. Poizner (2001). Virtual reality-enhanced stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering 9: 308–318.

    Article  Google Scholar 

  28. R. V. Kenyon, J. Leigh and E. A. Keshner (2004). Considerations for the future development of virtual technology as a rehabilitation tool. J Neuroengineering Rehabilitation 1(1): 13.

    Article  Google Scholar 

  29. D. Kirsh (1995). The intelligent use of space. Artificial Intelligence 73(1–2): 31–68.

    Article  Google Scholar 

  30. Y.-F. Ma and H.-J. Zhang (2003). Contrast-based image attention analysis by using fuzzy growing, Proceedings of the eleventh ACM international conference on Multimedia, 1-58113-722-2, ACM Press, 374–381, Nov. 2003., Berkeley, CA, USA.

    Google Scholar 

  31. A. Mazalek, G. Davenport and H. Ishii (2002). Tangible viewpoints: a physical approach to multimedia stories, Proceedings of the tenth ACM international conference on Multimedia, ACM Press, 153--160,

    Google Scholar 

  32. J. Moreland and M. A. Thomson (1994). Efficacy of electromyographic biofeedback compared with conventional physical therapy for upper-extremity function in patients following stroke: a research overview and meta-analysis. Phys Ther 74(6): 534–543; discussion 544–537.

    Google Scholar 

  33. M. T. Schultheis and A. A. Rizzo (2001). The application of virtual reality technology for rehabilitation. Rehabilitation Psychology 46: 296–311.

    Article  Google Scholar 

  34. Y. Sun, H. Zhang, L. Zhang and M. Li (2002). A System for Home Photo Management and Processing, Proceedings of the 10th ACM international conference on Multimedia, pp. 81–82, Dec. 2002, Juan Les-Pins, France.

    Google Scholar 

  35. H. Sundaram and S.-F. Chang (2000). Determining Computable Scenes in Films and their Structures using Audio-Visual Memory Models, Proc. Of ACM International Conference on Multimedia 2000, pp. 95–104, Nov. 2000, Los Angeles, CA, USA.

    Google Scholar 

  36. G. Theocharous, K. Murphy and L. P. Kaelbling (2003). Representing hierarchical POMDPs as DBNs for multi-scale robot localization, Workshop on Reasoning about Uncertainty in Robotics, International Joint Conference on Artificial Intelligence, Acapulco, Mexico.

    Google Scholar 

  37. M. Tidwell, R. S. Johnston, D. Melville and T. A. Furness (1995). The virtual retinal display-a retinal scanning imaging system, Proceeding of Virtual Reality World' 95, 325–333, Heidelberg.

    Google Scholar 

  38. B. Ullmer and H. Ishii (2000). Emerging Frameworks for Tangible User Interfaces. IBM Systems Journal 39(3 & 4): pp. 915–931.

    Article  Google Scholar 

  39. J. P. Wann and J. D. Turnbull (1993). Motor skill learning in cerebral palsy: movement, action and computer-enhanced therapy. Baillieres Clinical Neurology 2(1): 15–28.

    Google Scholar 

  40. M. Weiser (1993). Some computer science issues in ubiquitous computing. Communication ACM 36(7): 75–84.

    Article  Google Scholar 

  41. D. White, K. Burdick, G. Fulk, J. Searleman and J. Carroll (2005). A virtual reality application for stroke patient rehabilitation, IEEE International Conference on Mechatronics & Automation Niagara Falls, July 2005, Canada.

    Google Scholar 

  42. S. L. Wolf, P. A. Catlin, S. Blanton, J. Edelman, N. Lehrer and D. Schroeder (1994). Overcoming limitations in elbow movement in the presence of antagonist hyperactivity. Physical Theraphy 74(9): 826–835.

    Google Scholar 

  43. S. H. You, S. H. Jang, Y. H. Kim, M. Hallett, S. H. Ahn, Y. H. Kwon, J. H. Kim and M. Y. Lee (2005). Virtual reality-induced cortical reorganization and associated locomotor recovery in chronic stroke: an experimenter-blind randomized study. Stroke 36(6): 1166–1171.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Arts Media and Engineering, Arizona State University, Tempe, AZ, USA

    Yinpeng Chen

Authors
  1. Yinpeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hari Sundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thanassis Rikakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Todd Ingalls
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Loren Olson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiping He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinpeng Chen .

Editor information

Editors and Affiliations

  1. Vision & Visualization Group, Sarnoff Corporation, Washington Rd. 201, Princeton, 08543, U.S.A.

    Ajay Divakaran

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Chen, Y., Sundaram, H., Rikakis, T., Ingalls, T., Olson, L., He, J. (2009). Experiential Media Systems – The Biofeedback Project. In: Divakaran, A. (eds) Multimedia Content Analysis. Signals and Communication Technology. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-76569-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-76569-3_8

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-76567-9

  • Online ISBN: 978-0-387-76569-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation