Personalized Emotion-Aware Video Streaming for the Elderly

  • Yi DongEmail author
  • Han Hu
  • Yonggang Wen
  • Han Yu
  • Chunyan Miao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10914)


We consider the problem of video therapy services for the elderly based on their current emotional status. Given long hours watching TV in the elder population, most of the existing TV services are not geared for them. The elderly cannot tolerate complexity and negativity due to decline in cognitive abilities. In addition, the program is not adapted to the user’s current emotional status. As a result, existing TV services can not achieve optimal performance across a broad set of user types and context. To provide content tailored to individual needs, and interests of the elderly, caregivers have to select an appropriate program manually. However, this can not scale well due to shortage of caregivers and high monetary cost. We present the personalized emotion-aware video streaming system, a redesign of conventional TV system to provide appropriate program flexibly, efficiently and responsively. Our proposed architecture adds video affective profiling, real-time emotion detection and Markov decision process based video program generation to the streaming service to this end. We present a complete implementation of our design. Trace-driven simulation has shown the effectiveness of our system.



This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its IDM Futures Funding Initiative; Nanyang Technological University, Nanyang Assistant Professorship and the Interdisciplinary Graduate School, Nanyang Technological University, Singapore.


  1. 1.
    Caserta, M.S., Lund, D.A.: Video respite® in an alzheimer’s care center: group versus solitary viewing. Activities, Adapt. Aging 27(1), 13–28 (2003)CrossRefGoogle Scholar
  2. 2.
    National Collaborating Centre: Dementia: A nice-scie guideline on supporting people with dementia and their carers in health and social care. British Psychological Society (2007)Google Scholar
  3. 3.
    Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)CrossRefGoogle Scholar
  4. 4.
    Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in opensmile, the munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM International Conference on Multimedia, MM 2013, pp. 835–838. ACM, New York (2013)Google Scholar
  5. 5.
    Gao, G., Hu, H., Wen, Y., Westphal, C.: Resource provisioning and profit maximization for transcoding in clouds: a two-timescale approach. IEEE Trans. Multimed. 19(4), 836–848 (2017)CrossRefGoogle Scholar
  6. 6.
    Gao, G., Wen, Y., Hu, H.: Qdlcoding: Qos-differentiated low-cost video encoding scheme for online video service. In: IEEE INFOCOM 2017 - IEEE Conference on Computer Communications, pp. 1–9, May 2017Google Scholar
  7. 7.
    Gao, G., Zhang, W., Wen, Y., Wang, Z., Zhu, W.: Towards cost-efficient video transcoding in media cloud: insights learned from user viewing patterns. IEEE Trans. Multimed. 17(8), 1286–1296 (2015)CrossRefGoogle Scholar
  8. 8.
    Hu, H., Wen, Y., Niyato, D.: Spectrum allocation and bitrate adjustment for mobile social video sharing: potential game with online qos learning approach. IEEE J. Sel. Areas Commun. 35(4), 935–948 (2017)CrossRefGoogle Scholar
  9. 9.
    Hu, H., Jin, Y., Wen, Y., Chua, T.S., Li, X.: Toward a biometric-aware cloud service engine for multi-screen video applications. In: Proceedings of the 2014 ACM Conference on SIGCOMM. SIGCOMM 2014, pp. 581–582. ACM, New York (2014)Google Scholar
  10. 10.
    Koelstra, S., Muhl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: Deap: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)CrossRefGoogle Scholar
  11. 11.
    Lang, P.J.: The emotion probe: studies of motivation and attention. Am. Psychol. 50(5), 372 (1995)CrossRefGoogle Scholar
  12. 12.
    Li, B., Wang, Z., Liu, J., Zhu, W.: Two decades of internet video streaming: a retrospective view. ACM Trans. Multimedi. Comput., Commun. Appl. (TOMM) 9(1s), 33 (2013)CrossRefGoogle Scholar
  13. 13.
    Lund, D.A., Hill, R.D., Caserta, M.S., Wright, S.D.: Video respite: an innovative resource for family, professional caregivers, and persons with dementia. Gerontologist 35(5), 683–687 (1995)CrossRefGoogle Scholar
  14. 14.
    Lux, M., Chatzichristofis, S.A.: Lire: lucene image retrieval: an extensible java CBIR library. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 1085–1088. ACM (2008)Google Scholar
  15. 15.
    Robinson, J., Godbey, G.: Time for Life: The Surprising Ways Americans Use Their Time. Penn State Press (2010)Google Scholar
  16. 16.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge, MA (2017)Google Scholar
  17. 17.
    Zhu, Y., Hanjalic, A., Redi, J.A.: QoE prediction for enriched assessment of individual video viewing experience. In: Proceedings of the 2016 ACM on Multimedia Conference, pp. 801–810. ACM (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Yi Dong
    • 1
    Email author
  • Han Hu
    • 2
  • Yonggang Wen
    • 2
  • Han Yu
    • 1
  • Chunyan Miao
    • 1
    • 2
  1. 1.NTU-UBC Research Center of Excellence in Active Living for the Elderly, IGSNanyang Technological UniversitySingaporeSingapore
  2. 2.School of Computer Science and EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations