Large-Scale Sleep Condition Analysis Using Selfies from Social Media

  • Xuefeng PengEmail author
  • Jiebo LuoEmail author
  • Catherine Glenn
  • Jingyao Zhan
  • Yuhan Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10354)


Sleep condition is closely related to an individual’s health. Poor sleep conditions such as sleep disorder and sleep deprivation affect one’s daily performance, and may also cause many chronic diseases. Many efforts have been devoted to monitoring people’s sleep conditions. However, traditional methodologies require sophisticated equipment and consume a significant amount of time. In this paper, we attempt to develop a novel way to predict individual’s sleep condition via scrutinizing facial cues as doctors would. Rather than measuring the sleep condition directly, we measure the sleep-deprived fatigue which indirectly reflects the sleep condition. Our method can predict a sleep-deprived fatigue rate based on a selfie provided by a subject. This rate is used to indicate the sleep condition. To gain deeper insights of human sleep conditions, we collected around 100,000 faces from selfies posted on Twitter and Instagram, and identified their age, gender, and race using automatic algorithms. Next, we investigated the sleep condition distributions with respect to age, gender, and race. Our study suggests among the age groups, fatigue percentage of the 0–20 youth and adolescent group is the highest, implying that poor sleep condition is more prevalent in this age group. For gender, the fatigue percentage of females is higher than that of males, implying that more females are suffering from sleep issues than males. Among ethnic groups, the fatigue percentage in Caucasian is the highest followed by Asian and African American.


Sleep condition prediction Fatigue analysis Social media Selfies 



We thank the support of New York State through the Goergen Institute for Data Science, and our corporate research sponsors Xerox and VisualDX.


  1. 1.
    Vedaldi, A., Fulkerson, B.: Vlfeat: an open and portable library of computer vision algorithms. In: Proceedings of the 18th ACM International Conference on Multimedia (MM 2010), New York, NY, USA, pp. 1469–1472 (2010)Google Scholar
  2. 2.
    Anon: 2015 Sleep in America Poll. Sleep Health 1, 2 (2015)Google Scholar
  3. 3.
    Griffith, C., Mahadevan, S.: Sleep deprivation effect on human performance: a meta-analysis approach (PSAM-0010). In: Proceedings of International Conference on Probabilistic Safety Assessment & Management (PSAM), pp. 1488–1496Google Scholar
  4. 4.
    Zhou, E., Fan, H., Cao, Z., Jiang, Y., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 386–391 (2013)Google Scholar
  5. 5.
    Han, H., Jain, A.K.: Age, Gender and Race Estimation from Unconstrained Face Images. Michigan State University, Technical Report (2014)Google Scholar
  6. 6.
    Axelsson, J., Sundelin, T., Ingre, M., Van Someren, E.J.W., Olsson, A., Lekander, M.: Beauty sleep: experimental study on the perceived health and attractiveness of sleep deprived people. BMJ 341, c6614 (2010)CrossRefGoogle Scholar
  7. 7.
    Desforges, J.F., Prinz, P.N., Vitiello, M.V., Raskind, M.A., Thorpy, M.J.: Sleep disorders and aging. New England J. Med. 323(8), 520–526 (1990)CrossRefGoogle Scholar
  8. 8.
    Lack, L., Wright, H.: Pittsburgh sleep quality index. In: Encyclopedia of Quality of Life and Well-Being Research, pp. 4814–4816 (2014)Google Scholar
  9. 9.
    He, L., Murphy, L., Luo, J.: Using social media to promote STEM education: matching college students with role models. In: Berendt, B., Bringmann, B., Fromont, É., Garriga, G., Miettinen, P., Tatti, N., Tresp, V. (eds.) ECML PKDD 2016. LNCS, vol. 9853, pp. 79–95. Springer, Cham (2016). doi: 10.1007/978-3-319-46131-1_17 CrossRefGoogle Scholar
  10. 10.
    Manikonda, L., De Choudhury, M.: Modeling and understanding visual attributes of mental health disclosures in social media. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI) (to appear, 2017)Google Scholar
  11. 11.
    Tripathi, M.: Technical notes for digital polysomnography recording in sleep medicine practice. Ann. Indian Acad. Neurol. 11(2), 129 (2008)CrossRefGoogle Scholar
  12. 12.
    Marascuilo, L.A.: Large-sample multiple comparisons. Psychol. Bull. 65(5), 280–290 (1966)CrossRefGoogle Scholar
  13. 13.
    Swain, M.G.: Fatigue in chronic disease. Clin. Sci. 99(1), 1 (2000)CrossRefGoogle Scholar
  14. 14.
    Gradisar, M., Gardner, G., Dohnt, H.: Recent worldwide sleep patterns and problems during adolescence: a review and meta-analysis of age, region, and sleep. Sleep Med. 12(2), 110–118 (2011)CrossRefGoogle Scholar
  15. 15.
    Wang, N., Gao, X., Tao, D., Li, X.: Facial feature point detection: a comprehensive survey. CoRR (2014)Google Scholar
  16. 16.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE Computer Vision and Pattern Recognition (2001)Google Scholar
  17. 17.
    Pang, R., Baretto, A., Kautz, H., Luo, J.: Monitoring adolescent alcohol use via multimodal data analysis in social multimedia. In: Proceedings of IEEE Big Data Conference on Special Session on Intelligent Mining (2015)Google Scholar
  18. 18.
    Abdullah, S., Murnane, E.L., Costa, J.M.R., Choudhury, T.: Collective smile: Measuring societal happiness from geolocated images. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work & Social Computing (2015)Google Scholar
  19. 19.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. Adv. Neural. Inf. Process. Syst. 25, 2960–2968 (2012)Google Scholar
  20. 20.
    Sundelin, T., Lekander, M., Kecklund, G., Van Someren, E.J.W., Olsson, A., Axelsson, J.: Cues of fatigue: effects of sleep deprivation on facial appearance. Sleep, January 2013Google Scholar
  21. 21.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 2003 (2003)CrossRefGoogle Scholar
  22. 22.
    Wu, Y., Yuan, J., You, Q., Luo, J.: The effect of pets on happiness: a data-driven approach via large-scale social media. In: Proceedings of IEEE Big Data Conference (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of RochesterRochesterUSA
  2. 2.Department of PsychologyUniversity of RochesterRochesterUSA

Personalised recommendations