Modeling behaviors and lifestyle with online and social data for predicting and analyzing sleep and exercise quality

  • Mehrdad FarajtabarEmail author
  • Emre Kıcıman
  • Girish Nathan
  • Ryen W. White
Regular Paper


While recent data studies have focused on associations between sleep and exercise patterns as captured by digital fitness devices, it is known that sleep and exercise quality are affected by a much broader set of factors not captured by these devices, such as general lifestyle, eating, and stress. Here, we conduct a large-scale data study of exercise and sleep effects through an analysis of 8 months of exercise and sleep data for 20 k users, combined with search query logs, location information and aggregated social media data. We analyze factors correlated with better sleep and more effective exercise, and confirm these relationships through causal inference analysis. Further, we build linear models to predict individuals’ sleep and exercise quality. This analysis demonstrates the potential benefits of combining online and social data sources with data from health trackers, and is a potentially rich computational benchmark for health studies. We discuss the implications of our work for individuals, health practitioners and health systems.


User modeling Health tracker Sleep and exercise quality Online and social features Prediction 


Compliance with ethical standards

Conflict of interest

This work is funded 100% through the authors employment (a full-time internship for MF and full-time employment for the other authors) at Microsoft.

Research involving Human Participants

All user identifying information was anonymized. We did not examine search queries with personally-identifiable information or other sensitive information. All data access and analysis performed for this research was done in accordance with the published end-user license agreement, which was worded as follows: “By connecting to Microsoft Health you agree to allow Microsoft to share your data between Cortana and Microsoft Health, to provide valuable personal insights and recommendations to help you reach your fitness and wellness goals.” Visits to businesses were logged by Cortana to offer local services and is agreed to by users. Twitter data were not connected to specific users, but rather was based on publicly available tweets and were aggregated across many users who visited the business location. Our work was conducted offline, on data collected to support existing business operations, and did not influence the user experience. All data were anonymized and deidentified prior to analyses. Each user was represented by an anonymous identifier. We filtered search queries to only those matching a whitelist of keywords relevant to our study. The Ethics Advisory Committee at Microsoft Research considers these precautions sufficient for triggering the Common Rule, exempting this work from detailed ethics review.

Informed consent

Our data were collected between August 2015 and April 2016 and from individuals who agreed to link their Cortana data and Microsoft Health data (including Band device data) for use in generating additional insights or recommendations about their sleep or activity.


  1. 1.
    Whelton, S., Chin, A., Xin, X., He, J.: Effect of aerobic exercise on blood pressure: a meta-analysis of randomized, controlled trials. Ann. Intern. Med. 136, 493–503 (2002)Google Scholar
  2. 2.
    Petruzzello, S., Landers, D., Kubitz, A., Salazar, W.: A meta-analysis on the anxiety-reducing effects of acute and chronic exercise. Sports Med. 11, 143–182 (1991)Google Scholar
  3. 3.
    Cappuccio, F.P., D’Elia, L., Strazzullo, P., Miller, M.A.: Sleep duration and all-cause mortality: a systematic review and meta-analysis of prospective studies. Sleep 33, 585–592 (2010)Google Scholar
  4. 4.
    Reed, J., Ones, D.: The effect of acute aerobic exercise on positive activated affect: a meta-analysis. Psychol. Sport Exerc. 7, 477–514 (2006)Google Scholar
  5. 5.
    Fortier, E., Beaulieu, S., Ivers, H., Morin, C.: Insomnia and daytime cognitive performance: a meta-analysis. Sleep Med. Rev. 16, 83–94 (2012)Google Scholar
  6. 6.
    Rosekind, M., Gregory, K., Mallis, M., Brandt, S., Seal, B., Lerner, D.: The cost of poor sleep: workplace productivity loss and associated costs. J. Occup. Environ. Med. 52, 91–98 (2010)Google Scholar
  7. 7.
    Pilcher, J., Huffcutt, A.: Effects of sleep deprivation on performance: a meta-analysis. Sleep 19, 318–326 (1996)Google Scholar
  8. 8.
    Fox, K.R.: The influence of physical activity on mental well-being. Public Health Nutr. 2(3a), 411–418 (1999)Google Scholar
  9. 9.
    Standage, M., Gillison, F., Ntoumanis, N., Treasure, D.: Predicting students physical activity and health-related well-being: a prospective cross-domain investigation of motivation across school physical education and exercise settings. J. Sport Exerc. Psychol. 34, 37–60 (2012)Google Scholar
  10. 10.
    Fernández-Luque, L., Bau, T.: Health and social media: perfect storm of information. Healthc. Inform. Res. 21(2), 67–73 (2015)Google Scholar
  11. 11.
    Culotta, A.: Estimating county health statistics with twitter. In: SIGCHI (2014)Google Scholar
  12. 12.
    Crispim, C., Zimberg, I., Diniz, R., Tufik, S., Mello, M.: Relationship between food intake and sleep pattern in healthy individuals. J. Clin. Sleep Med. 7, 659 (2011)Google Scholar
  13. 13.
    Burgard, S., Ailshire, J.: Putting work to bed: stressful experiences on the job and sleep quality. J. Health Soc. Behav. 50, 476–492 (2009)Google Scholar
  14. 14.
    Tamaki, M., Bang, J., Watanabe, T., Sasaki, Y.: Night watch in one brain hemisphere during sleep associated with the first-night effect in humans. Curr. Biol. 26, 1190–1194 (2016)Google Scholar
  15. 15.
    Santillana, M., Nguyen, A., Dredze, M., Paul, M., Nsoesie, E., Brownstein, J.: Combining search, social media, and traditional data sources to improve influenza surveillance. PLoS Comput. Biol. 11, e1004513 (2015)Google Scholar
  16. 16.
    Smith, M., Wegener, S.: Measures of sleep: the insomnia severity index, medical outcomes study (mos) sleep scale, pittsburgh sleep diary (psd), and pittsburgh sleep quality index (psqi). Arthritis Care Res. 49, S184–S196 (2003)Google Scholar
  17. 17.
    Harvey, A.G., Stinson, K., Whitaker, K.L., Moskovitz, D., Virk, H.: The subjective meaning of sleep quality: a comparison of individuals with and without insomnia. Sleep 31(3), 383 (2008)Google Scholar
  18. 18.
    Schutte, S., Broch, L., Buysse, D., Sateia, M.: Clinical guideline for the evaluation and management of chronic insomnia in adults. J. Clin. Sleep Med. 4, 487 (2008)Google Scholar
  19. 19.
    American College of Sports Medicine et al.: ACSM’s Guidelines for Exercise Testing and Prescription. Lippincott Williams & Wilkins (2013)Google Scholar
  20. 20.
    Waldeck, M.R., Lambert, M.I.: Heart rate during sleep: implications for monitoring training status. J. Sports Sci. Med. 2(4), 133 (2003)Google Scholar
  21. 21.
    Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Chen, Z., Lin, M., Chen, F., Lane, N.D., Cardone, G., Wang, R., Li, T., Chen, Y., Choudhury, T., Campbell, A.T.: Unobtrusive sleep monitoring using smartphones. In: Pervasive Health (2013)Google Scholar
  23. 23.
    Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., Campbell, A.T.: Studentlife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In: UBICOMP, pp. 3–14 (2014)Google Scholar
  24. 24.
    Gu, W., Yang, Z., Shangguan, L., Sun, W., Jin, K., Liu, Y.: Intelligent sleep stage mining service with smartphones. In: UBICOMP (2014)Google Scholar
  25. 25.
    Hao, T., Xing, G., Zhou, G.: iSleep: unobtrusive sleep quality monitoring using smartphones. In: SenSys (2013)Google Scholar
  26. 26.
    Gu, W., Shangguan, L., Yang, Z., Liu, Y.: Sleep hunter: towards fine grained sleep stage tracking with smartphones. IEEE Trans. Mob. Comput. 15, 1514–1527 (2016)Google Scholar
  27. 27.
    Pernek, I., Kurillo, G., Stiglic, G., Bajcsy, R.: Recognizing the intensity of strength training exercises with wearable sensors. J. Biomed. Inf. 58, 145–155 (2015)Google Scholar
  28. 28.
    Spina, G., Huang, G., Vaes, A., Spruit, M., Amft, O.: COPDTrainer: a smartphone-based motion rehabilitation training system with real-time acoustic feedback. In: UBICOMP, pp. 597–606 (2013)Google Scholar
  29. 29.
    Bai, Y., Xu, B., Ma, Y., Sun, G., Zhao, Y.: Will you have a good sleep tonight?: sleep quality prediction with mobile phone. In: BODYNETS (2012)Google Scholar
  30. 30.
    Min, J., Doryab, A., Wiese, J., Amini, S., Zimmerman, J., Hong, J.: Toss’n’turn: smartphone as sleep and sleep quality detector. In: SIGCHI (2014)Google Scholar
  31. 31.
    Jayarajah, K., Radhakrishnan, M., Hoi, S., Misra, A.: Candy crushing your sleep. In: UBICOMP (2015)Google Scholar
  32. 32.
    Nguyen, A., Alqurashi, R., Halbower, A.C., Vu, T.: mSleepWatcher: Why didn’t i sleep well?. In: MCSE (2015)Google Scholar
  33. 33.
    Krishna, A., Mallick, M., Mitra, B.: Sleepsensei: an automated sleep quality monitor and sleep duration estimator. In: IoT of Health 2016 (2016)Google Scholar
  34. 34.
    Akbar, F., Weber, I.: # Sleep\_as\_android: feasibility of using sleep logs on twitter for sleep studies. In: ICHI (2016)Google Scholar
  35. 35.
    Wu, K., Ma, J., Zhumin, C., Ren, P.: Sleep quality evaluation of active microblog users. In: Asia-Pacific Web Conference (2015)Google Scholar
  36. 36.
    Jamison-Powell, S., Linehan, C., Daley, L., Garbett, A., Lawson, S: I can’t get no sleep: discussing# insomnia on twitter. In: SIGCHI (2012)Google Scholar
  37. 37.
    Peng, X., Luo, J., Glenn, C., Zhan, J., Liu, Y.: Large-scale sleep condition analysis using selfies from social media. arXiv:1704.06853 (2017)
  38. 38.
    Sathyanarayana, A., Joty, S., Fernandez-Luque, L., Ofli, F., Srivastava, J., Elmagarmid, A., Arora, T., Taheri, S.: Sleep quality prediction from wearable data using deep learning. JMIR Mhealth Uhealth 4, e125 (2016)Google Scholar
  39. 39.
    Lauderdale, D.S., Knutson, K.L., Yan, L., Liu, K., Rathouz, P.J.: Self-reported and measured sleep duration: how similar are they? Epidemiology 19, 838–845 (2008)Google Scholar
  40. 40.
    Natale, V., Léger, D., Bayon, V., Erbacci, A., Tonetti, L., Fabbri, M., Martoni, M.: The consensus sleep diary: quantitative criteria for primary insomnia diagnosis. Psychosom. Med. 77(4), 413–418 (2015)Google Scholar
  41. 41.
    Lineberger, M.D., Carney, C.E., Edinger, J.D., Means, M.K.: Defining insomnia: quantitative criteria for insomnia severity and frequency. Sleep 29(4), 479–485 (2006)Google Scholar
  42. 42.
    Ancoli-Israel, S., Cole, R., Alessi, C., Chambers, M., Moorcroft, W., Pollak, C.: The role of actigraphy in the study of sleep and circadian rhythms. Sleep 26, 342–392 (2003)Google Scholar
  43. 43.
    Walch, O.J., Cochran, A., Forger, D.B.: A global quantification of normal sleep schedules using smartphone data. Sci. Adv. 2, e1501705 (2016)Google Scholar
  44. 44.
    Althoff, T., Horvitz, E., White, R.W., Zeitzer. J.: Population-scale study of sleep and performance. In: WWW (2017)Google Scholar
  45. 45.
    Vargas, P., Flores, M., Robles, E.: Sleep quality and body mass index in college students: the role of sleep disturbances. J. Am. College Health 62, 535–541 (2014)Google Scholar
  46. 46.
    Weeks, D., Borrousch, S., Bowen, A., Hepler, L., Sandau, A., Slevin, F.: The influence of age and gender of an exercise model on self-efficacy and quality of therapeutic exercise performance in the elderly. Physiother. Theory Pract. 21, 137–146 (2005)Google Scholar
  47. 47.
    Dearman, D., Sohn, T., Truong, K.N.: Opportunities exist: continuous discovery of places to perform activities. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2429–2438. ACM (2011)Google Scholar
  48. 48.
    Benetka, J.R., Balog, K., Nørvåg, K.: Anticipating information needs based on check-in activity. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 41–50. ACM (2017)Google Scholar
  49. 49.
    Iachello, G., Smith, I., Consolvo, S., Abowd, G.D., Hughes, J., Howard, J., Potter, F., Scott, J., Sohn, T., Hightower, J., et al.: Control, deception, and communication: evaluating the deployment of a location-enhanced messaging service. In: International Conference on Ubiquitous Computing, pp. 213–231. Springer (2005)Google Scholar
  50. 50.
    Yang, D., Zhang, D., Zheng, V.W., Yu, Z.: Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Trans. Syst. Man Cybern. Syst. 45(1), 129–142 (2015)Google Scholar
  51. 51.
    Dearman, D., Truong, K.N.: Identifying the activities supported by locations with community-authored content. In: Proceedings of the 12th ACM International Conference on Ubiquitous Computing, pp. 23–32. ACM (2010)Google Scholar
  52. 52.
    Hossain, N., Hu, T., Feizi, R., White, A.M., Luo, J., Kautz, H.: Inferring fine-grained details on user activities and home location from social media: detecting drinking-while-tweeting patterns in communities. arXiv:1603.03181 (2016)
  53. 53.
    White, R.: Beliefs and biases in web search. In: SIGIR (2013)Google Scholar
  54. 54.
    White, R.W., Horvitz, E.: Studies of the onset and persistence of medical concerns in search logs. In: SIGIR, pp. 265–274 (2012)Google Scholar
  55. 55.
    Stubbe, A., Ringlstetter, C., Schulz, K.U.: Genre as noise: noise in genre. Int. J. Doc. Anal. Recognit. (IJDAR) 10, 199–209 (2007)Google Scholar
  56. 56.
    Kıcıman, E.: OMG, i have to tweet that! a study of factors that influence tweet rates. In: AAAI ICWSM (2012)Google Scholar
  57. 57.
    De Choudhury, M., Sharma, S., Kiciman, E.: Characterizing dietary choices, nutrition, and language in food deserts via social media. In: CSCW (2016)Google Scholar
  58. 58.
    Salathé, M., Vu, D., Khandelwal, S., Hunter, D.: The dynamics of health behavior sentiments on a large online social network. EPJ Data Sci. 2, 4 (2013)Google Scholar
  59. 59.
    Liu, B.: Sentiment analysis and subjectivity. Handb. Nat. Lang. Process. 2, 627–666 (2010)Google Scholar
  60. 60.
    Prier, K., Smith, M., Giraud, C., Hanson, C.: Identifying health-related topics on twitter. In: International Conference on Social Computing, Behavioral Modeling, Prediction (2011)Google Scholar
  61. 61.
    Ali, A., Magdy, W., Vogel, S.: A tool for monitoring and analyzing healthcare tweets. In: HSD Workshop, SIGIR. Citeseer (2013)Google Scholar
  62. 62.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. JMLR 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  63. 63.
    Czeisler, C.A.: Perspective: casting light on sleep deficiency. Nature 497, S13 (2013)Google Scholar
  64. 64.
    Uchida, S., Shioda, K., Morita, Y., Kubota, C., Ganeko, M., Takeda, N.: Exercise effects on sleep physiology. Front. Neurol. 3, 48 (2012)Google Scholar
  65. 65.
    Youngstedt, S., O’connor, P., Dishman, R.: The effects of acute exercise on sleep: a quantitative synthesis. Sleep 20, 203–214 (1997)Google Scholar
  66. 66.
    Ashe, M.C., Khan, K.M.: Exercise prescription. J. Am. Acad. Orthop. Surg. 12, 21–27 (2004)Google Scholar
  67. 67.
    Van Helder, T., Radomski, M.W.: Sleep deprivation and the effect on exercise performance. Sports Med. 7, 235–247 (1989)Google Scholar
  68. 68.
    Lyubomirsky, S., King, L., Diener, E.: The benefits of frequent positive affect: Does happiness lead to success? Psychol. Bull. 131, 803 (2005)Google Scholar
  69. 69.
    Imbens, G.W., Rubin, D.B.: Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press, Cambridge (2015)zbMATHGoogle Scholar
  70. 70.
    Rubin, D.B.: Causal inference using potential outcomes. J. Am. Stat. Assoc. 100, 322–331 (2011)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Georgia TechAtlantaUSA
  2. 2.Microsoft ResearchRedmondUSA
  3. 3.MicrosoftRedmondUSA

Personalised recommendations