Deep Reinforcement Learning Based Personalized Health Recommendations

  • Jayraj Mulani
  • Sachin Heda
  • Kalpan Tumdi
  • Jitali PatelEmail author
  • Hitesh Chhinkaniwala
  • Jigna Patel
Part of the Studies in Big Data book series (SBD, volume 68)


In this age of informatics, it has become paramount to provide personalized recommendations in order to mitigate the effects of information overload. This domain of biomedical and health care informatics is still untapped as far as personalized recommendations are concerned. Most of the existing recommender systems have, to some extent, not been able to address sparsity of data and non-linearity of user-item relationships among other issues. Deep reinforcement learning systems can revolutionize the recommendation architectures because of its ability to use non-linear transformations, representation learning, sequence modelling and flexibility for implementation of these architectures. In this paper, we present a deep reinforcement learning based approach for complete health care recommendations including medicines to take, doctors to consult, nutrition to acquire and activities to perform that consists of exercises and preferable sports. We try to exploit an “Actor-Critic” model for enhancing the ability of the model to continuously update information seeking strategies based on user’s real-time feedback. Health industry usually deals with long-term issues. Traditional recommender systems fail to consider the long-term effects, hence failing to capture dynamic sentiments of people. This approach treats the process of recommendation as a sequential decision process, which addresses the aforementioned issues. It is estimated that over 700 million people will possess wearable devices that will monitor every step they take. Data collected with these smart devices, combined with other sources like, Electronic Health Records, Nutrition Data and data collected from surveys can be processed using Big Data Analysis tools, and fed to recommendation systems to generate desirable recommendations. These data, after encoding (state) into appropriate format, will be fed to the Actor network, which will learn a policy for prioritizing a particular recommendation (action). The action, state pair is fed to the critic network, which generates a reward associated with the action, state pair. This reward is used to update the policy of the Actor network. The critic network learns using a pre-defined Expected Reward. Hence, we find that using tools for Big Data Analytics, and intelligent approaches like Deep Reinforcement Learning can significantly improve recommendation results for health care, aiding in creating seamlessly personalized systems.


Big data Deep reinforcement learning Recommendation systems Biomedical and health informatics Actor critic model Electronic health records 


  1. 1.
    Elgendy, N., Elragal, A.: Big data analytics: a literature review paper. In: Industrial Conference on Data Mining, pp. 214–227. Springer, Cham (2014, July)CrossRefGoogle Scholar
  2. 2.
    Pan, C., Li, W.: Research paper recommendation with topic analysis. In: 2010 International Conference On Computer Design and Applications, vol. 4, pp. V4–264. IEEE (2010, June)Google Scholar
  3. 3.
    Han, Q., Ji, M., de Troya, I.M.D.R., Gaur, M., Zejnilovic, L.: A hybrid recommender system for patient-doctor matchmaking in primary care. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 481–490. IEEE (2018, Oct)Google Scholar
  4. 4.
    Wiesner, M., Pfeifer, D.: Health recommender systems: concepts, requirements, technical basics and challenges. Int. J. Environ. Res. Public Health 11(3), 2580–2607 (2014). Scholar
  5. 5.
    Patgiri, R., Ahmed, A.: Big data: the v’s of the game changer paradigm. In: 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pp. 17–24. IEEE (2016, Dec)Google Scholar
  6. 6.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. (2013)
  7. 7.
    Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., De Freitas, N.: Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581. (2015)
  8. 8.
    Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems, pp. 1057–1063. (2000)Google Scholar
  9. 9.
    Zhao, X., Zhang, L., Ding, Z., Yin, D., Zhao, Y., Tang, J.: Deep reinforcement learning for list-wise recommendations. CoRR, vol. abs/1801.00209. (2018)Google Scholar
  10. 10.
    Mokdad, A.H., Ford, E.S., Bowman, B.A., Dietz, W.H., Vinicor, F., Bales, V.S., Marks, J.S.: Prevalence of obesity, diabetes, and obesity-related health risk factors, 2001. JAMA 289(1), 76–79 (2003)CrossRefGoogle Scholar
  11. 11.
    Montañez, C.A.C., Fergus, P., Hussain, A., Al-Jumeily, D., Abdulaimma, B., Hind, J., Radi, N.: Machine learning approaches for the prediction of obesity using publicly available genetic profiles. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2743–2750. IEEE (2017, May)Google Scholar
  12. 12.
    Sharmila, R., Chellammal, S.: A conceptual method to enhance the prediction of heart diseases using the data techniques. Int. J. Comput. Sci. Eng. (2018, May)Google Scholar
  13. 13.
    Sisodia, D., Sisodia, D.S.: Prediction of diabetes using classification algorithms. Proc. Comput. Sci. 132, 1578–1585 (2018)CrossRefGoogle Scholar
  14. 14.
    Sindhuja, D., Priyadarsini, R.J.: A survey on classification techniques in data mining for analyzing liver disease disorder. Int. J. Comput. Sci. Mob. Comput. 5(5), 483–488 (2016)Google Scholar
  15. 15.
    Finkelstein, J.: Machine learning approaches to personalize early prediction of asthma exacerbations. Ann. New York Acad. Sci. 1387(1), 153–165 (2017)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Jammeh, E.A., Camille, B.C., Stephen, W.P., Escudero, J., Anastasiou, A., Zhao, P., Chenore, T., Zajicek, J., Ifeachor, E.: Machine-learning based identification of undiagnosed dementia in primary care: a feasibility study. BJGP open 2(2). bjgpopen18X101589. (2018)Google Scholar
  17. 17.
    Ozkan, I.A., Koklu, M., Sert, I.U.: Diagnosis of urinary tract infection based on artificial intelligence methods. Comput. Methods Progr. Biomed. 166, 51–59 (2018)CrossRefGoogle Scholar
  18. 18.
    Chae, S., Kwon, S., Lee, D.: Predicting infectious disease using deep learning and big data. Int. J. Environ. Res. Public Health 15(8), 1596 (2018)CrossRefGoogle Scholar
  19. 19.
    Liu, F., Tang, R., Li, X., Zhang, W., Ye, Y., Chen, H., et al.: Deep reinforcement learning based recommendation with explicit user-item interactions modeling. arXiv preprint arXiv:1810.12027. (2018)
  20. 20.
    Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., et al.: Hindsight experience replay. In: Advances in Neural Information Processing Systems, pp. 5048–5058. (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Jayraj Mulani
    • 1
  • Sachin Heda
    • 1
  • Kalpan Tumdi
    • 1
  • Jitali Patel
    • 1
    Email author
  • Hitesh Chhinkaniwala
    • 2
  • Jigna Patel
    • 1
  1. 1.Department of Computer Science and EngineeringInstitute of Technology Nirma UniversityAhmedabadIndia
  2. 2.Adani Institute of Infrastructure EngineeringAhmedabadIndia

Personalised recommendations