A systematic review into the assessment of medical apps: motivations, challenges, recommendations and methodological aspect


Recent years have shown significantly pervasive interest in mobile applications (hereinafter “apps”). The number and popularity of these apps are dramatically increasing. Even though mobile apps are diverse, countless ones are available through many platforms. Some of these apps are not useful nor do they possess rich content, which benefits end users as expected, especially in medical-related cases. This research aims to review and analyze articles associated with medical app assessment across different platforms. This research also aimed to provide the best practices and identify the academic challenges, motivations and recommendations related with quality assessments. In addition, a methodological approach followed in previous research in this domain was also discussed to give some insights for future comers with what to expect. We systematically searched articles on topics related to medical app assessment. The search was conducted on five major databases, namely, Science Direct, Springer, Web of Science, IEEE Xplore and PubMed from 2009 to September 2019. These indices were considered sufficiently extensive and reliable to cover our scope of the literature. Articles were selected on the basis of our inclusion and exclusion criteria (n = 72). Medical app assessment is considered a major topic which warrants attention. This study emphasizes the current standpoint and opportunities for research in this area and boosts additional efforts towards the understanding of this research field.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. 1.

    Holl K, Elberzhager F. Mobile application quality assurance. Adv Comput. 2019;112:1–77 Elsevier.

    Google Scholar 

  2. 2.

    Bahadori S, Wainwright TW, Ahmed OH. Smartphone apps for total hip replacement and total knee replacement surgery patients: a systematic review. Disabil Rehabil. 2018:1–6.

  3. 3.

    Orsini G, Bade D, Lamersdorf W. Context-aware computation offloading for mobile cloud computing: requirements analysis, survey and design guideline. Procedia Comput Sci. 2015;56:10–7.

    Google Scholar 

  4. 4.

    Beam L, Burrows B, Dobey Z, Poser R, Sopko M. RecycMe: the Ohio State University recycling phone application. 2013.

  5. 5.

    Ouhbi S, Fernández-Alemán JL, Pozo JR, El Bajta M, Toval A, Idri A. Compliance of blood donation apps with mobile OS usability guidelines. J Med Syst. 2015;39(6):63.

    Google Scholar 

  6. 6.

    Kim MS, Park JH, Park K-Y. Development and effectiveness of a drug dosage calculation training program using cognitive loading theory based on smartphone application. J Korean Acad Nurs. 2012;42(5):689–98.

    Google Scholar 

  7. 7.

    Nayebi F, Desharnais J-M, Abran A. The state of the art of mobile application usability evaluation. In 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE; 2012, p. 1–4.

  8. 8.

    Chen Z-S, Li R, Chen X, Xu H. A survey study on consumer perception of mobile-commerce applications. Procedia Environ Sci. 2011;11:118–24.

    Google Scholar 

  9. 9.

    Tang AK. A systematic literature review and analysis on mobile apps in m-commerce: implications for future research. Electron Commer Res Appl. 2019;37:100885.

    Google Scholar 

  10. 10.

    Darras KE, van Merriënboer JJG, Toom M, Roberson ND, de Bruin ABH, Nicolaou S, et al. Developing the evidence base for M-learning in undergraduate radiology education: identifying learner preferences for Mobile apps. Can Assoc Radiol J. 2019;70(3):320–6.

    Google Scholar 

  11. 11.

    Albrecht U-V, Hillebrand U, von Jan U. Relevance of trust marks and CE labels in German-language store descriptions of health apps: analysis. JMIR mHealth and uHealth. 2018;6(4):e10394.

    Google Scholar 

  12. 12.

    Anderson N, Steele J, O'Neill L-A, Harden LA. Pokémon Go: mobile app user guides. Br J Sports Med. 2016;2016:096762.

    Google Scholar 

  13. 13.

    Ahtinen A, Isomursu M, Huhtala Y, Kaasinen J, Salminen J, Häkkilä J. Tracking outdoor sports–user experience perspective. In: Aarts E, Crowley JL, Ruyter B, Gerhäuser H, Pflaum A, Schmidt J, Wichert R, editors. European Conference on Ambient Intelligence. Berlin: Springer; 2008. p. 192–209.

    Google Scholar 

  14. 14.

    Lior LN. Writing for interaction: crafting the information experience for web and software apps. Newnes. 2013, Writing Text for Interaction.

  15. 15.

    Cortimiglia MN, Ghezzi A, Renga F. Mobile applications and their delivery platforms. IT Prof. 2011;13(5):51–6.

    Google Scholar 

  16. 16.

    Beimborn D, Palitza M. Enterprise app stores for mobile applications-development of a benefits framework. 2013.

    Google Scholar 

  17. 17.

    Harman M, Jia Y, Zhang Y. App store mining and analysis: MSR for app stores. In Proceedings of the 9th IEEE Working Conference on Mining Software Repositories. IEEE Press; 2012, p. 108–111.

  18. 18.

    Freier A. App revenue reaches $92.1 billion in 2018 driven by mobile gaming apps. Business Apps. 2018;13(09).

  19. 19.

    Cheney S, Thompson E. The 2017–2022 app economy forecast: 6 billion devices, $157 billion in spend & more. App Annie. 2018.

  20. 20.

    Chen J, Cade JE, Allman-Farinelli M. The most popular smartphone apps for weight loss: a quality assessment. JMIR mHealth uHealth. 2015;3(4):e104.

    Google Scholar 

  21. 21.

    Kim BY, Sharafoddini A, Tran N, Wen EY, Lee J. Consumer mobile apps for potential drug-drug interaction check: systematic review and content analysis using the mobile app rating scale (MARS). JMIR mHealth uHealth. 2018;6(3):e74.

    Google Scholar 

  22. 22.

    Haskins BL, Lesperance D, Gibbons P, Boudreaux ED. A systematic review of smartphone applications for smoking cessation. Transl Behav Med. 2017;7(2):292–9.

    Google Scholar 

  23. 23.

    Ali EE, Teo AKS, Goh SXL, Chew L, Yap KY-L. MedAd-AppQ: a quality assessment tool for medication adherence apps on iOS and android platforms. Res Soc Adm Pharm. 2018;14(12):1125–33.

    Google Scholar 

  24. 24.

    Muntaner-Mas A, Martinez-Nicolas A, Lavie CJ, Blair SN, Ross R, Arena R, et al. A systematic review of fitness apps and their potential clinical and sports utility for objective and remote assessment of cardiorespiratory fitness. Sports Med. 2019;49(4):587–600.

    Google Scholar 

  25. 25.

    Zhao J, Freeman B, Li M. How do infant feeding apps in China measure up? A Content Quality Assessment. JMIR mHealth uHealth. 2017;5(12):e186.

    Google Scholar 

  26. 26.

    Weekly T, Walker N, Beck J, Akers S, Weaver M. A review of apps for calming, relaxation, and mindfulness interventions for pediatric palliative care patients. Children. 2018;5(2):16.

    Google Scholar 

  27. 27.

    Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. 2015;17(8):e198.

    Google Scholar 

  28. 28.

    Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 2012;10(1):144.

    Google Scholar 

  29. 29.

    Zhang MW, Ho RC, Hawa R, Sockalingam S. Analysis of the information quality of bariatric surgery smartphone applications using the silberg scale. Obes Surg. 2016;26(1):163–8.

    Google Scholar 

  30. 30.

    Bergeron D, et al. Mobile applications in neurosurgery: a systematic review, quality audit, and survey of Canadian neurosurgery residents. World neurosurg. 2019.

  31. 31.

    van Galen L, Xu X, Koh M, Thng S, Car J. Eczema apps conformance with clinical guidelines: a systematic assessment of functions, tools and content. Br J Dermatol. 2019.

  32. 32.

    Larco A, Enríquez F, Luján-Mora S. Review and evaluation of special education iOS Apps using MARS. In 2018 IEEE World Engineering Education Conference (EDUNINE). IEEE; 2018, p. 1–6.

  33. 33.

    Larco A, Montenegro C, Luján-Mora S, Quality improvement criteria of apps in Spanish for people with disabilities. In 2018 4th International Conference on Information Management (ICIM). IEEE; 2018, p. 260–264.

  34. 34.

    Alamoodi A, et al. A review of data analysis for early-childhood period: taxonomy, motivations, challenges, recommendation, and methodological aspects. IEEE Access. 2019;7:51069–103.

    Google Scholar 

  35. 35.

    Burgers C, Brugman BC, Boeynaems A. Systematic literature reviews: four applications for interdisciplinary research. J Pragmat. 2019;145:102–9.

    Google Scholar 

  36. 36.

    Loureiro SMC, Romero J, Bilro RG. Stakeholder engagement in co-creation processes for innovation: a systematic literature review and case stud. J Bus Res. 2019.

  37. 37.

    Kushwah S, Dhir A, Sagar M, Gupta B. Determinants of organic food consumption. A systematic literature review on motives and barriers. Appetite. 2019;143:104402.

    Google Scholar 

  38. 38.

    Daigneault P-M, Jacob S, Ouimet M. Using systematic review methods within a Ph. D. Dissertation in political science: challenges and lessons learned from practice. Int J Soc Res Methodol. 2014;17(3):267–83.

    Google Scholar 

  39. 39.

    Ain N, Vaia G, DeLone WH, Waheed M. Two decades of research on business intelligence system adoption, utilization and success – a systematic literature review. Decis Support Syst. 2019;125:113113.

    Google Scholar 

  40. 40.

    Dani VS, Freitas CMDS, Thom LH. Ten years of visualization of business process models: a systematic literature review. Comput Stand Inter. 2019.

  41. 41.

    Owens OL, Beer JM, Reyes LI, Thomas TL. Systematic review of commercially available mobile phone applications for prostate cancer education. Am J Men’s Health. 2019;13(1):1557988318816912.

    Google Scholar 

  42. 42.

    Huckvale K, Morrison C, Ouyang J, Ghaghda A, Car J. The evolution of mobile apps for asthma: an updated systematic assessment of content and tools. BMC Med. 2015;13(1):58.

    Google Scholar 

  43. 43.

    El-Gayar O, Timsina P, Nawar N, Eid W. Mobile applications for diabetes self-management: status and potential. J Diabetes Sci Technol. 2013;7(1):247–62.

    Google Scholar 

  44. 44.

    Huang Z, Lum E, Jimenez G, Semwal M, Sloot P, Car J. Medication management support in diabetes: a systematic assessment of diabetes self-management apps. BMC Med. 2019;17(1):127.

    Google Scholar 

  45. 45.

    Dayer L, Heldenbrand S, Anderson P, Gubbins PO, Martin BC. Smartphone medication adherence apps: potential benefits to patients and providers. J Am Pharm Assoc. 2013;53(2):172–81.

    Google Scholar 

  46. 46.

    Nguyen AD, Baysari MT, Kannangara DRW, Tariq A, Lau AYS, Westbrook JI, et al. Mobile applications to enhance self-management of gout. Int J Med Inform. 2016;94:67–74.

    Google Scholar 

  47. 47.

    Payo RM, Harris J, Armes J. Prescribing fitness apps for people with cancer: a preliminary assessment of content and quality of commercially available apps. J Cancer Surviv. 2019;13:1–9.

    Google Scholar 

  48. 48.

    Rusin M, Årsand E, Hartvigsen G. Functionalities and input methods for recording food intake: a systematic review. Int J Med Inform. 2013;82(8):653–64.

    Google Scholar 

  49. 49.

    Bhattarai P, Newton-John T, Phillips JL. Quality and usability of arthritic pain self-management apps for older adults: a systematic review. Pain Med. 2017;19(3):471–84.

    Google Scholar 

  50. 50.

    Devan H, Farmery D, Peebles L, Grainger R. Evaluation of self-management support functions in apps for people with persistent pain: systematic review. JMIR mHealth uHealth. 2019;7(2):e13080.

    Google Scholar 

  51. 51.

    Metelmann B, Metelmann C, Schuffert L, Hahnenkamp K, Brinkrolf P. Medical correctness and user friendliness of available apps for cardiopulmonary resuscitation: systematic search combined with guideline adherence and usability evaluation. JMIR mHealth uHealth. 2018;6(11):e190.

    Google Scholar 

  52. 52.

    Taki S, Campbell KJ, Russell CG, Elliott R, Laws R, Denney-Wilson E. Infant feeding websites and apps: a systematic assessment of quality and content. Interact J Med Res. 2015;4(3):e18.

    Google Scholar 

  53. 53.

    Lee H, Sullivan SJ, Schneiders AG, Ahmed OH, Balasundaram AP, Williams D, et al. Smartphone and tablet apps for concussion road warriors (team clinicians): a systematic review for practical users. Br J Sports Med. 2015;49(8):499–505.

    Google Scholar 

  54. 54.

    Moglia ML, Nguyen HV, Chyjek K, Chen KT, Castaño PM. Evaluation of smartphone menstrual cycle tracking applications using an adapted APPLICATIONS scoring system. Obstet Gynecol. 2016;127(6):1153–60.

    Google Scholar 

  55. 55.

    Brouard B, Bardo P, Bonnet C, Mounier N, Vignot M, Vignot S. Mobile applications in oncology: is it possible for patients and healthcare professionals to easily identify relevant tools? Ann Med. 2016;48(7):509–15.

    Google Scholar 

  56. 56.

    Latorre GF, de Fraga R, Seleme MR, Mueller CV, Berghmans B. An ideal e-health system for pelvic floor muscle training adherence: systematic review. Neurourol Urodyn. 2019;38(1):63–80.

    Google Scholar 

  57. 57.

    Luo D, Wang P, Lu F, Elias J, Sparks JA, Lee YC. Mobile apps for individuals with rheumatoid arthritis: a systematic review. JCR: J Clin Rheumatol. 2019;25(3):133–41.

    Google Scholar 

  58. 58.

    Sun C, Malcolm JC, Wong B, Shorr R, Doyle M-A. Improving glycemic control in adults and children with type 1 diabetes with the use of smartphone-based mobile applications: a systematic review. Can J Diabetes. 2019;43(1):51–8. e3.

    Google Scholar 

  59. 59.

    Bry LJ, Chou T, Miguel E, Comer JS. Consumer smartphone apps marketed for child and adolescent anxiety: a systematic review and content analysis. Behav Ther. 2018;49(2):249–61.

    Google Scholar 

  60. 60.

    Milani P, Coccetta CA, Rabini A, Sciarra T, Massazza G, Ferriero GJP. Mobile smartphone applications for body position measurement in rehabilitation: a review of goniometric tools. PM&R. 2014;6(11):1038–43.

    Google Scholar 

  61. 61.

    Kalz M, et al. Smartphone apps for cardiopulmonary resuscitation training and real incident support: a mixed-methods evaluation study. J Med Internet Res. 2014;16(3):e89.

    Google Scholar 

  62. 62.

    Piran P, et al. Medical mobile applications for stroke survivors and caregivers. J Stroke Cerebrovasc Dis. 2019;28(11):104318.

    Google Scholar 

  63. 63.

    Adam A, Hellig JC, Perera M, Bolton D, Lawrentschuk N. ‘Prostate Cancer risk Calculator’ mobile applications (apps): a systematic review and scoring using the validated user version of the Mobile application rating scale (uMARS). World J Urol. 2018;36(4):565–73.

    Google Scholar 

  64. 64.

    Schumer H, Amadi C, Joshi A. Evaluating the dietary and nutritional apps in the Google play store. Healthcare Inform Res. 2018;24(1):38–45.

    Google Scholar 

  65. 65.

    Torous J, Levin ME, Ahern DK, Oser ML. Cognitive behavioral mobile applications: clinical studies, marketplace overview, and research agenda. Cogn Behav Pract. 2017;24(2):215–25.

    Google Scholar 

  66. 66.

    Anthony Berauk VL, Murugiah MK, Soh YC, Chuan Sheng Y, Wong TW, Ming LC. Mobile health applications for caring of older people: review and comparison. Ther Innov Regul Sci. 2018;52(3):374–82.

    Google Scholar 

  67. 67.

    Meghani SH, MacKenzie MA, Morgan B, Kang Y, Wasim A, Sayani S. Clinician-targeted mobile apps in palliative care: a systematic review. J Palliat Med. 2017;20(10):1139–47.

    Google Scholar 

  68. 68.

    Rincon E, Monteiro-Guerra F, Rivera-Romero O, Dorronzoro-Zubiete E, Sanchez-Bocanegra CL, Gabarron E. Mobile phone apps for quality of life and well-being assessment in breast and prostate cancer patients: systematic review. JMIR mHealth uHealth. 2017;5(12):e187.

    Google Scholar 

  69. 69.

    Chen E, Mangone ER. A systematic review of apps using mobile criteria for adolescent pregnancy prevention (mCAPP). JMIR mHealth uHealth. 2016;4(4):e122.

    Google Scholar 

  70. 70.

    Alyami M, Giri B, Alyami H, Sundram F. Social anxiety apps: a systematic review and assessment of app descriptors across mobile store platforms. Evid-Based Ment Health. 2017;20(3):65–70.

    Google Scholar 

  71. 71.

    Park JYE, Li J, Howren A, Tsao NW, De Vera M. Mobile phone apps targeting medication adherence: quality assessment and content analysis of user reviews. JMIR mHealth uHealth. 2019;7(1):e11919.

    Google Scholar 

  72. 72.

    Byambasuren O, Sanders S, Beller E, Glasziou P. Prescribable mHealth apps identified from an overview of systematic reviews. Digit Med. 2018;1(1):12.

    Google Scholar 

  73. 73.

    Short CE, Finlay A, Sanders I, Maher C. Development and pilot evaluation of a clinic-based mHealth app referral service to support adult cancer survivors increase their participation in physical activity using publicly available mobile apps. BMC Health Serv Res. 2018;18(1):27.

    Google Scholar 

  74. 74.

    Bender JL, Yue RYK, To MJ, Deacken L, Jadad AR. A lot of action, but not in the right direction: systematic review and content analysis of smartphone applications for the prevention, detection, and management of cancer. J Med Internet Res. 2013;15(12):e287.

    Google Scholar 

  75. 75.

    Bychkov D, Young SD. Facing up to nomophobia: a systematic review of mobile phone apps that reduce Smartphone usage. In Big data in engineering applications. Springer; 2018, p. 161–171.

  76. 76.

    Olivero E, Bert F, Thomas R, Scarmozzino A, Raciti IM, Gualano MR, et al. E-tools for hospital management: an overview of smartphone applications for health professionals. Int J Med Inform. 2019;124:58–67.

    Google Scholar 

  77. 77.

    Rodríguez AQ, Wägner AM. Mobile phone applications for diabetes management: a systematic review. Endocrinol Diab Nutr. 2019;66(5):330–7.

    Google Scholar 

  78. 78.

    Reynoldson C, Stones C, Allsop M, Gardner P, Bennett MI, Closs SJ, et al. Assessing the quality and usability of smartphone apps for pain self-management. Pain Med. 2014;15(6):898–909.

    Google Scholar 

  79. 79.

    Rajani NB, Weth D, Mastellos N, Filippidis FT. Adherence of popular smoking cessation mobile applications to evidence-based guidelines. BMC Public Health. 2019;19(1):743.

    Google Scholar 

  80. 80.

    Xiao Q, Wang Y, Sun L, Lu S, Wu Y. Current status and quality assessment of cardiovascular diseases related smartphone apps in China. Nurs Inform. 2016;225:1030–1.

    Google Scholar 

  81. 81.

    Sedrati H, Nejjari C, Chaqsare S, Ghazal H. Mental and physical mobile health apps. Procedia Comput Sci. 2016;100:900–6.

    Google Scholar 

  82. 82.

    Huckvale K, Prieto JT, Tilney M, Benghozi P-J, Car J. Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment. BMC Med. 2015;13(1):214.

    Google Scholar 

  83. 83.

    Larco A, Yanez C, Almendáriz V, Luján-Mora S. Thinking about inclusion: Assessment of multiplatform apps for people with disability. In 2018 IEEE Global Engineering Education Conference (EDUCON). IEEE; 2018, pp. 350–354.

  84. 84.

    Seabrook HJ, Stromer JN, Shevkenek C, Bharwani A, de Grood J, Ghali WA. Medical applications: a database and characterization of apps in Apple iOS and Android platforms. BMC Res Notes. 2014;7(1):573.

    Google Scholar 

  85. 85.

    Buechi R, et al. Evidence assessing the diagnostic performance of medical smartphone apps: a systematic review and exploratory meta-analysis. BMJ Open. 2017;7(12):e018280.

    Google Scholar 

  86. 86.

    Mangone ER, Lebrun V, Muessig KE. Mobile phone apps for the prevention of unintended pregnancy: a systematic review and content analysis. JMIR mHealth uHealth. 2016;4(1):e6.

    Google Scholar 

  87. 87.

    Williams JP, Schroeder D. Popular glucose tracking apps and use of mHealth by Latinos with diabetes. JMIR mHealth uHealth. 2015;3(3):e84.

    Google Scholar 

  88. 88.

    Con D, De Cruz P. Mobile phone apps for inflammatory bowel disease self-management: a systematic assessment of content and tools. JMIR mHealth uHealth. 2016;4(1):e13.

    Google Scholar 

  89. 89.

    Linares-Del Rey M, Vela-Desojo L, Cano-de la Cuerda R. Mobile phone applications in Parkinson’s disease: a systematic review. Neurología (English Edition). 2018.

  90. 90.

    Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PloS One. 2016;11(4):e0152285.

    Google Scholar 

  91. 91.

    Huckvale K, Adomaviciute S, Prieto JT, Leow MK-S, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med. 2015;13(1):106.

    Google Scholar 

  92. 92.

    Bachmann DJ, Jamison NK, Martin A, Delgado J, Kman NE. Emergency preparedness and disaster response: there’s an app for that. Prehosp Dis Med. 2015;30(5):486–90.

    Google Scholar 

  93. 93.

    Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health. 2016;3(1):e7.

    Google Scholar 

  94. 94.

    Wurzer P, Parvizi D, Lumenta DB, Giretzlehner M, Branski LK, Finnerty CC, et al. Smartphone applications in burns. Burns. 2015;41(5):977–89.

    Google Scholar 

  95. 95.

    Hayes W, Naziri Q, De Tolla JE, Akamnonu CP, Merola AA, Paulino C. A systematic review of all smart phone applications specifically aimed for use as a scoliosis screening tool. Spine J. 2013;13(9):S38.

    Google Scholar 

Download references


this research is supported by Universiti Pendidikan Sultan Idris under University Research Grant (2017–0310–107-01).

Author information



Corresponding author

Correspondence to A. H. Alamoodi.

Ethics declarations

Disclosure of potential conflicts of interest

The authors declare that they have no conflict of interest.

Research involving human participants and/or animals

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

The authors declare that they have no competing of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material


(DOCX 14 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Alamoodi, A.H., Garfan, S., Zaidan, B.B. et al. A systematic review into the assessment of medical apps: motivations, challenges, recommendations and methodological aspect. Health Technol. (2020). https://doi.org/10.1007/s12553-020-00451-4

Download citation


  • Apps assessment
  • Mobile apps
  • Medical apps
  • Assessment
  • Evaluation