Monitoring and Evaluating Public Health Interventions

Part of the STEAM-H: Science, Technology, Engineering, Agriculture, Mathematics & Health book series (STEAM)


Health Technology Assessment (HTA) has become the preferred approach that health systems use for evaluating and monitoring health technologies. Nevertheless, it has mainly focused on pharmaceuticals and medical equipment, while HTAs on public health interventions (PHIs) are rarely performed. The limitations of the traditional methods to evaluate PHIs with a national scope could be one of the reasons for the lack of studies. This situation suggests the need to propose new approaches for evaluating this type of technology. The chapter proposes the use of intervention analysis on time series, using the Box and Tiao approach, as a method for HTA on PHI. Additionally, to illustrate the advantages of the method, a case study is presented in which it is used to assess the impact that the establishment of the National Information System on Breast Cancer, in June 2009, has had on the mortality rates in the five regions of Brazil.


Health technology assessment Interrupted time series analysis Public health intervention National health programs Breast neoplasms 


  1. 1.
    Adeagbo, C. U., Rattanavipapong, W., Guinness, L., & Teerawattananon, Y. (2018). The Development of the Guide to Economic Analysis and Research (GEAR) online resource for low-and middle-income countries’ health economics practitioners: A commentary. Value in Health, 21(5), 569–572.CrossRefGoogle Scholar
  2. 2.
    Banta, D., & Jonsson, E. (2009). History of HTA: Introduction. International Journal of Technology Assessment in Health Care, 25, 1–6.CrossRefGoogle Scholar
  3. 3.
    Bernal, J. L., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46, 348–355.Google Scholar
  4. 4.
    Box, G., Jenkins, G., Reinsel, G., & Ljung, G. (2015). Time series analysis: Forecasting and control (5th ed.). Hoboken: Wiley.zbMATHGoogle Scholar
  5. 5.
    Box, G. E., & Tiao, G. (1975). Intervention analysis with applications to economic and environmental problems. Journal of the American Statistical Association, 70, 70–79.MathSciNetCrossRefGoogle Scholar
  6. 6.
    BRASIL. (2015). Sistema de Informação sobre Mortalidade (SIM) – Informações de Saúde (TABNET) [WWW Document]. Minist. Saúde – Portal Saúde. URL Accessed 13 Apr 2016.
  7. 7.
    Chandran, A., Pérez-Núñez, R., Bachani, A. M., Híjar, M., Salinas-Rodríguez, A., & Hyder, A. A. (2014). Early impact of a national multi-faceted road safety intervention program in Mexico: Results of a time-series analysis. PLoS One, 9, e87482.CrossRefGoogle Scholar
  8. 8.
    Coldman, A., Phillips, N., Warren, L., & Kan, L. (2007). Breast cancer mortality after screening mammography in British Columbia women. International Journal of Cancer, 120, 1076–1080.CrossRefGoogle Scholar
  9. 9.
    Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: The new medical research council guidance. BMJ, 337, a1655.CrossRefGoogle Scholar
  10. 10.
    Draborg, E., Gyrd-Hansen, D., Poulsen, P. B., & Horder, M. (2005). International comparison of the definition and the practical application of health technology assessment. International Journal of Technology Assessment in Health Care, 21, 89–95.CrossRefGoogle Scholar
  11. 11.
    Felix, J. D., Castro, D. S., Amorim, M. H., & Zandonade, E. (2011). Breast cancer mortality trends among women in the state of Espirito Santo between 1980 and 2007. Revista Brasileira de Cancerologia, 57, 159–166.Google Scholar
  12. 12.
    Freitas, R., Gonzaga, C. M., Freitas, N. M., Martins, E., & Dardes, R. M. (2012). Disparities in female breast cancer mortality rates in Brazil between 1980 and 2009. Clinics, 67, 731–737.CrossRefGoogle Scholar
  13. 13.
    Girianelli, V. R., Gamarra, C. J., & Azevedo, G. (2014). Disparities in cervical and breast cancer mortality in Brazil. Revista de Saúde Pública, 48, 459–467.CrossRefGoogle Scholar
  14. 14.
    Grant, A., Treweek, S., Dreischulte, T., Foy, R., & Guthrie, B. (2013). Process evaluations for cluster-randomised trials of complex interventions: A proposed framework for design and reporting. Trials, 14(1), 15.CrossRefGoogle Scholar
  15. 15.
    IBGE. (2015). Instituto Brasileiro de Geografia e Estatística [WWW Document]. Minist. Planej. URL Accessed 12 Apr 2016.
  16. 16.
    INAHTA. (2019). HTA glossary – the international network of agencies for health technology assessment [WWW document]. URL Accessed 14 Jan 2019.
  17. 17.
    Jonsson, E., & Banta, D. (1999). Management of health technologies: An international view. BMJ, 319(7229), 1293.CrossRefGoogle Scholar
  18. 18.
    Lavis, J. N., Wilson, M. G., Grimshaw, J. M., Haynes, R. B., Ouimet, M., Raina, P., Gruen, R. L., & Graham, I. D. (2010). Supporting the use of health technology assessments in policy making about health systems. International Journal of Technology Assessment in Health Care, 26, 405–414.CrossRefGoogle Scholar
  19. 19.
    Liu, L.-M., Hudak, G. B., Box, G. E., Muller, M. E., & Tiao, G. C. (1992). Forecasting and time series analysis using the SCA statistical system (1st ed.). DeKalb: Scientific Computing Associates.Google Scholar
  20. 20.
    Makady, A., van Veelen, A., Jonsson, P., Moseley, O., D’Andon, A., de Boer, A., et al. (2018). Using real-world data in health technology assessment (HTA) practice: A comparative study of five HTA agencies. PharmacoEconomics, 36(3), 359–368.CrossRefGoogle Scholar
  21. 21.
    Malmgren, J. A., Parikh, J., Atwood, M. K., & Kaplan, H. G. (2012). Impact of mammography detection on the course of breast cancer in women aged 40–49 years. Radiology, 262, 797–806.CrossRefGoogle Scholar
  22. 22.
    Massat, N. J., Dibden, A., Parmar, D., Cuzick, J., Sasieni, P. D., & Duffy, S. W. (2016). Impact of screening on breast cancer mortality: The UK program 20 years on. Cancer Epidemiology, Biomarkers & Prevention, 25, 455–462.CrossRefGoogle Scholar
  23. 23.
    Masukawa, M. L. T., Moriwaki, A. M., Uchimura, N. S., Souza, E. M., & Uchimura, T. T. (2014). Intervention analysis of introduction of rotavirus vaccine on hospital admissions rates due to acute diarrhea. Cadernos de Saúde Pública, 30, 2101–2111.CrossRefGoogle Scholar
  24. 24.
    Mathes, T., Antoine, S.-L., Prengel, P., Bühn, S., Polus, S., & Pieper, D. (2017). Health technology assessment of public health interventions: A synthesis of methodological guidance. International Journal of Technology Assessment in Health Care, 33, 135–146.CrossRefGoogle Scholar
  25. 25.
    Mathes, T., Willms, G., Polus, S., Stegbauer, C., Messer, M., Klingler, C., Ehrenreich, H., Niebuhr, D., Marckmann, G., Gerhardus, A., & Pieper, D. (2018). Health technology assessment of public health interventions: An analysis of characteristics and comparison of methods-study protocol. Systematic Reviews, 7, 79.CrossRefGoogle Scholar
  26. 26.
    Mellou, K., Sideroglou, T., Papaevangelou, V., Katsiaflaka, A., Bitsolas, N., Verykouki, E., Triantafillou, E., Baka, A., Georgakopoulou, T., & Hadjichristodoulou, C. (2015). Considerations on the current universal vaccination policy against hepatitis a in Greece after recent outbreaks. PLoS One, 10, e0116939.CrossRefGoogle Scholar
  27. 27.
    Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O’Cathain, A., Tinati, T., & Wight, D. (2015). Process evaluation of complex interventions: Medical Research Council guidance. British Medical Journal, 350, h1258.CrossRefGoogle Scholar
  28. 28.
    Nilson, F., Bonander, C., & Andersson, R. (2015). The effect of the transition from the ninth to the tenth revision of the international classification of diseases on external cause registration of injury morbidity in Sweden. Injury Prevention, 21, 189–194.CrossRefGoogle Scholar
  29. 29.
    Olsen, A. H., Lynge, E., Njor, S. H., Kumle, M., Waaseth, M., Braaten, T., & Lund, E. (2013). Breast cancer mortality in Norway after the introduction of mammography screening. International Journal of Cancer, 132, 208–214.CrossRefGoogle Scholar
  30. 30.
    PAHO/WHO. (2016). PAHO’s role in health technology assessment in the Americas [WWW document]. Pan American Health Organization World Health Organization. URL Accessed 14 Feb 2019.
  31. 31.
    Pichon-Riviere, A., Soto, N. C., Augustovski, F. A., Martí, S. G., & Sampietro-Colom, L. (2018). Health technology assessment for decision making in Latin America: Good practice principles. International Journal of Technology Assessment in Health Care, 34(3), 241–247.CrossRefGoogle Scholar
  32. 32.
    Petticrew, M., Chalabi, Z., & Jones, D. R. (2011). To RCT or not to RCT: Deciding when ‘more evidence is needed for public health policy and practice. Journal of Epidemiology and Community Health, 66(5), 391–396.CrossRefGoogle Scholar
  33. 33.
    R Foundation. (2015). The R project for statistical computing [WWW Document]. URL Accessed 12 Feb 2016.
  34. 34.
    Ramsay, C. R., Matowe, L., Grilli, R., Grimshaw, J. M., & Thomas, R. E. (2003). Interrupted time series designs in health technology assessment: Lessons from two systematic reviews of behavior change strategies. International Journal of Technology Assessment in Health Care, 19, 613.CrossRefGoogle Scholar
  35. 35.
    Reinsperger, I., Rosian, K., & Winkler, R. (2019). Assessment of public health interventions for decision support: Methods & processes of the evaluation of the Austrian screening programme for pregnant women & children. Wiener Medizinische Wochenschrift, 169(11), 263–270.CrossRefGoogle Scholar
  36. 36.
    Rosales-López, A., Raposo, L. M., Nobre, F. F., de Almeida, R. T., Rosales-López, A., Raposo, L. M., Nobre, F. F., & de Almeida, R. T. (2018). The use of intervention analysis of the mortality rates from breast cancer in assessing the Brazilian screening programme. Research on Biomedical Engineering, 34, 285–290.CrossRefGoogle Scholar
  37. 37.
    Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002). Criteria for evaluating evidence on public health interventions. Journal of Epidemiology and Community Health, 56, 119–127.CrossRefGoogle Scholar
  38. 38.
    Silva-Illanes, N., & Espinoza, M. (2018). Critical analysis of Markov models used for the economic evaluation of colorectal cancer screening: A systematic review. Value in Health, 21(7), 858–873.CrossRefGoogle Scholar
  39. 39.
    Shiell, A., Hawe, P., & Gold, L. (2008). Complex interventions or complex systems? Implications for health economic evaluation. BMJ, 336, 1281.CrossRefGoogle Scholar
  40. 40.
    Solt, F. (2016). The standardized world income inequality database. Social Science Quarterly, 97, 1267–1281.CrossRefGoogle Scholar
  41. 41.
    Soumerai, S. B., Starr, D., & Majumdar, S. R. (2015). How do you know which health care effectiveness research you can trust? A guide to study Design for the Perplexed. Preventing Chronic Disease, 12, E101.CrossRefGoogle Scholar
  42. 42.
    Sweeting, M. J., Masconi, K. L., Jones, E., Ulug, P., Glover, M. J., Michaels, J. A., & Thompson, S. G. (2018). Analysis of clinical benefit, harms, and cost-effectiveness of screening women for abdominal aortic aneurysm. The Lancet, 392(10146), 487–495.CrossRefGoogle Scholar
  43. 43.
    Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002). Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics, 27, 299–309.CrossRefGoogle Scholar
  44. 44.
    WHO. (2017). WHO list of priority medical devices for cancer management, WHO medical device technical series. Geneva: World Health Organization.Google Scholar
  45. 45.
    WHO. (2012). Breast cancer: Prevention and control [WWW Document]. World Health Organization Programme Projects Cancer. URL Accessed 12 Apr 2016.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Gerencia de Infraestructura y Tecnología, Caja Costarricense de Seguro SocialSan JoséCosta Rica
  2. 2.Programa de Engenharia Biomédica, COPPE, Universidade Federal do Rio de JaneiroRio de JaneiroBrazil

Personalised recommendations