Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology
Evidence-based practices are effective only when implemented faithfully. This paper explicates the history, standardization, and methods for developing and validating measures of fidelity. We overviewed the past 20 years of developing fidelity measures, summarized standardization of the development procedures, and described needed psychometric assessments. Fidelity assessment has become the sine qua non of implementation, technical assistance, and research on evidence-based practices. Researchers have established standardized procedures for scale development and psychometric testing. Widescale use of fidelity measurement remains challenging. The implementation of evidence-based practice and the development and validation of fidelity measures are interdependent. International improvements of mental health care will require attention to both.
KeywordsFidelity Evidence-based practice Measurement Scale development
- Azrin, S. T., Goldstein, A. B., & Heinssen, R. K. (2016). Expansion of coordinated specialty care for first episode psychosis in the US. Focus, 30, 9–11.Google Scholar
- Becker, D. R., Swanson, S., Reese, S. L., Bond, G. R., & McLeman, B. M. (2015). Evidence-based supported employment fidelity review manual (3rd ed.). Lebanon, NH: Dartmouth Psychiatric Research Center.Google Scholar
- Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence-based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science and Practice, 18, 126–141.Google Scholar
- Bond, G. R., Becker, D. R., Drake, R. E., & Vogler, K. M. (1997). A fidelity scale for the individual placement and support model of supported employment. Rehabilitation Counseling Bulletin, 40, 265–284.Google Scholar
- Bond, G. R., Miller, L. D., Krumwied, R. D., & Ward, R. S. (1988). Assertive case management in three CMHCs: A controlled study. Hospital & Community Psychiatry, 39, 411–418.Google Scholar
- Bond, G. R., Williams, J., Evans, L., Salyers, M., Kim, H. W., Sharpe, H., & Leff, H. S. (2000b). Psychiatric rehabilitation fidelity toolkit. Cambridge, MA: Human Services Research Institute. Retrieved from http://www.hsri.org/publication/psychiatric_rehabilitation_fidelity_toolkit.
- Breitenstein, S. M., Gross, D., Garvey, C. A., Hill, C., Fogg, L., & Resnick, B. (2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33, 164–173.Google Scholar
- Brekke, J. S. (1988). What do we really know about community support programs? Strategies for better monitoring. Hospital & Community Psychiatry, 39, 946–952.Google Scholar
- Clausen, H., Landheim, A., Odden, S., Benth, J. Š., Heiervang, K. S., Stuen, H. K., … Ruud, T. (2016). Hospitalization of high and low inpatient service users before and after enrollment into Assertive Community Treatment teams: A naturalistic observational study. International Journal of Mental Health Systems, 10, 14.Google Scholar
- Davies, E., Dube, S., Farver, M., Silloway, T., Sutherland, K., & White, D. (2017). How states engage in evidence-based policymaking: A national assessment In Pew-MacArthur Results First Initiative. Washington, DC: Pew Charitable Trusts. Retrieved from http://www.pewtrusts.org/en/research-and-analysis/reports/2017/01/how-states-engage-in-evidence-based-policymaking.
- Drake, R. E., Frey, W. D., Bond, G. R., Goldman, H. H., Salkever, D. S., Miller, A. L., … Milfort, R. (2013). Assisting social security disability insurance beneficiaries with schizophrenia, bipolar disorder, or major depression in returning to work. American Journal of Psychiatry, 170, 1433–1441.CrossRefPubMedPubMedCentralGoogle Scholar
- Dumas, J. S., & Redish, J. (1999). A practical guide to usability testing. Exeter, UK: Intellect Books.Google Scholar
- Egeland, K. M., Heiervang, K. S., Ruud, T., Drake, R. E., & Bond, G. R. (in press). Psychometric properties of a fidelity scale for Illness Management and Recovery. Administration and Policy in Mental Health and Mental Health Services Research. Google Scholar
- Fiske, D. W. (1971). Measuring the concepts of personality. Chicago: Aldine.Google Scholar
- Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida.Google Scholar
- Joa, I., Johannesen, J. O., Heiervang, K. S., Sviland, A. A., Nordin, H. A., Landers, M., … Bond, G. R. (submitted). The family psychoeducation (FPE) fidelity scale: properties and clinical use.Google Scholar
- Lieberman, M. A., Yalom, I. D., & Miles, M. B. (1973). Encounter groups: First facts. New York: Basic Books.Google Scholar
- Lloyd-Evans, B., Bond, G. R., Ruud, T., Ivanecka, A., Gray, R., Osborn, D., … Johnson, S. (2016). Development of a measure of model fidelity for mental health Crisis Resolution Teams. BMC Psychiatry, 16, 427. https://doi.org/10.1186/s12888-12016-11139-12884.
- Lloyd-Evans, B., Osborn, D., Ambler, G., Marston, L., Hunter, R., Mason, O., … Johnson, S. (2019). The CORE Service Improvement Programme for mental health Crisis Resolution Teams: Results from a cluster-randomised trial. British Journal of Psychiatry. https://doi.org/10.1192/bjp.2019.21.
- Luo, X., Law, S. F., Wang, X., Shi, J., Ma, X., Chow, W., … Phillips, M. R. (2018). Effectiveness of an Assertive Community Treatment program for people with severe schizophrenia in mainland China: A 12-month randomized controlled trial. Psychological Medicine. https://doi.org/10.1017/s0033291718001629.CrossRefPubMedPubMedCentralGoogle Scholar
- Odden, S., Landheim, A., Clausen, H., Stuen, H. K., Heiervang, K. S., & Ruud, T. (2019). Model fidelity and team members’ experiences of assertive community treatment in Norway: A sequential mixed-methods study. International Journal of Mental Health Systems, 13, 65. https://doi.org/10.1186/s13033-019-0321-8.CrossRefPubMedPubMedCentralGoogle Scholar
- Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., … Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76.CrossRefGoogle Scholar
- Rollins, A. L., Kukla, M., Salyers, M. P., McGrew, J. H., Flanagan, M. E., Leslie, D. L., … McGuire, A. B. (2017). Comparing the costs and acceptability of three fidelity assessment methods for assertive community treatment. Administration and Policy in Mental Health and Mental Health Services Research, 44, 810–816.CrossRefPubMedPubMedCentralGoogle Scholar
- Rollins, A. L., McGrew, J. H., Kukla, M., McGuire, A. B., Flanagan, M. E., Hunt, M. G., … Salyers, M. P. (2016). Comparison of assertive community treatment fidelity assessment methods: Reliability and validity. Administration and Policy in Mental Health and Mental Health Services Research, 43, 157–167.CrossRefGoogle Scholar
- Ruud, T., Drivenes, K., Haaland, V. Ø., Drake, R. E., Torrey, W., Landers, M., & Bond, G. R. (submitted, a). Development, properties and clinical use of a fidelity scale for antipsychotic medication.Google Scholar
- Ruud, T., Høifødt Sørensen, T., Høie, A., Drake, R. E., Cimpean Hendrick, D., Landers, M., & Bond, G. R. (submitted, b). Development, properties and clinical use of a fidelity scale for physical health care.Google Scholar
- Taylor, A. C., Bond, G. R., Tsai, J., Howard, P. B., El-Mallakh, P., Finnerty, M., … Miller, A. L. (2009). A scale to evaluate quality of medication management: Development and psychometric properties. Administration and Policy in Mental Health and Mental Health Services Research, 36, 247–254.CrossRefPubMedPubMedCentralGoogle Scholar