Evidence-based practices are effective only when implemented faithfully. This paper explicates the history, standardization, and methods for developing and validating measures of fidelity. We overviewed the past 20 years of developing fidelity measures, summarized standardization of the development procedures, and described needed psychometric assessments. Fidelity assessment has become the sine qua non of implementation, technical assistance, and research on evidence-based practices. Researchers have established standardized procedures for scale development and psychometric testing. Widescale use of fidelity measurement remains challenging. The implementation of evidence-based practice and the development and validation of fidelity measures are interdependent. International improvements of mental health care will require attention to both.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Addington, D. E., McKenzie, E., Norman, R., Wang, J. L., & Bond, G. R. (2013). Identification of essential evidence-based components of first episode psychosis services. Psychiatric Services, 64, 452–457.
Addington, D. E., Norman, R., Bond, G. R., Sale, T., Melton, R., McKenzie, E., et al. (2016). Development and testing of the First Episode Psychosis Services Fidelity Scale. Psychiatric Services, 67, 1023–1025.
Azrin, S. T., Goldstein, A. B., & Heinssen, R. K. (2016). Expansion of coordinated specialty care for first episode psychosis in the US. Focus, 30, 9–11.
Becker, D. R., Drake, R. E., & Bond, G. R. (2014). The IPS supported employment learning collaborative. Psychiatric Rehabilitation Journal, 37, 79–85.
Becker, D. R., Swanson, S., Reese, S. L., Bond, G. R., & McLeman, B. M. (2015). Evidence-based supported employment fidelity review manual (3rd ed.). Lebanon, NH: Dartmouth Psychiatric Research Center.
Blakely, C., Mayer, J., Gottschalk, R., Schmitt, N., Davidson, W., Roitman, D., et al. (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253–268.
Bond, G. R. (1991). Variations in an assertive outreach model. New Directions for Mental Health Services, 52, 65–80.
Bond, G. R. (2004). Supported employment: Evidence for an evidence-based practice. Psychiatric Rehabilitation Journal, 27, 345–359.
Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence-based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science and Practice, 18, 126–141.
Bond, G. R., Becker, D. R., Drake, R. E., & Vogler, K. M. (1997). A fidelity scale for the individual placement and support model of supported employment. Rehabilitation Counseling Bulletin, 40, 265–284.
Bond, G. R., Drake, R. E., Becker, D. R., & Noel, V. A. (2016). The IPS Learning Community: A longitudinal study of sustainment, quality, and outcome. Psychiatric Services, 67, 864–869.
Bond, G. R., Drake, R. E., McHugo, G. J., Rapp, C. A., & Whitley, R. (2009). Strategies for improving fidelity in the National Evidence-Based Practices Project. Research on Social Work Practice, 19, 569–581.
Bond, G. R., Drake, R. E., & Pogue, J. A. (2019). Expanding Individual Placement and Support to populations with conditions and disorders other than serious mental illness. Psychiatric Services. https://doi.org/10.1176/appi.ps.201800464.
Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. K. (2000a). Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research, 2, 75–87.
Bond, G. R., Miller, L. D., Krumwied, R. D., & Ward, R. S. (1988). Assertive case management in three CMHCs: A controlled study. Hospital & Community Psychiatry, 39, 411–418.
Bond, G. R., Peterson, A. E., Becker, D. R., & Drake, R. E. (2012). Validation of the revised Individual Placement and Support Fidelity Scale (IPS-25). Psychiatric Services, 63, 758–763.
Bond, G. R., & Salyers, M. P. (2004). Prediction of outcome from the Dartmouth ACT Fidelity Scale. CNS Spectrums, 9, 937–942.
Bond, G. R., Williams, J., Evans, L., Salyers, M., Kim, H. W., Sharpe, H., & Leff, H. S. (2000b). Psychiatric rehabilitation fidelity toolkit. Cambridge, MA: Human Services Research Institute. Retrieved from http://www.hsri.org/publication/psychiatric_rehabilitation_fidelity_toolkit.
Boyle, P. E., & Kroon, H. (2006). Integrated dual disorder treatment comparing facilitators and challenges of implementation for Ohio and the Netherlands. International Journal of Mental Health, 35, 70–88.
Breitenstein, S. M., Gross, D., Garvey, C. A., Hill, C., Fogg, L., & Resnick, B. (2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33, 164–173.
Brekke, J. S. (1988). What do we really know about community support programs? Strategies for better monitoring. Hospital & Community Psychiatry, 39, 946–952.
Brekke, J. S., & Test, M. A. (1987). An empirical analysis of services delivered in a model community support program. Psychosocial Rehabilitation Journal, 10(4), 51–61.
Brekke, J. S., & Test, M. A. (1992). A model for measuring the implementation of community support programs: Results from three sites. Community Mental Health Journal, 28, 227–247.
Brekke, J. S., & Wolkon, G. H. (1988). Monitoring program implementation in community mental health settings. Evaluation and the Health Professions, 11, 425–440.
Burns, T., Catty, J., Becker, T., Drake, R. E., Fioritti, A., Knapp, M., … Wiersma, D. (2007a). The effectiveness of supported employment for people with severe mental illness: A randomised controlled trial. Lancet, 370, 1146–1152.
Burns, T., Catty, J., Dash, M., Roberts, C., Lockwood, A., & Marshall, M. (2007b). Use of intensive case management to reduce time in hospital in people with severe mental illness: Systematic review and meta-regression. British Medical Journal, 335, 336–340.
Chandler, D. W. (2011). Fidelity and outcomes in six integrated dual disorders treatment programs. Community Mental Health Journal, 47, 82–89.
Chinman, M., McCarthy, S., Mitchell-Miland, C., Daniels, K., Youk, A., & Edelen, M. (2016). Early stages of development of a peer specialist fidelity measure. Psychiatric Rehabilitation Journal, 39, 256–265.
Clausen, H., Landheim, A., Odden, S., Benth, J. Š., Heiervang, K. S., Stuen, H. K., … Ruud, T. (2016). Hospitalization of high and low inpatient service users before and after enrollment into Assertive Community Treatment teams: A naturalistic observational study. International Journal of Mental Health Systems, 10, 14.
Cuddeback, G. S., Domino, M. E., Morrissey, J. P., Monroe-DeVita, M., Teague, G. B., & Moser, L. L. (2013). Fidelity to recovery-oriented ACT practices and consumer outcomes. Psychiatric Services, 64, 318–323.
Davies, E., Dube, S., Farver, M., Silloway, T., Sutherland, K., & White, D. (2017). How states engage in evidence-based policymaking: A national assessment In Pew-MacArthur Results First Initiative. Washington, DC: Pew Charitable Trusts. Retrieved from http://www.pewtrusts.org/en/research-and-analysis/reports/2017/01/how-states-engage-in-evidence-based-policymaking.
Deci, P. A., Santos, A. B., Hiott, D. W., Schoenwald, S., & Dias, J. K. (1995). Dissemination of assertive community treatment teams. Psychiatric Services, 46, 676–678.
Drake, R. E., Bond, G. R., & Becker, D. R. (2012). IPS supported employment: An evidence-based approach to supported employment. New York: Oxford University Press.
Drake, R. E., Frey, W. D., Bond, G. R., Goldman, H. H., Salkever, D. S., Miller, A. L., … Milfort, R. (2013). Assisting social security disability insurance beneficiaries with schizophrenia, bipolar disorder, or major depression in returning to work. American Journal of Psychiatry, 170, 1433–1441.
Drake, R. E., Goldman, H. H., Leff, H. S., Lehman, A. F., Dixon, L., Mueser, K. T., et al. (2001). Implementing evidence-based practices in routine mental health service settings. Psychiatric Services, 52, 179–182.
Dumas, J. S., & Redish, J. (1999). A practical guide to usability testing. Exeter, UK: Intellect Books.
Durbin, J., Selick, A., Langill, G., Cheng, C., Archie, S., Butt, S., et al. (2019). Using fidelity measurement to assess quality of early psychosis intervention services in Ontario. Psychiatric Services. https://doi.org/10.1176/appi.ps.201800581.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.
Egeland, K. M., Heiervang, K. S., Ruud, T., Drake, R. E., & Bond, G. R. (in press). Psychometric properties of a fidelity scale for Illness Management and Recovery. Administration and Policy in Mental Health and Mental Health Services Research.
Einhorn, H. J., & Hogarth, R. M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.
Evans, L. J., & Bond, G. R. (2008). Expert ratings on the critical ingredients of supported employment for people with severe mental illness. Psychiatric Rehabilitation Journal, 31, 318–331.
Fiander, M., Burns, T., McHugo, G. J., & Drake, R. E. (2003). Assertive community treatment across the Atlantic: Comparison of model fidelity in the UK and USA. British Journal of Psychiatry, 182, 248–254.
Fiske, D. W. (1971). Measuring the concepts of personality. Chicago: Aldine.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida.
Fukui, S., Goscha, R., Rapp, C. A., Mabry, A., Liddy, P., & Marty, D. (2012). Strengths model case management fidelity scores and client outcomes. Psychiatric Services, 63, 708–710.
Gagnon, R., Jacques, A., Billard, M., & Goulet, F. (2010). Determining the number of patient charts necessary for a reliable assessment of practicing family physicians’ performance. Evaluation and the Health Professions, 33, 109–122.
Harrison, J., Spybrook, J., Curtis, A., & Cousins, L. (2017). Integrated dual disorder treatment: Fidelity and implementation over time. Social Work Research, 41, 111–119.
Howard, P. B., El-Mallakh, P., Miller, A. L., Rayens, M. K., Bond, G. R., Henderson, K., et al. (2009). Prescriber fidelity to a medication management evidence-based practice in the treatment of schizophrenia. Psychiatric Services, 60, 929–935.
Ito, J., Oshima, I., Nishio, M., & Kuno, E. (2009). Initiative to build a community-based mental health system including assertive community treatment for people with severe mental illness in Japan. American Journal of Psychiatric Rehabilitation, 12, 247–260.
Joa, I., Johannesen, J. O., Heiervang, K. S., Sviland, A. A., Nordin, H. A., Landers, M., … Bond, G. R. (submitted). The family psychoeducation (FPE) fidelity scale: properties and clinical use.
Kealey, E. M., Leckman-Westin, E., Jewell, T. C., & Finnerty, M. T. (2015). Multifamily group psychoeducation in New York state: Implementation and fidelity outcomes. Psychiatric Services, 66, 1194–1199.
Kim, S. J., Bond, G. R., Becker, D. R., Swanson, S. J., & Reese, S. L. (2015). Predictive validity of the Individual Placement and Support Fidelity Scale (IPS-25): A replication study. Journal of Vocational Rehabilitation, 43, 209–216.
Latimer, E. (1999). Economic impacts of assertive community treatment: A review of the literature. Canadian Journal of Psychiatry, 44, 443–454.
Lee, N., & Cameron, J. (2009). Differences in self and independent ratings on an organisational dual diagnosis capacity measure. Drug and Alcohol Review, 28, 682–684.
Lieberman, M. A., Yalom, I. D., & Miles, M. B. (1973). Encounter groups: First facts. New York: Basic Books.
Lloyd-Evans, B., Bond, G. R., Ruud, T., Ivanecka, A., Gray, R., Osborn, D., … Johnson, S. (2016). Development of a measure of model fidelity for mental health Crisis Resolution Teams. BMC Psychiatry, 16, 427. https://doi.org/10.1186/s12888-12016-11139-12884.
Lloyd-Evans, B., Osborn, D., Ambler, G., Marston, L., Hunter, R., Mason, O., … Johnson, S. (2019). The CORE Service Improvement Programme for mental health Crisis Resolution Teams: Results from a cluster-randomised trial. British Journal of Psychiatry. https://doi.org/10.1192/bjp.2019.21.
Lockett, H., Waghorn, G., Kydd, R., & Chant, D. (2016). Predictive validity of evidence-based practices in supported employment: A systematic review and meta-analysis. Mental Health Review Journal. https://doi.org/10.1108/MHRJ-1112-2015-0040.
Luo, X., Law, S. F., Wang, X., Shi, J., Ma, X., Chow, W., … Phillips, M. R. (2018). Effectiveness of an Assertive Community Treatment program for people with severe schizophrenia in mainland China: A 12-month randomized controlled trial. Psychological Medicine. https://doi.org/10.1017/s0033291718001629.
McGovern, M. P., Lambert-Harris, C., McHugo, G. J., Giard, J., & Mangrum, L. (2010). Improving the dual diagnosis capability of addiction and mental health treatment services: Implementation factors associated with program level changes. Journal of Dual Diagnosis, 6, 237–250.
McGrew, J. H., & Bond, G. R. (1995). Critical ingredients of assertive community treatment: Judgments of the experts. Journal of Mental Health Administration, 22, 113–125.
McGrew, J. H., Bond, G. R., Dietzen, L. L., & Salyers, M. P. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62, 670–678.
McGrew, J. H., Stull, L. G., Rollins, A. L., Salyers, M. P., & Hicks, L. (2011). A comparison of phone-based and onsite-based fidelity for Assertive Community Treatment (ACT): A pilot study in Indiana. Psychiatric Services, 62, 670–674.
McGrew, J. H., White, L., Stull, L., & Wright-Berryman, J. (2013). A comparison of self-reported and phone-based fidelity for assertive community treatment (ACT): A pilot study in Indiana. Psychiatric Services, 64, 272–276.
McGrew, J. H., Wilson, R., & Bond, G. R. (1996). Client perspectives on helpful ingredients of assertive community treatment. Psychiatric Rehabilitation Journal, 19(3), 13–21.
McHugo, G. J., Drake, R. E., Teague, G. B., & Xie, H. (1999). Fidelity to assertive community treatment and client outcomes in the New Hampshire Dual Disorders Study. Psychiatric Services, 50, 818–824.
McHugo, G. J., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., … Finnerty, M. T. (2007). Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatric Services, 58, 1279–1284.
Metcalfe, J. D., Riley, J., McGurk, S., Hale, T., Drake, R. E., & Bond, G. R. (2018). Comparing predictors of employment in Individual Placement and Support: A longitudinal analysis. Psychiatry Research, 264, 85–90.
Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266.
Monroe-DeVita, M., Morse, G. A., & Bond, G. R. (2012). Program fidelity and beyond: Multiple strategies and criteria for ensuring quality in ACT. Psychiatric Services, 63, 743–750.
Monroe-DeVita, M., Teague, G. B., & Moser, L. L. (2011). The TMACT: A new tool for measuring fidelity to assertive community treatment. Journal of the American Psychiatric Nurses Association, 17, 17–29.
Mowbray, C. T., Holder, M., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.
Mowbray, C. T., Plum, T. B., & Masterton, T. (1997). Harbinger II: Deployment and evolution of assertive community treatment in Michigan. Administration and Policy In Mental Health, 25, 125–139.
Mueser, K. T., Corrigan, P. W., Hilton, D. W., Tanzman, B., Schaub, A., Gingerich, S., … Herz, M. I. (2002). Illness management and recovery: A review of the research. Psychiatric Services, 53, 1272–1284.
Odden, S., Landheim, A., Clausen, H., Stuen, H. K., Heiervang, K. S., & Ruud, T. (2019). Model fidelity and team members’ experiences of assertive community treatment in Norway: A sequential mixed-methods study. International Journal of Mental Health Systems, 13, 65. https://doi.org/10.1186/s13033-019-0321-8.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., … Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76.
Rapp, C. A., Etzel-Wise, D., Marty, D., Coffman, M., Carlson, L., Asher, D., … Whitley, R. E. (2008). Evidence-based practice implementation strategies: Results of a qualitative study. Community Mental Health Journal, 44, 213–224.
Rollins, A. L., Kukla, M., Salyers, M. P., McGrew, J. H., Flanagan, M. E., Leslie, D. L., … McGuire, A. B. (2017). Comparing the costs and acceptability of three fidelity assessment methods for assertive community treatment. Administration and Policy in Mental Health and Mental Health Services Research, 44, 810–816.
Rollins, A. L., McGrew, J. H., Kukla, M., McGuire, A. B., Flanagan, M. E., Hunt, M. G., … Salyers, M. P. (2016). Comparison of assertive community treatment fidelity assessment methods: Reliability and validity. Administration and Policy in Mental Health and Mental Health Services Research, 43, 157–167.
Ruud, T., Drivenes, K., Haaland, V. Ø., Drake, R. E., Torrey, W., Landers, M., & Bond, G. R. (submitted, a). Development, properties and clinical use of a fidelity scale for antipsychotic medication.
Ruud, T., Høifødt Sørensen, T., Høie, A., Drake, R. E., Cimpean Hendrick, D., Landers, M., & Bond, G. R. (submitted, b). Development, properties and clinical use of a fidelity scale for physical health care.
Salyers, M. P., Bond, G. R., Teague, G. B., Cox, J. F., Smith, M. E., Hick, M. L., et al. (2003). Is it ACT yet? Real-world examples of evaluating the degree of implementation for assertive community treatment. Journal of Behavioral Health Services & Research, 30, 308–324.
Salyers, M. P., Godfrey, J. L., McGuire, A. B., Gearhart, T., Rollins, A. L., & Boyle, C. (2009). Implementing the illness management and recovery program for consumers with severe mental illness. Psychiatric Services, 60, 483–490.
Salyers, M. P., McKasson, M., Bond, G. R., McGrew, J. H., Rollins, A. L., & Boyle, C. (2007). The role of technical assistance centers in implementing evidence-based practices: Lessons learned. American Journal of Psychiatric Rehabilitation, 10, 85–101.
Sasaki, N., Yamaguchi, S., Shimodaira, M., Sato, S., Taneda, A., Yoshida, K., et al. (2017). Development and validation of a Japanese fidelity scale for supported employment. Administration and Policy in Mental Health and Mental Health Services Research, 45, 318–327.
Silva, T. F. (2014). Assessing fidelity in psychosocial interventions: A systematic literature review (Avaliando a fidelidade de intervenções psicossociais: uma revisão sistemática da literatura). Jornal Brasileiro de Psiquiatria. https://doi.org/10.1590/0047-2085000000034.
Siskind, D., & Wiley-Exley, E. (2009). Comparison of assertive community treatment programs in urban Massachusetts and rural North Carolina. Administration and Policy in Mental Health and Mental Health Services Research, 36, 236–246.
Sloane, R. B., Staples, F. R., Cristol, A. H., Yorkston, N. J., & Whipple, K. (1975). Psychotherapy versus behavior therapy. Cambridge: Harvard University Press.
Stein, L. I., & Test, M. A. (1980). An alternative to mental health treatment. I: Conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry, 37, 392–397.
Taylor, A. C., Bond, G. R., Tsai, J., Howard, P. B., El-Mallakh, P., Finnerty, M., … Miller, A. L. (2009). A scale to evaluate quality of medication management: Development and psychometric properties. Administration and Policy in Mental Health and Mental Health Services Research, 36, 247–254.
Teague, G. B., Bond, G. R., & Drake, R. E. (1998). Program fidelity in assertive community treatment: Development and use of a measure. American Journal of Orthopsychiatry, 68, 216–232.
Tsai, J., & Bond, G. R. (2008). A comparison of electronic medical records to conventional paper records in community mental health centers. International Journal for Quality in Health Care, 20, 136–143.
Tsoi, W. S. (2016). Understanding and evaluating strengths model case management in Hong Kong mental health services. Pokfulam, Hong Kong: University of Hong Kong.
van Vugt, M., Kroon, H., Delespaul, P., Dreef, F., Nugter, A., van Weeghel, J., et al. (2011). Assertive Community Treatment in the Netherlands: Outcome and model fidelity. Canadian Journal of Psychiatry, 56, 154–160.
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620–630.
Weaver, A., Capobianco, J., & Ruffolo, M. (2015). Systematic review of EBPs for SMI in rural America. Journal of Evidence-Informed Social Work, 12, 155–165.
Winter, S. G., & Szulanski, G. (2001). Replication as strategy. Organization Science, 12, 730–743.
Yung, A. R. (2012). Early intervention in psychosis: Evidence, evidence gaps, criticism, and confusion. Australian and New Zealand Journal of Psychiatry, 46, 7–9.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bond, G.R., Drake, R.E. Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology. Adm Policy Ment Health 47, 874–884 (2020). https://doi.org/10.1007/s10488-019-00991-6
- Evidence-based practice
- Scale development