Skip to main content
SpringerLink
Log in

Measuring Outcomes in Schools

  • Chapter
  • First Online:
Handbook of Australian School Psychology

Abstract

In light of their training in psychometrics, evaluation, and statistical analysis, school psychologists may have the most knowledge, of any professional in the school setting, of evaluating the impact of educational and psychotherapeutic interventions. School settings provide a natural opportunity for important, real-world, and meaningful data that will enhance the delivery of interventions and decision-making as to who benefits from what intervention under what conditions. In this chapter, we provide detailed guidelines for strategies for the working school psychologist to measure system-level change, classroom-wide change, and individual change. We first present various methodologies for collecting data to evaluate change at all levels of intervention implementation. Next, we describe in detail straightforward approaches to calculate the effectiveness of an intervention. Throughout the chapter, we also review the practical barriers that may exist in the Australian educational system that challenge the collection of outcome data, and we present potential strategies to help manage these barriers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 309.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 399.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Albers, C. A., & Kettler, R. J. (2014). Best practices in universal screening. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Data-based and collaborative decision making (pp. 121–131). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Alberto, P., & Troutman, A. C. (2013). Applied behaviour analysis for teachers (9th ed.). Upper Saddle River,NJ: Pearson Education, Inc.

    Google Scholar 

  • American Psychological Association. (2002). Criteria for evaluating treatment guidelines. American Psychologist, 57, 1052–1059. doi:10.1037/0003-066X.57.12.1052.

    Article  Google Scholar 

  • Australian Psychological Society. (2013). Framework for the effective delivery of school psychological services. Flinders Lane, VIC: The Australian Psychological Society Limited.

    Google Scholar 

  • Batsche, G. M., Elliot, J., Graden, J. L., Grimes, J., Kovaleski, J. F., Prasse, D., et al. (2005). Response to intervention: Policy considerations and implementation. Alexandria, VA: National Association of State Directors of Special Education.

    Google Scholar 

  • Berryhill, J. C., & Prinz, R. J. (2003). Environmental interventions to enhance student adjustment: Implications for prevention. Prevention Science, 4, 65–87.

    Article  PubMed  Google Scholar 

  • Bradshaw, C., Mitchell, M., & Leaf, P. (2010). Examining the effects of school-wide positive behavioural interventions and supports on student outcomes. Journal of Positive Behavior Interventions, 12, 133–148.

    Article  Google Scholar 

  • Brown, R., Steege, M. W., & Bickford, R. (2014). Best practices in evaluating the effectiveness of interventions using single-case methods. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (pp. 371–380). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Campbell, J. M., & Hammond, R. K. (2014). Best practices in rating scale assessment of children’s behaviour. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Data-based and collaborative decision making (pp. 287–304). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Cappella, E., Massetti, G. M., & Yampolsky, S. (2009). Rigorous, responsive, and responsible: Experimental designs in school intervention research. In L. M. Dinella (Ed.), Conducting science based psychology research in schools (pp. 51–78). Washington, DC: American Psychological Association. doi:10.1037/11881-003.

    Google Scholar 

  • Castillo, J. M. (2014). Best practices in program evaluation in a model of response to intervention/multitiered system of supports. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (pp. 329–342). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Chafouleas, S. M., Riley-Tillman, T. C., & Christ, T. J. (2009). Direct Behavior Rating (DBR): An emerging method for assessing social behaviour within a tiered intervention system. Assessment for Effective Intervention, 34, 195–200. doi:10.1177/1534508409340391.

    Article  Google Scholar 

  • Commonwealth of Australia. Australian Education Act 2013, No. 67, 2013. Retrieved from https://www.comlaw.gov.au/Details/C2013A00067/Html/Text#_Toc361054816

  • Cone, J. D. (2001). Evaluating outcomes: Empirical tools for effective practice. Washington, DC: American Psychological Association. doi:10.1037/10384-000.

    Book  Google Scholar 

  • Cooke, N., Heward, W. L., Test, D. W., Spooner, F., & Courson, F. H. (1991). Student performance data in the classroom: Measurement and evaluation of student progress. Teacher Education and Special Education, 14, 155–161. doi:10.1177/088840649101400301.

    Article  Google Scholar 

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behaviour analysis (2nd ed.). Upper Saddle River, NJ: Pearson Prentice Hall.

    Google Scholar 

  • Cordray, D. S., & Pion, G. M. (2006). Treatment strength and integrity: Models and methods. In R. R. Bootzin & P. E. McKnight (Eds.), Strengthening research methodology: Psychological measurement and evaluation (pp. 103–124). Washington, DC: American Psychological Association. doi:10.1037/11384-006.

    Google Scholar 

  • Cummings, J. A., & Martinez, R. S. (2012). Visual representation of progress monitoring and academic achievement data. In R. Brown-Chidsey & K. J. Andren (Eds.), Assessment for intervention: A problem-solving approach (2nd ed., pp. 321–343). New York, NY: Guilford Press.

    Google Scholar 

  • Deno, S. L. (1986). Formative evaluation of individual student programs. A new role for school psychologists. School Psychology Review, 15, 358–374.

    Google Scholar 

  • Dix, K. L., Slee, P. T., Lawson, M. J., & Keeves, J. P. (2011). Implementation quality of whole-school mental health promotion and students’ academic performance. Child and Adolescent Mental Health, 17(1), 45–51.

    Article  Google Scholar 

  • Durlak, J. A., & Dupre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting the implementation. American Journal of Community Psychology, 41, 327–350. doi:10.1007/s10464-008-9165-0.

    Article  PubMed  Google Scholar 

  • Erchul, W. P., & Martens, B. K. (2010). School consultation: Conceptual and empirical bases of practice (3rd ed.). New York, NY: Springer Science + Business Media.

    Google Scholar 

  • Fagan, T. K., & Wise, P. S. (2000). Chapter 4: Roles and functions of school psychologists (School psychology: Past, present, and future). Bethesda, MD: NASP Publications.

    Google Scholar 

  • Farlow, L., & Snell, M. (1994). Making the most of student performance data. Washington, DC: American Association on Mental Retardation.

    Google Scholar 

  • Fuchs, L. S., & Deno, S. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57(6), 488–500.

    Article  Google Scholar 

  • Fuchs, L., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208.

    PubMed  Google Scholar 

  • Gibbons, K., & Brown, S. (2014). Best practices in evaluating psychoeducational services based on student outcome data. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (pp. 355–370). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Glover, T. A., & DiPerna, J. C. (2007). Service delivery for response to intervention: Core components and directions for future research. School Psychology Review, 36(4), 526–540.

    Google Scholar 

  • Goffreda, C. T., & DiPerna, J. C. (2010). An empirical review of psychometric evidence for the Dynamic Indicators of Basic Early Literacy Skills. School Psychology Review, 39(3), 463–483.

    Google Scholar 

  • Gresham, F. M., & Elliott, S. N. (2008). Social skills improvement system: Rating scales. Bloomington, MN: Pearson Assessments.

    Google Scholar 

  • Hamre, B. K., & Pianta, R. C. (2007). Learning opportunities in preschool and early elementary classrooms. In R. Pianta, M. Cox, & K. Snow (Eds.), School readiness & the transition to school (pp. 49–84). Baltimore: Brookes.

    Google Scholar 

  • Hamre, B. K., Pianta, R. C., & Chomat-Mooney, L. (2009). Conducting classroom observations in school-based research. In L. M. Dinella (Ed.), Conducting science based psychology research in schools (pp. 79–105). Washington, DC: American Psychological Association. doi:10.1037/11881-004.

    Google Scholar 

  • Horner, R. H., & Spaulding, S. A. (2010). Single-case research designs. In N. J. Salkind (Ed.), Encyclopedia of research design (pp. 1386–1394). Thousand Oaks, CA: SAGE.

    Google Scholar 

  • Howell, K. W., & Hosp, J. L. (2014). Best practices in curriculum-based evaluation. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Data-based and collaborative decision making (pp. 159–171). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Howell, M. K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM: A practical guide to curriculum-based measurement. New York, NY: Guilford Press.

    Google Scholar 

  • Howell, K. W., Hosp, J. L., & Kurns, S. (2008). Best practices in curriculum-based evaluation. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (Vth ed., pp. 349–362). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • Individuals With Disabilities Education Act, 20 U.S.C. § 1400 (2004).

    Google Scholar 

  • Kazdin, A. E. (1981). Drawing valid inferences from case studies. Journal of Consulting and Clinical Psychology, 49, 183–192. doi:10.1037/0022-006X.49.2.183.

    Article  PubMed  Google Scholar 

  • Kazdin, A. E. (1998). Research design in clinical psychology (3rd ed.). Boston: Allyn & Bacon.

    Google Scholar 

  • Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press.

    Google Scholar 

  • Kennedy, C. (2005). Single-case designs for educational research. Boston: Allyn & Bacon.

    Google Scholar 

  • Kettler, R. J., Glover, T. A., Albers, C. A., & Feeney-Kettler, K. A. (2014). An introduction to universal screening in educational settings. In R. J. Kettler, T. A. Glover, C. A. Albers, & K. A. Feeney-Kettler (Eds.), Universal screening in educational settings: Evidence-based decision making for schools (pp. 3–16). Washington, DC: American Psychological Association. doi:10.1037/14316-001.

    Google Scholar 

  • Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M. et al. (2010). Single-case designs technical documentation. In What Works Clearinghouse: Procedures and standards handbook (version 2.0). Retrieved from What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_procedures_v2_standards_handbook.pdf.

  • Kratochwill, T. R., Hitchock, J. H., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., et al. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38. doi:10.1177/0741932512452794.

    Article  Google Scholar 

  • Kratochwill, T. R., & Piersel, W. C. (1983). Time-series research: Contributions to empirical clinical practice. Behavioral Assessment, 5, 165–176.

    Google Scholar 

  • Mager, R. (1997). Preparing instructional objectives (3rd ed.). Atlanta, GA: The Center for Effective Performance, Inc.

    Google Scholar 

  • Martens, B. K., & McIntyre, L. L. (2009). The importance of treatment integrity in school-based behavioural interventions. In A. Akin-Little, S. Little, M. Bray, & T. Kehle (Eds.), Behavioral interventions in schools: Evidence-based positive strategies (pp. 59–71). Washington, DC: American Psychological Association. doi:10.1037/11886-004.

    Google Scholar 

  • McConaughy, S. H., & Ritter, D. R. (2008). Best practices in multimethod assessment of emotional and behavioural disorders. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (Vth ed., pp. 697–715). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • McDougal, J. L., Bardos, A. N., & Meier, S. T. (2010). Behavior intervention monitoring assessment system (BIMAS). Toronto: Multi-Health Systems.

    Google Scholar 

  • McKevitt, B. C., & Braaksma Fynaardt, A. (2014). Best practices in developing a positive behaviour support system at the school level. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Systems-level services (pp. 165–180). Washington, DC: National Association of School Psychologists.

    Google Scholar 

  • Merrell, K. W. (2008). Behavioral, social, and emotional assessment of children and adolescents (3rd ed.). New York, NY: Lawrence Erlbaum Associates.

    Google Scholar 

  • Miltenberger, R. G. (2012). Behavior modifications: Principles and procedures (5th ed.). Belmont, CA: Thomson/Wadsworth.

    Google Scholar 

  • National Assessment Program – Literacy and Numeracy (NAPLAN). (2013). Retrieved from http://www.nap.edu.au/.

  • National Association of School Psychologists. (2010). Model for comprehensive and integrated school psychological services. Bethesda, MD: National Association of School Psychologists. Retrieved from: http://www.nasponline.org/standards/2010standards/2_PracticeModel.pdf.

    Google Scholar 

  • Neely, L., Davis, H., Davis, J., & Rispoli, M. (2015). Review of reliability and treatment integrity trends in autism-focused research. Research in Autism Spectrum Disorders, 9(1), 1–12. doi:10.1016/j.rasd.2014.09.011.

    Article  Google Scholar 

  • No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107-110, § 115, Stat. 1425 (2002).

    Google Scholar 

  • Reynolds, C. R., & Kamphaus, R. W. (2009). BASC-2 progress monitor manual. Bloomington, MN: Pearson.

    Google Scholar 

  • Riley-Tillman, C., & Burns, M. K. (2009). Evaluating educational interventions: Single-case designs for measuring response to intervention. New York, NY: Guilford Press.

    Google Scholar 

  • Sandall, S. R., Schwartz, I. S., & Lacroix, B. (2004). Interventionists’ perspectives about data collection in integrated early childhood classrooms. Journal of Early Intervention, 26, 161–174. doi:10.1177/105381510402600301.

    Article  Google Scholar 

  • Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction To the special series. School Psychology Review, 38, 445–459. Retrieved from: http://www.nasponline.org/.

    Google Scholar 

  • Sanetti, L. M., & Kratochwill, T. R. (Eds.). (2014). Treatment integrity: A foundation for evidence-based practice in applied psychology. Washington, DC: American Psychological Association.

    Google Scholar 

  • Shapiro, E. S. (2003). Behavioral Observation of Students in Schools (BOSS). Computer Software. San Antonio, TX: Psychological Corporation.

    Google Scholar 

  • Shapiro, E. S., & Heick, P. F. (2004). School psychologist assessment practices in the evaluation of students referred for social/behavioural/emotional problems. Psychology in the Schools, 41, 551–561. doi:10.1002/pits.10176.

    Article  Google Scholar 

  • Sheridan, S. M., Swanger-Gagné, M., Welch, G. W., Kwon, K., & Garbacz, S. A. (2009). Fidelity measurement in consultation: Psychometric issues and preliminary examination. School Psychology Review, 38(4), 476–495.

    Google Scholar 

  • Shinn, M. R. (Ed.). (1998). Advanced applications of curriculum-based measurement. New York: Guilford.

    Google Scholar 

  • Shinn, M. R. (2007). Identifying students at risk, monitoring performance, and determining eligibility within response to intervention: Research on educational need and benefit from academic intervention. School Psychology Review, 36, 601–617.

    Google Scholar 

  • Shinn, M. R. (2010). Building a scientifically based data system for progress monitoring and universal screening across three tiers, including RTI using curriculum-based measurement. In M. R. Shinn & H. M. Walker (Eds.), Interventions for achievement and behaviour problems in a three-tier model including RTI (pp. 259–292). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • Slee, P. T., Lawson, M. J., Russell, A., Askell-Williams, H., Dix, K. L., Owens, L., et al. (2009). KidsMatter Primary Evaluation Final Report. Centre for Analysis of Educational Futures, Flinders University of South Australia.

    Google Scholar 

  • Stecker, P. M., Fuchs, L. S., & Fuchs, D. (2005). Using curriculum-based measurement to improve student achievement: Review of research. Psychology in the Schools, 42(8), 795–819. doi:10.1002/pits.20113.

    Article  Google Scholar 

  • Stoiber, K. C., & Kratochwill, T. R. (2002). Outcomes: Planning, monitoring, evaluating. San Antonio, TX: The Psychological Corporation.

    Google Scholar 

  • Sugai, G., & Horner, R. H. (2006). A promising approach for expanding and sustaining school-wide positive behaviour support. School Psychology Review, 35, 245–259.

    Google Scholar 

  • Sugai, G., Horner, R. H., & Lewis-Palmer, T. (2001). Team implementation checklist. Eugene, OR: Educational and Community Supports, University of Oregon.

    Google Scholar 

  • Sugai, G., Lewis-Palmer, T. L., Todd, A. W., & Horner, R. H. (2001). School-wide evaluation tool (SET). Eugene, OR: Center for Positive Behavioral Supports, University of Oregon.

    Google Scholar 

  • Technical Assistance Center on Positive Behavioral Interventions and Supports. (2010). Implementation blueprint and self-assessment: Positive behavioural interventions and supports. Washington, DC: Technical Assistance Center on Positive Behavioral Interventions and Supports. Retrieved from http://www.pbis.org/common/pbisresources/publications/SWPBS_ImplementationBlueprint_vSep_23_2010.pdf.

    Google Scholar 

  • University of Oregon Center on Teaching and Learning. (2009). Dynamic indicators of basic early literacy skills. Retrieved January 10, 2009, from http://dibels.uoregon.edu/.

  • Upah, K. R. F. (2008). Best practices in designing, implementing, and evaluating quality interventions. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (Vth ed., pp. 209–223). Bethesda, MD: National Association of School Psychologist.

    Google Scholar 

  • VanDerHeyden, A. M., & Burns, M. K. (2005). Using curriculum-based assessment and curriculum-based measurement to guide elementary mathematics instruction: Effect on individual and group accountability scores. Assessment for Effective Intervention, 30(3), 15–31. doi:10.1177/073724770503000302.

    Article  Google Scholar 

  • VanDerHeyden, A., & Harvey, M. (2013). Using data to advance learning outcomes in schools. Journal of Positive Behavior Interventions, 15(4), 205–213. doi:10.1177/1098300712442387.

    Article  Google Scholar 

  • Vollmer, T. R., Roane, H. S., Ringdahl, J. E., & Marcus, B. A. (1999). Evaluating treatment challenges with differential reinforcement of alternative behavior. Journal of Applied Behavior Analysis, 32(1), 9–23. doi:10.1901/jaba.1999.32-9.

    Article  PubMed Central  Google Scholar 

  • Volpe, R. J., DiPerna, J. C., Hintze, J. M., and Shapiro, E. S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454–474.

    Google Scholar 

  • Waldron, N., & Prus, J. (2006). A guide for performance-based assessment, accountability, and program development in school psychology training programs (2nd ed.). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • Walker, H. M., & Shinn, M. R. (2010). Systemic, evidence-based approaches for promoting positive student outcomes within a multitier framework: Moving from efficacy to effectiveness. In M. R. Shinn & H. M. Walker (Eds.), Interventions for achievement and behaviour problems in a three-tier model including RTI (pp. 1–26). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • Watson, T. S., & Watson, T. S. (2009). Behavioral assessment in the schools. In A. Akin-Little, S. G. Little, M. A. Bray, & T. J. Kehle (Eds.), Behavioral interventions in schools: Evidence-based positive strategies (pp. 27–41). Washington, DC: American Psychological Association. doi:10.1037/11886-002.

    Chapter  Google Scholar 

  • Wilder, D. A., Atwell, J., & Wine, B. (2006). The effects of varying levels of treatment integrity on child compliance during treatment with a three-step prompting procedure. Journal of Applied Behavior Analysis, 39, 369–373. doi:10.1901/jaba.2006.144-05.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wolery, M., Bailey, D. B., & Sugai, G. M. (1988). Effective teaching: Principles and procedures of applied behaviour analysis with exceptional children. Boston: Allyn & Bacon.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Old Dominion University, 110 Lions Child Study Center, Norfolk, VA, 23529, USA

    Laura C. Chezan

  2. University of Wisconsin, Madison, WI, USA

    Thomas R. Kratochwill

  3. St. John’s University, New York, NY, USA

    Mark D. Terjesen & Kim Van H. Nguyen

Authors
  1. Laura C. Chezan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas R. Kratochwill
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark D. Terjesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kim Van H. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura C. Chezan .

Editor information

Editors and Affiliations

  1. Department of Psychological Sciences, Swinburne University of Technology, Hawthorn, Victoria, Australia

    Monica Thielking

  2. St. John’s University, Jamaica, New York, USA

    Mark D. Terjesen

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Chezan, L.C., Kratochwill, T.R., Terjesen, M.D., Nguyen, K.V.H. (2017). Measuring Outcomes in Schools. In: Thielking, M., Terjesen, M. (eds) Handbook of Australian School Psychology. Springer, Cham. https://doi.org/10.1007/978-3-319-45166-4_35

Download citation

Publish with us

Policies and ethics

Search

Navigation