Procedures and Accuracy of Discontinuous Measurement of Problem Behavior in Common Practice of Applied Behavior Analysis

Abstract

Discontinuous measurement involves dividing an observation into intervals and recording whether a behavior occurred during some or all of each interval (i.e., interval recording) or at the exact time of observation (i.e., momentary time sampling; MTS). Collecting discontinuous data is often easier for observers than collecting continuous data, but it also produces more measurement error. Smaller intervals (e.g., 5 s, 10 s, 15 s) tend to produce less error but may not be used in everyday practice. This study examined the most common intervals used by a large sample of data collectors and evaluated the effect of these intervals on measurement error. The most commonly used intervals fell between 2 and 5 min. We then analyzed over 800 sessions to evaluate the correspondence between continuous and discontinuous data at each commonly used interval. Intervals of 3 min or less produced the greatest correspondence, and MTS outperformed interval recording.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. Alvero, A. M., Struss, K., & Rappaport, E. (2008). Measuring safety performance: A comparison of whole, partial, and momentary time-sampling recording methods. Journal of Organizational Behavior Management, 27, 1–28. https://doi.org/10.1300/J075v27n04_01.

    Article  Google Scholar 

  2. Baer, D. M., Wolf, M. M., & Risley, T. (1968). Current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97. https://doi.org/10.1901/jaba.1968.1-91.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Beavers, G. A., Iwata, B. A., & Lerman, D. C. (2013). Thirty years of research on the functional analysis of problem behavior. Journal of Applied Behavior Analysis, 46, 1–21. https://doi.org/10.1002/jaba.30.

    Article  PubMed  Google Scholar 

  4. Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River, NJ: Pearson.

    Google Scholar 

  5. Devine, S. L., Rapp, J. T., Testa, J. R., Henrickson, M. L., & Schnerch, G. (2011). Detecting changes in simulated events using partial-interval recording and momentary time sampling III: Evaluating sensitivity as a function of session length. Behavioral Interventions, 26, 103–124. https://doi.org/10.1002/bin.328.

    Article  Google Scholar 

  6. Fiske, K., & Delmolino, L. (2012). Use of discontinuous methods of data collection in behavioral intervention: Guidelines for practitioners. Behavior Analysis in Practice, 5(2), 77–81. https://doi.org/10.1007/BF03391826.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Gardenier, N. C., MacDonald, R., & Green, G. (2004). Comparison of direct observational methods for measuring stereotypic behavior in children with autism spectrum disorders. Research in Developmental Disabilities, 25, 99–118. https://doi.org/10.1016/j.ridd.2003.05.004.

    Article  PubMed  Google Scholar 

  8. Guntner, P. L., Venn, M. L., Patrick, J., Miller, K. A., & Kelly, L. (2003). Efficacy of using momentary time samples to determine on-task behavior of students with emotional/behavioral disorders. Education and Treatment of Children, 26, 400–412.

    Google Scholar 

  9. Hanley, G. P., Cammilleri, A. P., Tiger, J. H., & Ingvarsson, E. T. (2007). A method for describing preschoolers’ activity preferences. Journal of Applied Behavior Analysis, 40, 603–618. https://doi.org/10.1901/jaba.2007.603-618.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Harrop, A., & Daniels, M. (1986). Methods of time sampling: A reappraisal of momentary time sampling and partial interval recording. Journal of Applied Behavior Analysis, 19, 73–77. https://doi.org/10.1901/jaba.1993.26-277.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Jessel, J., Metras, R., Hanley, G. P., Jessel, C., & Ingvarsson, E. T. (in press). Evaluating the boundaries of analytic efficiency and control: A consecutive controlled case series of 26 functional analyses. Journal of Applied Behavior Analysis. https://doi.org/10.1002/jaba.544.

  12. Johnston, J. M., & Pennypacker, H. S. (2009). Observing and recording. In Strategies and tactics of behavioral research (3rd ed., pp. 115–138). New York, NY: Routledge.

  13. Kolt, L. D., & Rapp, J. T. (2014). Assessment of therapists’ preferences for discontinuous measurement systems. Behavioral Interventions, 29, 304–314. https://doi.org/10.1002/bin.1392.

    Article  Google Scholar 

  14. LeBlanc, L. A., Raetz, P. B., Sellers, T. P., & Carr, J. E. (2016). A proposed model for selecting measurement procedures for the assessment and treatment of problem behavior. Behavior Analysis in Practice, 9, 77–83. https://doi.org/10.1007/s40617-015-0063-2.

    Article  Google Scholar 

  15. Machado, M. A., Luczynski, K. C., & Hood, S. A. (2019). Evaluation of the accuracy, reliability, efficiency and acceptability of fast forwarding to score problem behavior. Journal of Applied Behavior Analysis, 50, 315–334. https://doi.org/10.1002/jaba.510.

    Article  Google Scholar 

  16. Meany-Daboul, M. G., Roscoe, E. M., Bourret, J. C., & Ahearn, W. H. (2007). A comparison of momentary time sample and partial-interval recording for evaluating functional relations. Journal of Applied Behavior Analysis, 40, 501–514. https://doi.org/10.1901/jaba.2007.40-501.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Morris, C., et al. (in press). Using Microsoft Excel® to build a customized partial-interval data collection system. Behavior Analysis in Practice.

  18. Mudford, O. C., Beale, I. L., & Singh, N. N. (1990). The representativeness of observational samples of different durations. Journal of Applied Behavior Analysis, 23, 323–331. https://doi.org/10.1901/jaba.1990.23-323.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Mudford, O. C., Taylor, S. A., & Martin, N. T. (2009). Continuous recording and interobserver agreement algorithms reported in the. Journal of Applied Behavior Analysis, 42, 165–169. https://doi.org/10.1901/jaba.2009.42-165.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Powell, J. (1984). On the misrepresentation of behavioral realities by a widely practiced direct observation procedure: Partial interval (one-zero) sampling. Behavioral Assessment, 6, 209–219.

    Google Scholar 

  21. Powell, J., Martindale, A., & Kulp, S. (1975). An evaluation of time-sample measures of behavior. Journal of Applied Behavior Analysis, 8, 463–469. https://doi.org/10.1901/jaba.1975.8-463.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Powell, J., Martindale, B., Kulp, S., Martindale, A., & Bauman, R. (1977). Taking a closer look: Time sampling and measurement error. Journal of Applied Behavior Analysis, 10, 325–332. https://doi.org/10.1901/jaba.1975.8-463.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Rapp, J. T., Colby, A. M., Vollmer, T., Roan, H. S., Lomas, J., & Britton, L. N. (2007). Interval recording for duration events: A re-evaluation. Behavioral Interventions, 22, 319–345. https://doi.org/10.1002/bin.239.

    Article  Google Scholar 

  24. Rapp, J. T., Colby-Dirksen, A. M., Michalski, D. N., Carroll, R. A., & Lindenberg, D. N. (2008). Detecting changes in simulated events using partial-interval recording and momentary time sampling. Behavioral Interventions, 23, 237–269. https://doi.org/10.1002/bin.269.

    Article  Google Scholar 

  25. Saini, V., Fisher, W. W., & Retzlaff, B. J. (2018). Predictive validity and efficiency of ongoing visual-inspection criteria for interpreting functional analyses. Journal of Applied Behavior Analysis, 51, 303–320. https://doi.org/10.1002/jaba.450.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Sanson-Fisher, R. W., Poole, A. D., & Dunn, J. (1980). An empirical method for determining an appropriate interval length for recording behavior. Journal of Applied Behavior Analysis, 13, 493–500. https://doi.org/10.1901/jaba.1980.13-493.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Saudargas, R. A., & Zanolli, K. (1990). Momentary time sampling as an estimate of percentage time: A field validation. Journal of Applied Behavior Analysis, 4, 533–537. https://doi.org/10.1901/jaba.1990.23-533.

    Article  Google Scholar 

  28. Schmidt, M. G., Rapp, J. T., Novotny, M., & Lood, E. (2013). Detecting changes in non-simulated events using partial-interval recording and momentary time sampling: Evaluating false positives, false negatives, and trending. Behavioral Interventions, 28, 58–81. https://doi.org/10.1002/bin.1354.

    Article  Google Scholar 

  29. Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York, NY: Basic Books.

    Google Scholar 

  30. Wirth, O., Slaven, J., & Taylor, M. A. (2013). Interval sampling methods and measurement error: A computer simulation. Journal of Applied Behavior Analysis, 47, 83–100. https://doi.org/10.1002/jaba.93.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author Note

The authors thank Jamila Pitts for assistance with coding data from research articles.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Linda A. LeBlanc.

Ethics declarations

Conflict of Interest

The authors of this manuscript are employed by or serve on the advisory board of DataFinch Technologies (i.e., Catalyst).

Informed Consent

All agencies whose data were included in the analysis provided consent.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Research Highlights

• Practitioners often use discontinuous measurement intervals that are longer than recommended in the empirical literature.

• Intervals and samples longer than 3 min produced error on every metric examined, whereas those at or below 3 min were generally quite accurate.

• The default settings in an electronic data-collection system were often the ones used in programming.

• The level of problem behavior in the session systematically impacted the correlations between discontinuous and continuous measures with very small amounts of problem behavior producing the lowest correlations at all values.

• Practitioners should use small intervals or samples for discontinuous measurement and should not use discontinuous measurement of any type for very low amounts of problem behavior.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

LeBlanc, L.A., Lund, C., Kooken, C. et al. Procedures and Accuracy of Discontinuous Measurement of Problem Behavior in Common Practice of Applied Behavior Analysis. Behav Analysis Practice 13, 411–420 (2020). https://doi.org/10.1007/s40617-019-00361-6

Download citation

Keywords

  • Data collection
  • Discontinuous measurement
  • Partial interval
  • Momentary time sample
  • Observation interval