Skip to main content
SpringerLink
Log in

Methodology Challenges and Cutting Edge Designs for Rural Education Research

  • Chapter
  • First Online:
Rural Education Research in the United States

  • 849 Accesses

Abstract

The unique contexts and features of rural education systems lead to the need for unique and innovative solutions. In particular, rural research is commonly perceived to face major logistical research hurdles such as small populations, low densities, poor access, and geographic isolation. These limitations make the rural setting a challenging context within which to conduct education research. This chapter presents considerations for overcoming such challenges while still striving towards employing rigorous methodologies, achieving desired generalizability, and reaching causal inferences when relevant. To accomplish this, a number of interdisciplinary statistical and design-based solutions can be translated to rural education research. In particular, this chapter discusses: (a) using advanced statistical modeling to preserve and feature the uniqueness of rural settings, (b) alternatives to traditional simple random assignment, (c) measurement paradigms to reduce the amount of data required, and (d) innovations for working with small samples and complex models. Most of these topics and approaches can be combined to accommodate the complexities and realities of conducting rural research. The fundamental message is that all research contexts present their own unique challenges, but as researchers, we can look outside of our disciplines to find solutions that can help us pursue our necessary research agendas.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Participants in the control condition are also observed on a total of eight occasions, with no phase shift, creating a counterfactual condition for the larger RCT.

  2. 2.

    The CBC in Rural Communities project was still undergoing final data analysis at the time that this chapter was written, preventing its use as an example in this chapter.

References

  • Allison, D. B., Allison, R. L., Faith, M. S., Paultre, F., & Pi-Sunyer, F. X. (1997). Power and money: Designing statistically powerful studies while minimizing financial costs. Psychological Methods, 2, 20–33.

    Article  Google Scholar 

  • Armitage, P. (1975). Sequential medical trials (2nd ed.). New York: Wiley.

    Google Scholar 

  • Arnold, M., Newman, J., Gaddy, B., & Dean, C. (2005). A look at the condition of rural education research: Setting a direction for future research. Journal of Research in Rural Education, 20, 1–25.

    Google Scholar 

  • Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97.

    Article  Google Scholar 

  • Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). New York: Pergamon Press.

    Google Scholar 

  • Bell, R. Q. (1953). Convergence: An accelerated longitudinal approach. Child Development, 24(2), 145–152.

    Article  Google Scholar 

  • Boker, S. M., & Laurenceau, J. (2008). Coupled dynamics and mutually adaptive context. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies (pp. 299–324). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Bolger, N., & Shrout, P. E. (2007). Accounting for statistical dependency in longitudinal data on dyads). In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies (pp. 285–298). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Bovaird, J. A. (2007). Multilevel structural equation models for contextual factors. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Bovaird, J. A. (2010, August). Exploring sequential design of cluster randomized trials. Paper presented at the American Psychological Association annual meeting. San Diego, CA.

    Google Scholar 

  • Bovaird, J. A., Chumney, F., & Wu, C. (2008, June). On the finite population correction in multilevel modeling: Implications for nesting within geographic region. Paper presented at the International Meeting of the Psychometric Society. Durham, NH.

    Google Scholar 

  • Bovaird, J. A., Koziol, N. A., & Chumney, F. (2011, October). An ecological model of school readiness: A methodological investigation into the role of rurality. Paper presented at the Building Solutions to Poverty: Methods and Metrics for Identifying Success Conference. Columbus, OH.

    Google Scholar 

  • Bovaird, J. A., Martinez, S., & Stuber, G. (2006, August). Multilevel structural equation modeling of kindergarten readiness with finite samples. Paper presented at the American Psychological Association annual meeting. New Orleans, LA.

    Google Scholar 

  • Bovaird, J. A., Sheridan, S. M., Glover, T. A., & Garbacz, S. A. (2009, June). Fixed vs. sequential experimental designs: implications for cluster randomized trials in education. Paper presented at the Institute for Education Science Research Conference. Washington, DC.

    Google Scholar 

  • Box, G. E., Hunter, J. S., & Hunter, W. G. (2005). Statistics for experimenters: Design, innovation, and discovery. Hoboken, NJ: Wiley.

    Google Scholar 

  • Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. Developmental Psychology, 22, 723–742.

    Article  Google Scholar 

  • Bronfenbrenner, U. (1988). Interacting systems in human development. Research paradigms: Present and future. In N. Bolger, A. Caspi, G. Downey, & M. Moorehouse (Eds.), Persons in context: Developmental processes (pp. 25–49). New York: Cambridge University Press.

    Google Scholar 

  • Brown, C. A., & Lilford, R. J. (2006). The stepped wedge trial design: A systematic review. BMC Medical Research Methodology, 6, 1–9.

    Article  Google Scholar 

  • Burstein, L., Linn, R. L., & Cappel, F. J. (1978). Analyzing multilevel data in the presence of heterogeneous within-class regression. Journal of Educational Statistics, 3, 347–383.

    Article  Google Scholar 

  • Campbell, D. T. (1969). Reforms as experiments. American psychologist, 24(4), 409–429. Retrieved from http://www.cem.org/attachments/publications/CEMWeb027%20Reforms%20As%20Experiments.pdf

  • Chernoff, H. (1959). Sequential design of experiments. The Annals of Mathematical Statistics, 30, 755–770.

    Article  Google Scholar 

  • Chin, W. W. (1998). The partial least squares approach to structural equation modeling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). Mahwah, NJ: Lawrence Erlbaum Associates Inc.

    Google Scholar 

  • Chin, W. W. (2001). PLS-Graph user’s guide version 3.0. Houston, TX: C.T. Bauer College of Business, University of Houston.

    Google Scholar 

  • Christ, T. J. (2007). Experimental control and threats to internal validity of concurrent and nonconcurrent multiple baseline designs. Psychology in the Schools, 44, 451–459.

    Article  Google Scholar 

  • Chromy, J. R. (1998). The effects of finite sampling corrections on state assessment sample requirements. Palo Alto, CA: NAEP Validity Studies, American Institutes for Research.

    Google Scholar 

  • Cochran, W. G. (1977). Sampling techniques (3rd ed.). New York: Wiley.

    Google Scholar 

  • Coladarci, T. (2007). Improving the yield of rural education research: An editor’s swan song. Journal of Research in Rural Education, 22(3). Retrieved from http://jrre.psu.edu/articles/22-3.pdf

  • Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.

    Google Scholar 

  • Enders, C. K. (2010). Applied missing data analysis. New York: Guilford Press.

    Google Scholar 

  • Fu, J. R. (2006). VisualPLS—Partial least square (PLS) regression—An enhanced GUI for lvpls (PLS 1.8 PC) version 1.04. Taiwan, ROC: National Kaohsiung University of Applied Sciences.

    Google Scholar 

  • Gamse, B. C., Jacob, R. T., Horst, M., Boulay, B., & Unlu, F. (2008). Reading first impact study final report (NCEE 2009-4038). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

    Google Scholar 

  • Goldstein, H. I. (1986). Multilevel mixed linear model analysis using iterative general least squares. Biometrika, 73, 43–56.

    Article  Google Scholar 

  • Graham, J. W., Hofer, S. M., & Piccinin, A. M. (1994). Analysis with missing data in drug prevention research. In L. M. Collins & L. Seitz (Eds.), Advances in data analysis for prevention intervention research: National Institute on Drug Abuse Research monograph series (Vol. 142, pp. 13–63). Washington, DC: National Institute on Drug Abuse.

    Google Scholar 

  • Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned missing data designs in psychological research. Psychological Methods, 11, 323–343. doi:10.1037/1082-989X.11.4.323

    Article  Google Scholar 

  • Grimm, K. J., & McArdle, J. J. (2007). A dynamic structural analysis of the impacts of context on shifts in lifespan cognitive development. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies (pp. 363–368). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Harmon, H., Henderson, S., & Royster, W. (2003). A research agenda for improving science and mathematics education in rural schools. Journal of Research in Rural Education, 18, 52–58.

    Google Scholar 

  • Hawley, L., Koziol, N., & Bovaird, J. A. (this volume). Defining and communicating rural: An overview and empirical illustration. In G. C. Nugent, G. M. Kunz, S. M. Sheridan, T. A. Glover, & L. L. Knoche (Eds.) Rural education research: State of the Science and emerging directions. New York: Springer.

    Google Scholar 

  • Hussey, M. A., & Hughes, J. P. (2007). Design and analysis of stepped wedge cluster randomized trials. Contemporary Clinical Trials, 28, 182–191.

    Article  Google Scholar 

  • Hwang, H., & Takane, Y. (2004). Generalized structured component analysis. Psychometrika, 69(1), 81–99.

    Article  Google Scholar 

  • Johnson, E. G. (1992). The design of the national assessment of educational progress. Journal of Educational Measurement, 29, 95–110.

    Article  Google Scholar 

  • Kaplan, D. (2014). Bayesian statistics for the social sciences. New York: Guilford Press.

    Google Scholar 

  • Kazak, A. E., Rourke, M. T., & Crump, T. A. (2003). Families and other systems in pediatric psychology. In M. C. Roberts (Ed.), Handbook of pediatric psychology (3rd ed., pp. 159–175). New York: Guilford.

    Google Scholar 

  • Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford.

    Google Scholar 

  • Kreft, I. G. G., & de Leeuw, J. (1998). Introduction to multilevel modeling. London: Sage.

    Book  Google Scholar 

  • Kunz, G. M., Nugent, G. C., Pedersen, J. E., DeChenne, S. E., & Houston, J. (2013). Meeting rural science teachers’ needs: Professional development with on-going technology-delivered instructional coaching (R2Ed Working Paper No. 2013-8). Retrieved from the National Center for Research on Rural Education: http:\\r2ed.unl.edu

  • Laird, N. M., & Ware, J. H. (1982). Random effects models for longitudinal data. Biometrics, 38, 963–974.

    Article  Google Scholar 

  • Lee, S., & Song, X. (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39, 653–686.

    Article  Google Scholar 

  • Lei, P. W., & Wu, Q. (2012). Estimation in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 164–180). New York: Guilford Press.

    Google Scholar 

  • Little, T. D., Bovaird, J. A., & Card, N. A. (Eds.). (2007). Modeling contextual effects in longitudinal studies. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Longford, N. T. (1987). A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested effects. Biometrika, 74, 817–827.

    Article  Google Scholar 

  • McArdle, J. J. (1994). Structural factor analysis experiments with incomplete data. Multivariate Behavioral Research, 29, 409–454.

    Article  Google Scholar 

  • Nelson, T. D., Aylward, B. S., & Steele, R. G. (2008). Structural equation modeling in pediatric psychology: Overview and review of applications. Journal of Pediatric Psychology, 33, 679–687. doi:10.1093/jpepsy/jsm107

    Article  Google Scholar 

  • Nesselroade, J. R., & Baltes, P. B. (1979). Longitudinal research in the study of behavior and development. San Diego, CA: Academic Press.

    Google Scholar 

  • Nugent, G. C., Kunz, G. M., Pedersen, J., Houston, J., & Lee, S. C. (this volume). Technology-based coaching for rural teachers’ science instruction. In G. C. Nugent, G. M. Kunz, S. M. Sheridan, T. A. Glover, & L. L. Knoche (Eds.) Rural education research: State of the Science and emerging directions. New York: Springer.

    Google Scholar 

  • Oliver, J. S. (2007). Rural science education research and the frameworks that give it form. The Rural Educator, 28, 1–3.

    Google Scholar 

  • Power, T., DuPaul, G., Shapiro, E., & Kazak, A. (2003). Promoting children’s health: Integrating school, family, and community. New York: Guilford.

    Google Scholar 

  • Raghunathan, T. E., & Grizzle, J. E. (1995). A split questionnaire survey design. Journal of the American Statistical Association, 90, 54–63. doi:10.1080/01621459.1995.10476488

    Article  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Ringle, C. M., Wende, S., & Will A. (2005). SmartPLS 2.0 (beta). Computer software. Downloaded from www.smartpls.de

  • Schaie, K. W. (1965). A general model for the study of developmental problems. Psychological Bulletin, 64, 92–107.

    Article  Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin.

    Google Scholar 

  • Sheridan, S. M., Bovaird, J. A., Glover, T. G., Garbacz, S. A., Witte, A., & Kwon, K. (2012). A randomized trial examining the effects of conjoint behavioral consultation and the mediating role of the parent–teacher relationship. School Psychology Review, 41, 23–46.

    Google Scholar 

  • Sheridan, S. M., Kunz, G. M., Witte, A., Holmes, S., & Coutts, M. (2014). Rural parents and teachers as partners: Preliminary results of a randomized trial (R2Ed Working Paper No. 2014-4). Retrieved from the National Center for Research on Rural Education: http:\\r2ed.unl.edu

  • Shoemaker, D. M. (1973). Principles and procedures of multiple matrix sampling. Cambridge, MA: Ballinger Publishing Company.

    Google Scholar 

  • Snijders, T., & Bosker, R. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Thousand Oakes, CA: Sage.

    Google Scholar 

  • Steele, R. G., & Aylward, B. S. (2010). An overview of systems in pediatric psychology research and practice. In M. C. Roberts & R. G. Steele (Eds.), Handbook of pediatric psychology (4th ed.). New York: Guilford.

    Google Scholar 

  • Stroup, W. W. (2013). Generalized linear mixed models: Modern concepts, methods and applications. Boca Raton, FL: CRC Press.

    Google Scholar 

  • Tonry, M., Ohlin, L. E., & Farrington, D. P. (1991). Human development and criminal behavior: New ways of advancing knowledge. New York: Springer.

    Book  Google Scholar 

  • U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, March). What works clearinghouse: Procedures and standards handbook (version 3.0). Retrieved from http://whatworks.ed.gov

  • Wald, A. (1945). Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics, 16, 117–186.

    Article  Google Scholar 

  • Watson, P. J., & Workman, E. A. (1981). The non-concurrent multiple baseline across-individuals design: An extension of the traditional multiple baseline design. Journal of Behavior Therapy and Experimental Psychiatry, 12, 257–259.

    Article  Google Scholar 

  • Wendorf, C. A. (2002). Comparisons of structural equation modeling and hierarchical linear modeling approaches to couples’ data. Structural Equation Modeling, 9, 126–140.

    Article  Google Scholar 

  • Woertman, W., de Hoop, E., Moerbeek, M., Zuidema, S. U., Gerritsen, D. L., & Teerenstra, S. (2013). Stepped wedge designs could reduce the required sample size in cluster randomized trials. Journal of Clinical Epidemiology, 66, 752–758.

    Article  Google Scholar 

  • Wold, H. (1975). Path models with latent variables: The NIPALS approach. In H. M. Blalock, A. Aganbegian, F. M. Borodkin, R. Boudon, & V. Capecchi (Eds.), Quantitative sociology: International perspectives on mathematical and statistical modeling (pp. 307–357). New York: Academic.

    Chapter  Google Scholar 

  • Wold, H. (1982). Soft modeling: The basic design and some extensions. In H. Wold & K. Jöreskog (Eds.), Systems under indirect observation: Causality, structure, prediction II (pp. 589–591). Amsterdam: North-Holland.

    Google Scholar 

Download references

Acknowledgments

Preparation of this manuscript was supported by a grant awarded to Susan M. Sheridan and colleagues (IES #R305C090022) by the Institute of Education Sciences. The opinions expressed herein are those of the author and should not be considered reflective of the funding agency.

Author information

Authors and Affiliations

  1. National Center for Research on Rural Education (R²Ed), The Nebraska Center for Research on Children, Youth, Families & Schools (CYFS), University of Nebraska–Lincoln, Lincoln, USA

    James A. Bovaird & Kirstie L. Bash

  2. University of Nebraska–Lincoln, 114 Teachers College Hall, Lincoln, NE, 68588-0345, USA

    James A. Bovaird

Authors
  1. James A. Bovaird
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kirstie L. Bash
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James A. Bovaird .

Editor information

Editors and Affiliations

  1. University of Nebraska - Lincoln Nat. Center for Research on Rural Edu, Lincoln, Nebraska, USA

    Gwen C. Nugent

  2. University of Nebraska - Lincoln Nat. Center for Research on Rural Edu, Lincoln, Nebraska, USA

    Gina M. Kunz

  3. University of Nebraska - Lincoln Nat. Center for Research on Rural Edu., Lincoln, Nebraska, USA

    Susan M. Sheridan

  4. Rutgers University, Piscataway, New Jersey, USA

    Todd A. Glover

  5. University of Nebraska-Lincoln, Lincoln, Nebraska, USA

    Lisa L. Knoche

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Bovaird, J.A., Bash, K.L. (2017). Methodology Challenges and Cutting Edge Designs for Rural Education Research. In: Nugent, G., Kunz, G., Sheridan, S., Glover, T., Knoche, L. (eds) Rural Education Research in the United States. Springer, Cham. https://doi.org/10.1007/978-3-319-42940-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42940-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42938-0

  • Online ISBN: 978-3-319-42940-3

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation