Abstract
This chapter describes the experiences from the most recent phase of a 7-year research project in South Africa on school-based monitoring of pupil performance in some 22 primary schools. The project aimed to generate knowledge as well as to design and develop a well-functioning feedback system to provide data to schools on learner performance. The feedback system that was developed is known as the South African Monitoring system for Primary schools (SAMP). A key objective of this phase of the project was to evaluate the use of the performance data at school and classroom level and to design an intervention for effective use of the data within the primary school environment. It is hoped that a deeper understanding of how data travel in schools (data paths) and how schools can appropriately use data may assist policymakers in developing monitoring policies and provide guidance to school leaders and teachers. This chapter focuses on the data generated through observations, journals, and interviews in the evaluation of one of these design cycles. The sample consists of three schools participating in SAMP that were purposefully selected. The evaluation data collected during this cycle of development focused particularly on how data were used by schools and how data moved within the schools. Three distinct approaches to data use that appeared to be appropriate for their specific contexts (schools) were identified: Team, Cascade, and Top-down. The data suggest that the most appropriate and effective approach of use may depend on the culture of the school, school leadership approach, level of teacher development, and context and level of functioning of the school. There are, however, certain commonalities in the approaches to effective data use. An effective feedback system should thus try to establish or encourage these conditions for data use. The data in this chapter seem to suggest that policy on data use should be flexible and provide exemplars of various possible approaches, which are appropriate for different contexts. It is important that there are layers of sophistication (different levels of detail, complexity of presentation, and disaggregation) within the data, which the school can access as needed for its particular milieu.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The concept of culture, as used here, has many facets, it includes a certain cultural heritage associated with a specific language and racial group, the level of urbanization of staff, the sense of school culture, specifically the spirit of collaboration, approach to shared learning, and a drive for improvement. It also encapsulates the values and norms seen as part of the school culture. All of these factors may have an influence on whether or not data are seen as important and valuable in school improvement. These factors may influence both the likelihood that data will be used as well as the approach to data use that will be employed. For instance, in a culture that highly values communal processes, a team approach may be employed to work with data while in another culture data may be seen as the domain of those in leadership positions and therefore would not be interrogated and interacted with independently by staff members.
- 2.
For an in-depth discussion of the cycles of design, development, implementation, and evaluation of the SAMP feedback prototypes, see Archer (2010).
- 3.
Naturalistic observations are unstructured observations where the observer is a non-participant observer and tries to capture all that takes place in as natural as possible form by trying to minimize interference due to his or her presence.
- 4.
Schools in South Africa are categorized into quintiles for each province based on rates of income, unemployment, and illiteracy in catchment area. Quintile 1 represents the poorest schools, while quintile 5 represents the least poor schools. The quintile system is used to allocate funds differentially to schools in order to redress the large difference between schools (Van den Berg and Burger 2002).
- 5.
The particular data collection method from which the data originated is indicated in brackets. In this case, the quotation was taken from an interview with Pieter, the principal.
- 6.
Policies are referred to here as relating to a country, it may, however, refer to policies for a specific region, state, or province depending on how a specific county’s education system is structured.
References
Archer, E. (2010). Bridging the gap: Optimising a feedback system for monitoring learner performance. Pretoria: University of Pretoria.
Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–75.
Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Retrieved June 7, 2004, from http://www.pdkintl.org/kappan/kbla9810.htm.
Campbell, C., & Levin, B. (2009). Using data to support educational improvement. Educational Assessment, Evaluation and Accountability, 21(1), 47–65.
Cradler, J. (2008). Data-driven decision making and the electronic learning assessment resources (ELAR). Retrieved May 12, 2010 from http://www.clrn.org/elar/dddm.cfm.
De Villiers, M. R. (2005). Three approaches as pillars for interpretive information systems research: Development research, action research and grounded theory. Research for a changing world: Proceedings of South African Institute for Computer Scientists and Information Technologists 2005 conference (pp. 111–120). White River: SAICSIT.
Department of Education. (2002). Revised National Curriculum Statement Grades R-9 (schools). Retrieved October 25, 2006, from http://www.polity.org.za/pdf/Curriculum.pdf
Department of Education. (2006a). Grade 3 Foundation Phase National systemic evaluation report. Pretoria: Government Printer.
Department of Education. (2006b). Grade 6 Intermediate Phase Systemic Evaluation Report. Pretoria: Government Printer.
Department of Education. (2006c). The national policy framework for teacher education and development in South Africa: More teachers; better teachers. Pretoria: Department of Education.
Earl, L. M., & Katz, S. (2006). Leading schools in a data-rich world. London: Sage.
Education Labour Relations Council. (2003). Training manual on the Integrated Quality Management System. Retrieved April 16, 2007, from www.elrc.co.za/UploadedDocuments/IQMS%20Training%20Manual.doc.
Gawe, N., & Heyns, R. (2004). Quality assurance. In J. G. Maree & W. J. Fraser (Eds.), Outcomes-based assessment (pp. 159–184). Sandown: Heinemann.
Halverson, R. (2010). School formative feedback systems. Peabody Journal of Education, 85, 130–146.
Harris, A., Chapman, C., Muijs, D., Russ, J., & Stoll, L. (2006). Improving schools in challenging circumstance: Exploring the possible. School Effectiveness and School Improvement, 17(4), 399–407.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
Hibberd, B. J., & Evatt, A. (2004). Mapping information flows: A practical guide. The Information Management Journal, 1, 58–64.
Howie, S. J., Venter, E., Van Staden, S., Zimmerman, L., Long, C., Scherman, V., et al. (2008). Progress in International Reading Literacy Study (PIRLS) 2006 summary report: South African children’s reading lite racy achievement. Pretoria: University of Pretoria.
Howie, S. J. (2008). Measuring the health of the education system: Lessons from international studies for South Africa. Keynote address, presented at Association for the Study of Evaluation and Assessment national conference, Pretoria, July 2008.
Howie, S. J. (2010). ICT-supported pedagogical policies and practices in South Africa and Chile: Emerging economies and realities. Journal for Computer Assisted Learning, 26(6), 507–522.
Kamel, N., Narasipuram, M. M., & Toraskar, K. (1997). An approach to value-based modelling of information flows. The Information Society, 13, 93–105.
Kluger, A. N., & Denisi, A. (1996). The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284.
Krovi, R., Chandra, A., & Rajagopalan, B. (2003). Information flow parameters for managing organisational processes. Communications of the ACM, 46(2), 77–82
Mano, M. M., & Kime, C. R. (2004). Logic and computer design fundamentals (3rd ed). Englewood Cliffs: Prentice Hall.
Martin, M. O., Mullis, V. X., Gonzalez, E. J., & Chrostowski, S. J. (2004). TIMSS 2003 International science report. Boston: TIMSS & PIRLS International Study Centre.
Naker, D. (2007). What is a good school? Imagining beyond the limits of today to create a better tomorrow. Kampala: Raising Voices.
Nieveen, N. (1997). Computer support for curriculum developers: A study of the potential of computer support in the domain of formative curriculum evaluation. Enschede: PrintPartners Ipskamp.
Plomp, T. (2009). Educational design research: An introduction. In T. Plomp, & N. Nieveen (Eds.), An introduction to educational design research: Proceedings of the seminar conducted at the East China Nor mal University, Shanghai (PR China), November 23–26, 2007 (pp. 9–37). Enschede: SLO.
Richey, R. T., Klein, J. D., & Nelson, W. A. (1996). Developmental research. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1213–1245). New York: Macmillan.
Salpeter, J. (2004). Data: Mining with a mission. Technology, March, 30–36.
Saunders, L. (1999). Who or what is ‘self’-evaluation for? School Effectiveness and School Improvement, 10(4), 414–429.
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26, 482–496.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
The Urban Institute. (2004). Analysing outcome information: Getting the most of data. Retrieved November 5, 2009 from http://www.urban.org/UploadedPDF/310973_OutcomeInformation.pdf.
Thijs, A. (1999). Supporting science curriculum reform in Botswana: The potential of peer coaching. Enschede: PrintPartners Ipskamp.
Tymms, P., & Coe, R. (2003). Celebration of the success of distributed research with schools: The CEM Centre, Durham. British Educational Research Journal, 29, 639–653.
United Nations Children’s Fund. (UNICEF). (2000). Defining quality in education. New York: United Nations Children’s Fund.
Van den Akker, J. (1999). Principles and methods of development research. In J. J. Van den Akker, R. Branch, K. Gustafson, N. M. Nieveen & T. Plomp (Eds.), Design approaches and tools in education and training. Dordrecht: Kluwer Academic.
Van den Berg, S., & Burger, R. (2002, October 22–24). Education and socio-economic differentials: A study of school performance in the Western Cape. Paper presented at the 2002 DPRU/FES conference, Johannesburg, South Africa Retrieved June 9, 2010, from www.commerce.uct.ac.za/Research_Units/DPRU/Conf2003PDF/Conf2002pdf/VdBergBurger.pdf.
Van Petegem, P., & Vanhoof, J. (2005). Feedback of performance indicators: A tool for school improvement? Flemish case studies as a starting point for constructing a model for school feedback. Revista Eletronca Iberoamericana sobre Calidad, Eficaciay Cambio en Educación, 3(1), 222–234, Retrieved January 30, 2007, from http://www.ice.deusto.es/rinace/reice/vol3n1_e/VanPetegemVanhoof.pdf.
World Bank. (2008). World development indicators. Washington: Word Bank.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Archer, E., Scherman, V., Howie, S. (2013). Approaches to Effective Data Use: Does One Size Fit All?. In: Schildkamp, K., Lai, M., Earl, L. (eds) Data-based Decision Making in Education. Studies in Educational Leadership, vol 17. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4816-3_6
Download citation
DOI: https://doi.org/10.1007/978-94-007-4816-3_6
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-4815-6
Online ISBN: 978-94-007-4816-3
eBook Packages: Humanities, Social Sciences and LawEducation (R0)