Descriptive Validity and Transparent Reporting in Randomised Controlled Trials

  • Amanda E. Perry


While the concept of validity has been around for a number of decades, descriptive validity has been considered to a lesser extent in criminal justice research. In this chapter, we illustrate and evaluate several examples of descriptive validity in randomized controlled trials conducted in criminal justice. In particular, we introduce the idea of the CONSORT Statement, which was originally developed in healthcare to assess the descriptive validity of trials. This Statement was developed originally over concerns about the appropriateness of synthesising trials together in meta-analyses with differing levels of descriptive validity. We then go on to demonstrate the use of descriptive validity, using two recent studies. We examine examples of descriptive validity which report poorly in criminal justice trials and provide examples of good practice.Throughout this chapter, we emphasize the importance of descriptive validity to criminal justice researchers in upholding the quality of research within the field. A comparison and discussion about the applicability of the CONSORT Statement with criminal justice and healthcare trials suggests that some amendments may need to be made to make relevance to trials conducted in criminal justice. Generally adoption of a CONSORT-like Statement in criminal justice is thought to improve the reporting of descriptive validity in future research.


Criminal Justice Allocation Concealment Journal Editor Juvenile Offender Outcome Assessor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Altman DG (2001) The revised CONSORT Statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694Google Scholar
  2. Altman DG (2005) Endorsement of the CONSORT statement by high impact medical journal: a survey of instructions for authors. Br Med J 330:1056–1057CrossRefGoogle Scholar
  3. Borduin CM, Mann BJ, Cone LT, Henggeler SW, Ficci BR, Blaske DM et al. (1995) Multisystemic treatment of serious juvenile offenders: long term prevention of criminality and violence. J Consult Clin Psychol 63:569–578CrossRefGoogle Scholar
  4. Boutron I, Moher D, Tugwell P, Giraudeau B, Poiraudeau S, Nizard R, Ravaud P (2005) A checklist to evaluate a report of a nonpharmocological trial (CLEAR NPT) was developed using consensus. J Clin Epidemiol 58(10):1233–12240CrossRefGoogle Scholar
  5. Boutron I, Estellat C, Guittet L, Dechartres A, Sackett DL, Hróbjartsson A et al. (2006) Methods of blinding in reports of randomised controlled trials assessing pharmacologic treatments: a systematic review. PLoS Med 3(10):1931–1939CrossRefGoogle Scholar
  6. Boruch RF (1997) Randomized field experiments for planning and evaluation: a practical guide. Sage Publications, Newbury Park, CAGoogle Scholar
  7. Campbell DT (1957) Factors relevant to the validity of experiments in social settings. Psychol Bull 54:297–312CrossRefGoogle Scholar
  8. Campbell DT & Stanley JC (1963) Experimental and quasi-experimental designs for research on teaching. In: Gage NL (ed) Handbook of research on teaching. Chicago, Rand McNally, 171–246Google Scholar
  9. Campbell FA, Ramey CT, Pungello E, Sparling J, Miller-Johnson S (2002) Early childhood education: young adult outcomes from the Abercedarian project. Appl Dev Sci 6:42–57CrossRefGoogle Scholar
  10. Devereaux PJ, Manns BJ, Ghali WA, Quan H, Guyatt GH (2002) The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist. Control Clin Trials 23:380–388CrossRefGoogle Scholar
  11. Egger M, Smith GD (1998) Bias in location and selection of studies. Br Med J 316:61–66Google Scholar
  12. Emerson JD, Burdick E, Hoaglin DC, Mosteller F, Chalmers TC (1990) An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials 11:339–352CrossRefGoogle Scholar
  13. Farrington DP (2003) Methodological quality standards for evaluation research. Ann Am Acad Pol Soc Sci 587:49–68CrossRefGoogle Scholar
  14. Farrington DP, Welsh BC (2005) Randomized experiments in criminology: what have we learned in the last two decades? J Exp Criminol 1(1):9–38CrossRefGoogle Scholar
  15. Fiorucci S, Mencarelli A, Lechi A, Renga B et al. (2004) Co administration of nitric oxide aspirin prevents platelet and monocyte activation and protects against gastric damage induced by aspirin in humans. J Am Coll Cardiol 44:635–641CrossRefGoogle Scholar
  16. Francis CW, Berkowitz SD, Comp PC, Lieberman JR Ginsberg JS et al. (2003) Comparison of ximelagatran and warfarin for the prevention of venous thromboembolism after total knee replacement. N Engl J Med 349: 1703–1712Google Scholar
  17. Freemantle N (2001) Interpreting the results of secondary end points and subgroup analyses in clinical trials: should we lock the crazy aunt in the attic? Br Med J 322(7292):989–991CrossRefGoogle Scholar
  18. Graf J, Doig GS, Cook DJ, Vincent JL, Sibbald WJ (2002) Randomized controlled trials in sepsis: has methodological quality improved over time? Crit Care Med 30(2):461–472CrossRefGoogle Scholar
  19. Grossman JB, Tierney JP (1998) Does mentoring work? An impact study of the Big Brothers Big Sisters Program. Eval Rev 22:403–426CrossRefGoogle Scholar
  20. Hewitt C, Hahn S, Torgerson DJ, Watson J, Bland MJ (2005) Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals. Br Med J 330:1057–1058CrossRefGoogle Scholar
  21. Hopewell S, Altman DG, Moher D, Schulz KF (2008) Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journals ‘Instructions to Authors’. Trials 9:20CrossRefGoogle Scholar
  22. Ioannidis JP, Cappelleri JC, Lau J (1998) Issues in comparisons between meta-analyses and large trials. JAMA 279:1089–1093CrossRefGoogle Scholar
  23. Jüni P, Altman DG, Egger M (2001) Assessing the quality of controlled clinical trials. In: Egger M, Davey-Smith G, Altman DG (eds) Systematic reviews in health care: meta-analysis in context, 2nd ed. BMJ Books, LondonGoogle Scholar
  24. Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M (2002) Direction and impact of languate bias in meta analyses of controlled trials: empirical study. International Journal of Epidemiology 31:115–123CrossRefGoogle Scholar
  25. Jüni P, Tallon D, Egger M (2000) ‘Garbage in – garbage out’? Assessment of the quality of controlled trials in meta-analyses published in leading journals. In: Proceedings of the 3rd symposium on systematic reviews: beyond the basics. St. Catherines College, Centre for Statistics in Medicine, OxfordGoogle Scholar
  26. Kjaergard LL, Villumsen J, Gluud C (1999) Quality of randomised clinical trials affects estimates of intervention efficacy. In: Proceedings of the 7th Cochrane Colloquium. Universitia S. Tommaso D’Aquino Rome. Milan, Centrro Cochrane Italiano, poster B10Google Scholar
  27. Kjaergard LL, Nikolova J, Gluud C (1999) Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135:982–989Google Scholar
  28. Kjaergard LL, Villumsen J, Gluud C (2001) Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Intern Medicine 135:982–989Google Scholar
  29. Lao L, Bergman S, Hamilton GR, Langenberg P, Berman B (1999) Evaluation of acupuncture for pain control after oral surgery. Arch Otolaryngol Head Neck Surg 125:567–572Google Scholar
  30. Latessa E, Moon (1992) The effectiveness of acupuncture in an outpatient drug treatment program, Journal of Contemporary Criminal Justice 8(4):317–331Google Scholar
  31. Lösel F, Köferl P (1989) Evaluation research on correctional treatment in West Germany: a meta-analysis. In: Wegener H, Lösel F, Haisch J (Eds) Criminal behavior and the justice system: psychological perspectives. Springer-Verlag, New York, pp 334–355Google Scholar
  32. Moher D, Jadad AR, Nichol G Penman M, Tugwell M, Wash S (1995) Assessing the quality of RCT: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73Google Scholar
  33. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613CrossRefGoogle Scholar
  34. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, et al. (1999) Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess 3(i–iv):1–98Google Scholar
  35. Moher D, Schulz KF, Altman D, for the CONSORT Group (2001a) The CONSORT Statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 285:1987–1991CrossRefGoogle Scholar
  36. Moher D, Jones A, Lepage L (2001b) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before and after evaluation. JAMA 285:1992–1995CrossRefGoogle Scholar
  37. Moher D, Altman D, Schulz K, for the CONSORT Group. The CONSORT statement: revised recommendation for improving the quality of reports of parallel-group randomised trials [Chinese]. Chin J Evid-Based Med 2005; 5(9):702–707Google Scholar
  38. Needles K, James-Burdumy S, Burghardt J (2005) Community case management for former jail inmates: its impacts on re-arrest, drug use and HIV risk. J Urban Health 82(3):420–431Google Scholar
  39. Perry AE, Johnson M (2008) Applying the Consolidated Standards of Reporting Trials (CONSORT) to studies of mental health provision for juvenile offenders: a research note. J Exp Criminol 4:165–185CrossRefGoogle Scholar
  40. Perry AE, Gilbody S, Akers J, Light K (2008) Access and provision of services for young people with mental health needs. Youth Justice Board, England and WalesGoogle Scholar
  41. Perry AE, Weisburd D, Hewitt C (in press) Are criminologists reporting experiments in ways that allow us to assess? J Exp CriminolGoogle Scholar
  42. Peters M, Thomas D, Zanberlon C (1997) Boot camps for juvenile offenders. Office of Juvenile Justice and Delinquency Prevention (Program Summary, Washington, DCGoogle Scholar
  43. Petrosino A, Turpin-Petrosino C, Buehler J (2002) Scared straight and other juvenile awareness programs for preventing juvenile delinquency: a systematic review of randomized experimental evidence. Campbell Collaboration, USAGoogle Scholar
  44. Plint AC, Moher D, Morrison A, Schulz K, Altman D, Hill C, Gaboury I (2006) Does the CONSORT checklist improve the quality of reporting randomized controlled trials? A systematic review. Med J Aust 185(5):263–267Google Scholar
  45. Prady SL, Richmond SJ, Morton VM, MacPherson H (2008) A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLos One 3(2):1–10CrossRefGoogle Scholar
  46. Rhodes W, Gross M (1997) Case management reduces drug use and criminality among drug-involved arrestees: an experimental study of an HIV prevention intervention (PDF Version). Washington, D.C.: National Institute of Justice, U.S. Department of JusticeGoogle Scholar
  47. Robinson D (1995) The impact of cognitive skills training on post release recidvisim among Canadian federal offenders. Correctional Service of Canada, Ottowa, Research Report R-41Google Scholar
  48. Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412CrossRefGoogle Scholar
  49. Shea B, Boers M, Grimshaw JM, Hamel C, Bouter LM (2006) Does updating improve the methodological and reporting quality of systematic reviews? BMC Med Res Methodol 6:27CrossRefGoogle Scholar
  50. Sherman LW, Farrington DP, Welsh BC, Mackenzie DL (eds) (2002) Evidence-Based Crime Prevention. Routledge, LondonGoogle Scholar
  51. Torgerson CJ & Elbourne D (2002) A systematic review and meta analysis of the effectiveness of information and communication technology (ICT) on the teaching of spelling. J Res Read 35:129–143CrossRefGoogle Scholar
  52. Torgerson CJ, Torgerson DJ, Birks YF, Porthouse J (2005) A comparison of RCTs in health and education. Br Educ Res J 31(6):761–785CrossRefGoogle Scholar
  53. Torgerson CJ, Torgerson DJ (2008) Designing and running randomised trials in health and the social sciences, Palgrave, MacMillanCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Amanda E. Perry
    • 1
  1. 1.Centre for Criminal Justice Economics and PsychologUniversity of YorkHeslingtonUK

Personalised recommendations