# Are ordinal rating scales better than percent ratings? a statistical and “psychological” view

## Abstract

Disease incidence and severity are often assessed by either an ordinal rating scale, e.g., with scores from 1 to 9, or a percentage rating scale. This paper compares three different rating scales regarding accuracy, precision, and time needed for scoring. Pictograms of mildew diseased cereal leaves were generated following a right skewed beta-distribution. Persons with different rating experience were asked to rate the leaves on three different scales: two different percentage scales [1%-steps (P1) and 5%-steps (P5)] and an ordinal 9-point rating scale (R9) where thresholds followed a logarithmic pattern with respect to the underlying percentage scale. A transformed value of the estimated disease severity as well as the transformed time needed to estimate per leaf was documented and evaluated using mixed models. In most cases both percent ratings performed better than the ordinal rating scale. For the time needed per leaf by the untrained group, method R9 was better. With the trained group P5 performed better than both other methods. The raters mostly preferred R9, especially when untrained. Nevertheless, the results suggest that P5 can be recommended in terms of accuracy.

## Keywords

Accuracy Disease severity Mixed model Percentages Precision Visual assessment## Notes

### Acknowledgments

We are indebted to Mr. Hilbert (Rechenzentrum Universität Hohenheim) for the support in developing the MS Access program. We also pay thanks to all raters for their help and time. The comments from Prof. Dr. Zebitz and Dr. K. Emrich are greatfully acknowledged. This research is supported by the German Research Foundation (DFG), grant number PI 377/5.

## References

- Bundessortenamt (2000) Richtlinien für die Durchführung von landwirtschaftlichen Wertprüfungen und Sortenversuchen. Landbuch Verlag, Hannover, DeutschlandGoogle Scholar
- Burnham KP, Anderson DR (1998) Model selection and inference: a practical information-theoretic approach. Springer-Verlag, New York, USA Google Scholar
- Campbell CL, Madden LV (1990) Introduction to plant disease epidemiology. Wiley, New YorkGoogle Scholar
- Duveiller E (1994) A pictorial series of disease assessment keys for bacterial leaf streak of cereals. Plant Dis 78:137–141CrossRefGoogle Scholar
- Forbes GA, Jeger MJ (1987) Factors affecting the estimation of disease intensity in simulated plant structures. J Plant Dis Prot 94:113–120Google Scholar
- Forbes GA, Korva JT (1994) The effect of using a Horsfall-Barratt scale on precision and accuracy of visual estimation of potato late blight severity in the field. Plant Pathol 43:675–682Google Scholar
- Garett KA, Madden LV, Hughes G, Pfender WF (2002) New applications of statistical tools in plant pathology. Phytopathology 94:999–1003Google Scholar
- Hau B, Kranz J (1989) Fehler beim Schätzen von Befallsstärken bei Pflanzenkrankheiten. J Plant Dis Prot 96:649–674Google Scholar
- Hock J, Kranz J, Renfro BL (1992) Test of standard diagrams for field use in assessing the tarspot disease complex of maize (
*Zea mays*). Trop Pest Manage 38:314–318Google Scholar - Horsfall JG, Barratt RW (1945) An improved grading system for measuring plant diseases. Phytopathology 35:655Google Scholar
- James WC (1974) Assessment of plant diseases and loss. Annu Rev Phytopathol 12:27–48CrossRefGoogle Scholar
- Jenkins SF Jr, Wehner TC (1983) A system for the measurement of folia diseases of cucumber. Cucurbit Genet Coop Rep 6:10–12Google Scholar
- McCullagh P, Nelder J (1989) Generalized linear models, 2nd edn. Chapman and Hall, LondonGoogle Scholar
- Nita M, Ellis MA, Madden LV (2003) Reliability and accuracy of visual estimation of phomopsis leaf blight of strawberry. Phytopathology 93:995–1005PubMedGoogle Scholar
- Nutter FW, Schultz PM (1995) Improving the accuracy and precision of disease assessments: selection of methods and use of computer-aided training programs. Can J Plant Pathol 17:174–184CrossRefGoogle Scholar
- O’Brien RD, van Bruggen AHC (1992) Accuracy, precision, and correlation to yield loss of disease severity scales for corky root of lettuce. Phytopathology 82:91–96Google Scholar
- Piepho HP (1999) Analysing disease incidence data from designed experiments by generalized nonlinear mixed-effect models. Plant Pathol 48:668–674CrossRefGoogle Scholar
- Piepho HP, Büchse A, Emrich K (2003) A hitchhiker’s guide to the mixed model analysis of randomized experiments. J Agron Crop Sci 189:310–322CrossRefGoogle Scholar
- Rice JA (1995) Mathematical statistics and data analysis, 2nd edn. Wadsworth Publishing Co Inc., Duxbury Press, Belmond, CAGoogle Scholar
- SAS Institute Inc (1999) SAS user’s guide, version 8. Cary, NCGoogle Scholar
- Schabenberger O, Pierce J (2002) Contemporary statistical models for the plant and soil sciences. CRC Press, New YorkGoogle Scholar
- Schumacher E, Bleiholder H, Thöni H (1995). Methodische Untersuchungen zur biometrischen Analyse von Boniturwerten aus Freilandversuchen mit Herbiziden. In: Proceedings of the 9th EWRS symposium, vol 1. Budapest, pp 283–290Google Scholar
- Shah DA, Madden LV (2004) Nonparametric analysis of ordinal data in designed factorial experiments. Phytopathology 94:33–43PubMedGoogle Scholar
- Sherwood RT, Berg CC, Hoover MR, Zeiders KE (1983) Illusions in visual assessments of stangonospora leaf spot of orchardgrass. Phytopathology 73:173–177CrossRefGoogle Scholar
- Tomerlin JR, Howell TA (1988) DISTRAIN: a computer program for training people to estimate disease severity on cereal leaves. Plant Dis 72:455–459Google Scholar