Skip to main content
SpringerLink
Log in

Validation Issues in the Public Sector

  • Chapter
  • First Online:
Practitioner's Guide to Legal Issues in Organizations

  • 638 Accesses

Abstract

This chapter addresses test development and validation strategies for public safety jobs. Authoritative guidance provided by the Uniform Guidelines, Standards, and Principles is discussed along with best practices in job analysis, test administration, and validation. Case studies illustrating the inherent complexities in public safety assessment are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Private employers, state and federal governments, unions, and employment agencies.

  2. 2.

    The newest version of the Standards was released in September 2014 and was not yet published at the time of the writing of this chapter.

References

  • Aguinis, H. (2004). Test score banding in human resources selection: Legal, technical and societal issues. Westport: Praeger.

    Google Scholar 

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3), 478–494.

    Article  Google Scholar 

  • Bradley v. City of Lynn, et al. 443 F.Supp. 2d 145 (D. Mass. 2006).

    Google Scholar 

  • Bray, D. W., & Grant, D. L. (1966). The assessment center in the measurement of potential for business management. Psychological Monographs: General and Applied, 80,1–27.

    Article  Google Scholar 

  • Bray, D. W., Campbell, R. J., & Grant, D. L. (1974). Formative years in business: A long-term AT & T study of managerial lives. New York: Wiley.

    Google Scholar 

  • Buster, M. A., Roth, P. L., & Bobko, P. (2005). A process for content validation of education and experience-based minimum qualifications: An approach resulting in federal court approval. Personnel Psychology, 58,771–799.

    Article  Google Scholar 

  • Campion, M. A. (1983). Personnel selection for physically demanding jobs. Personnel Psychology, 36,527–550.

    Article  Google Scholar 

  • Campion, M. A., Outtz, J. L., Zedeck, S., Schmidt, F. L., Kehoe, J. F., Murphy, K. R., & Guion, R. M. (2001). The controversy over score banding in personnel selection: Answers to 10 key questions. Personnel Psychology, 54,149–185.

    Article  Google Scholar 

  • Cascio, W. F., Jacobs, R. R., & Silva, J. (2010). Validity, utility, and adverse impact: Practical implications from 30 years of data (pp. 271–288). In J. Outtz (Ed.), Adverse impact. New York: Routledge.

    Google Scholar 

  • Cronshaw, S. F., & Fine, S. A. (1999). Functional job analysis: A foundation for human resources management. Mahwah: Erlbaum.

    Google Scholar 

  • Dean, M. A., Roth, P. L., & Bobko, P. (2008). Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis. Journal of Applied Psychology, 93,685–691.

    Article  PubMed  Google Scholar 

  • DuBois, P. H. (1970). Varieties of psychological test homogeneity. American Psychologist, 25(6), 532–536.

    Article  Google Scholar 

  • Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register, 43(166), 38290–38315.

    Google Scholar 

  • Equal Employment Opportunity Commission, Office of Personnel Management, Department of Justice, Department of Labor, & Department of the Treasury. (1979). Adoption of questions and answers to clarify and provide a common interpretation of the uniform guidelines on employee selection procedures. Federal Register, 44(43), 11996–12009.

    Google Scholar 

  • Franke W. (1963). The reform and abolition of the traditional Chinese examination system. Cambridge: Harvard University Press.

    Google Scholar 

  • Gibson, W. M., & Caplinger, J. A. (2007). Transportation of validation results. In S. M. McPhail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence (pp. 29–81). San Francisco: Jossey-Bass.

    Google Scholar 

  • Gregory, R. J. (1996). Psychological testing: History, principles, and applications (2nd ed.). Boston: Allyn and Bacon.

    Google Scholar 

  • Jacobs, R. R., & Denning, D. (2010). Public sector employment. In J. Farr & N. Tippins (Eds.), Handbook of employee selection (pp. 705–719). New York: Routledge.

    Google Scholar 

  • Jacobs, R. R., Cushenbery, L., & Grabarek, P. E. (2011a). Assessments for selection and promotion of police officers. In J. Kitaeff (Ed.), Handbook of police psychology (pp. 193–210). New York: Routledge.

    Google Scholar 

  • Jacobs, R. R., Deckert, P. J., & Silva, J. (2011b). Adverse impact is far more complicated than the Uniform Guidelines indicate. Industrial and Organizational Psychology, 4,558–561.

    Google Scholar 

  • Jacobs, R. R., Murphy, K. R., & Silva, J. (2012). Unintended consequences of EEO enforcement policies: Being big is worse than being bad. Journal of Business and Psychology, 28(4), 467–471.

    Article  Google Scholar 

  • Kehoe, J. F. (2010). Cut scores and adverse impact. In J. L. Outtz (Ed.), Adverse impact: Implications for organizational staffing and high stakes selection (pp. 289–321). New York: Routledge.

    Google Scholar 

  • Landy, F. J. (1986). Stamp collecting versus science: Validation as hypothesis testing. American Psychologist, 41,1183–1192.

    Article  Google Scholar 

  • Landy, F. J. (2007). Changing work and the future of validation. In S. M. McPhail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence (pp. 409–426). San Francisco: Jossey-Bass.

    Google Scholar 

  • McDaniel, M. A., Kepes, S., & Banks, G. C. (2011). The Uniform Guidelines are a detriment to the field of personnel selection. Industrial and Organizational Psychology, 4,494–514.

    Article  Google Scholar 

  • McPhail, S. M. (Ed.). (2007). Alternative validation strategies: Developing new and leveraging existing validity evidence. San Francisco: Jossey-Bass.

    Google Scholar 

  • Murphy, K. R., & Jacobs, R. R. (2012). Using effect size measures to reform determination of adverse impact in equal employment. Psychology, Public Policy, and Law, 18,477–499.

    Article  Google Scholar 

  • Pearlman, K. (1980). Job families: A review and discussion of their implications for personnel selection. Psychological Bulletin, 87,1–28.

    Article  Google Scholar 

  • Reilly, R. R., Zedeck, S., & Tenopyr, M. L. (1979). Validity and fairness of physical ability tests for predicting performance in craft jobs. Journal of Applied Psychology, 64(3), 262–274.

    Article  Google Scholar 

  • Sackett, P. R., & Shen, W., et al. (2010). Perspectives from twenty-two countries on the legal environment for selection. In J. L. Farr & N. Tippins (Eds.), Handbook of employee selection (pp. 651–676). New York: Taylor & Francis.

    Google Scholar 

  • Society for industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green: Author.

    Google Scholar 

  • Stewart v. City of St. Louis and Paul E. Davis, et al., 2007 U.S. Dist. Lexis 38473 and 38421

    Google Scholar 

  • Taylor, H. C., & Russell, J. T. (1939). The relationship of validity coefficients to the practical tests of effectiveness in personnel selection. Journal of Applied Psychology, 23,565–578.

    Article  Google Scholar 

  • Theologus, G. C., Romashko, T., & Fleishman, E. A. (1973). Development of a taxonomy of human performance: A feasibility study of ability dimensions for classifying human tasks. JSAS Catalog of Selected Documents in Psychology, 3,25–26.

    Google Scholar 

  • Thornton, G. C. III, & Rupp, D. E. (2006). Assessment centers and human resource management. Mahwah: Lawrence Erlbaum.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Pennsylvania State University, 16802–3106, State College, PA, USA

    Rick Jacobs PhD

  2. EB Jacobs Consulting, 300 S. Burrowes, State College, 16801, PA, USA

    Rick Jacobs PhD

  3. Savina Consulting, 1401 Wewatta Street, #705, 80202, Denver, CO, USA

    Barbara Nett MA

Authors
  1. Rick Jacobs PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara Nett MA
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rick Jacobs PhD .

Editor information

Editors and Affiliations

  1. Berkeley Research Group, San Diego, California, USA

    Chester Hanvey

  2. DCI Consulting Group, Washington, District of Columbia, USA

    Kayo Sady

Appendices

Recommended Readings

The following three (readings) provide guidelines and the legal foundation for employee selection. They are important to review and fully understand prior to launching a selection program:

  • Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice (1978). Uniform Guidelines on employee selection procedures. Federal Register, 43(166), 38290–38315.

  • Equal Employment Opportunity Commission, Office of Personnel Management, Department of Justice, Department of Labor, & Department of the Treasury (1979). Adoption of Questions and Answers to clarify and provide a common interpretation of the Uniform Guidelines on Employee Selection Procedures. Federal Register, 44(43), 11996–12009.

  • Society for Industrial and Organizational Psychology (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.

Other recommended readings:

  • McDaniel, M. A., Kepes, S., & Banks, G. C. (2011). The Uniform Guidelines are a detriment to the field of personnel selection. Industrial and Organizational Psychology, 4, 494–514. Also commentary that follows, 515–570.

    • This series of articles and the following commentaries present a current view of the legal underpinnings of employment testing:

  • Jacobs, R. R., Cushenbery, L., & Grabarek, P. E. (2011). Assessments for selection and promotion of police officers. In J. Kitaeff (Ed.), Handbook of Police Psychology (pp. 193–210). New York: Routledge.

    • This chapter discusses the ins and outs of one form of public sector testing, selection and promotion in law enforcement:

  • Farr, J. L. & Tippins, N. T. (Eds.) (2010), Handbook of Employee Selection (pp. 705–719). New York: Routledge.

    • This is a very fine collection of chapters which pair an academic with a practitioner in each of 44 topic areas including basics of testing, types of tests that are used for selection, public sector testing, selection of different types of workers from blue collar to managerial, and legal and ethical issues of testing.

  • McPhail, S. M. (Ed.) (2007). Alternative validation strategies: Developing new and leveraging existing validity evidence. San Francisco: Jossey-Bass.

    • Another excellent collection of chapters by knowledgeable authors on various approaches to creating evidence for testing programs.

  • Outtz J. L. (Ed.) (2010). Adverse Impact. New York: Routledge.

    • The definitive book on understanding the issues surrounding adverse impact. Again, written by authors who research and work in the area of selection testing.

Glossary

  • Adverse impact: A substantially different rate of selection in hiring, promotion, or other employment decision that works to the disadvantage of members of a race, sex, or ethnic group, as described in the Uniform Guidelines. A “substantially different” rate is typically defined in government enforcement or Title VII litigation settings using the 80 % rule, statistical significance tests, and/or practical significance tests. Adverse impact is not in itself evidence of discrimination if the selection measure can be demonstrated to be valid.

  • Meta-analysis: A statistical method of research in which the results from several independent studies of comparable phenomena are combined to estimate a parameter or the degree of relationship between variables.

  • Reliability: The degree to which scores for a group of candidates are consistent or stable over one or more potential sources of error (e.g., time, raters, items, conditions of measurement) in the application of a measurement device.

  • Selection ratio: The relationship between the number of candidates assessed on a selection measure and the number of candidates actually hired; generally expressed as a value ranging from 0 to 1 and calculated by dividing the number of positions available by the total number of candidates.

  • Utility: The usefulness of a selection measure or selection system. Utility is quantified through utility analysis, which estimates the economic return on investment to the organization of one selection strategy over another. Estimates may be expressed in various forms such as dollar values or as percentage increases in productivity.

  • Validation: The process by which the validity of the proposed interpretation of test scores is investigated. Although there are a number of strategies available for evaluating evidence of validity, the three most commonly relied upon strategies are construct-oriented, content-oriented, and criterion-related.

  • Validity: The degree to which accumulated evidence and theory support specific interpretations of scores from a selection procedure entailed by the proposed uses of that selection procedure; the accurateness of inferences made based on test or performance data; also addresses whether a measure accurately and completely represents what was intended to be measured. It is not the test itself that is validated, but rather the inferences and conclusions that are reached on the basis of test scores. That is, validity is not a property of a test, but rather, a function of what the scores on that test mean. There are various strategies for establishing the validity of a measurement device.

  • Validity generalization: The application of meta-analysis to determine whether the results from validity studies in a wide range of jobs and settings produce results that are sufficiently consistent to allow scientific generalization to a particular job.

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Jacobs, R., Nett, B. (2015). Validation Issues in the Public Sector. In: Hanvey, C., Sady, K. (eds) Practitioner's Guide to Legal Issues in Organizations. Springer, Cham. https://doi.org/10.1007/978-3-319-11143-8_8

Download citation

Publish with us

Policies and ethics

Search

Navigation