Advertisement

Competence-Based Tests: Measurement Challenges of Competence Development in Vocational Education and Training

  • Christian Michaelis
  • Susan SeeberEmail author
Reference work entry

Abstract

In the current vocational education and training (VET) research, many empirical studies focus primarily on the development of competence-based test instruments that aim to measure the cognitive dispositions of a domain-specific competence construct. While these studies mainly focus on the analysis of competence structures and levels, the valid measurement of competence development represents a research gap. Studies with this focus differ significantly in their theoretical-conceptual foundations and their methodological approaches. Moreover, different challenges arise in the methodological aspects of a valid change measurement. For this purpose, the first chapter reviews the research on the measurement of competence development in the VET field. Given the increasing significance of sustainable development-related competencies in VET, an empirical analysis is performed on commercial trainees’ development of declarative knowledge related to sustainability from a societal perspective and sustainability in business processes. The in-depth analyses are based on data sets from cross-sectional and longitudinal studies. The differences and similarities between these research designs are compared with regard to their analysis potential, analysis methods, and results.

Keywords

Competence measurement Competence development Cross-sectional research design Panel studies 

References

  1. Abele S (2014) Modellierung und Entwicklung berufsfachlicher Kompetenz in der gewerblich-technischen Ausbildung. Steiner, StuttgartGoogle Scholar
  2. Andersen EB (1985) Estimating latent correlations between repeated testings. Psychometrika 50(1):3–16.  https://doi.org/10.1007/BF02294143CrossRefGoogle Scholar
  3. Anderson LW, Krathwohl DR (2001) A taxonomy for learning teaching and assessing: a revision of Bloom’s taxonomy of educational objectives. Longman Publishing, New YorkGoogle Scholar
  4. Atik D, Nickolaus R (2016) Die Entwicklung berufsfachlicher Kompetenzen von Anlagenmechanikern m ersten Ausbildungsjahr. Z Berufs- Wirtschaftspädagogik 112(2):243–269Google Scholar
  5. Barth M, Rieckmann M (2016) State of the art in research on higher education for sustainable development. In: Barth M, Michelsen G, Rieckmann M, Thomas I (eds) Routledge handbook of higher education for sustainable development. Routledge, London, pp 100–113Google Scholar
  6. Beck K, Landenberger M, Oser F (eds) (2016) Technologiebasierte Kompetenzmessung in der beruflichen Bildung: Ergebnisse aus der BMBF-Förderinitiative ASCOT. Bertelsmann, Gütersloh.  https://doi.org/10.3278/6004436wCrossRefGoogle Scholar
  7. Behrend S, Nickolaus R, Seeber S (2017) Die Entwicklung der Basiskompetenzen im Übergangssystem. Unterrichtswissenschaft 1:51–66.  https://doi.org/10.3262/UW1701051CrossRefGoogle Scholar
  8. Blömeke S, Gustafsson JE, Shavelson RJ (2015) Beyond dichotomies: competence viewed as a continuum. Z Psychol 223(1):3–13.  https://doi.org/10.1027/2151-2604/a000194CrossRefGoogle Scholar
  9. Blossfeld H-P, Schneider T (2011) Data on educational processes: national and international comparisons. In: Blossfeld H-P, Roßbach H-G, von Maurice J (eds) Education as a lifelong process: the German National Educational Panel Study, Sonderheft 14 – Zeitschrift für Erziehungswissenschaft. VS Verlag für Sozialwissenschaften, Wiesbaden, pp 35–50.  https://doi.org/10.1007/s11618-011-0180-9CrossRefGoogle Scholar
  10. Bond TG, Fox CM (2007) Applying the Rasch model: fundamental measurement in the human sciences, 2nd edn. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  11. Brötz R, Annen S, Kaiser F, Kock A, Krieger A, Noack A, … Tiemann M (2014) Gemeinsamkeiten und Unterschiede kaufmännisch-betriebswirtschaftlicher Aus- und Fortbildungsberufe (GUK) Abschlussbericht (ergänzte Fassung von 2014). BIBB, BonnGoogle Scholar
  12. Draney K, Wilson M (2008) A LLTM approach to the examination of teachers’ ratings of classroom assessment tasks. Psychol Sci 50(3):417–432Google Scholar
  13. Dreyfus SE, Dreyfus HL (1980) A five-stage model of the mental activities involved in directed skill acquisition. University of California Berkeley, BerkleyCrossRefGoogle Scholar
  14. Embretson SE (1991) A multidimensional latent trait model for measuring learning and change. Psychometrika 56(3):495–515.  https://doi.org/10.1007/BF02294487CrossRefGoogle Scholar
  15. Fischer GH, Molenaar IW (eds) (2012) Rasch models: foundations recent developments and applications. Springer, New YorkGoogle Scholar
  16. Fleischer DPJ, Koeppen K, Kenk DPM, Klieme E, Leutner D (2013) Kompetenzmodellierung: Struktur Konzepte und Forschungszugänge des DFG-Schwerpunktprogramms. Z Erzieh 16(1):5–22.  https://doi.org/10.1007/s11618-013-0379-zCrossRefGoogle Scholar
  17. Handfield R, Straube F, Pfohl H-C, Wieland A (2013) Trends and strategies in logistics and supply chain management – embracing global logistics complexity to drive market advantage. DVV Media Group GmbH, HamburgGoogle Scholar
  18. Helmke A, Weinert FE (1997) Bedingungsfaktoren schulischer Leistungen. In: Weinert FE (ed) Psychologie des Unterrichts und der Schule. Pädagogische Psychologie, 3rd edn. Hogrefe, Göttingen, pp 71–175Google Scholar
  19. Holtsch D, Rohr-Mentele S, Wenger E, Eberle F, Shavelson RJ (2016) Challenges of a cross-national computer-based test adaptation. Empir Res Vocat Educ Train 8(18):1.  https://doi.org/10.1186/s40461-016-0043-yCrossRefGoogle Scholar
  20. Howell DC (2013) Statistical methods for psychology, 8th edn. Wadsworth, BelmontGoogle Scholar
  21. Klieme E, Hartig J, Rauch D (2008) The concept of competence in educational contexts. In: Hartig J, Klieme E, Leutner D (eds) Assessment of competencies in educational contexts. Hogrefe, Göttingen, pp 3–22Google Scholar
  22. Klotz VK (2015) Diagnostik beruflicher Kompetenzentwicklung: Eine wirtschaftsdidaktische Modellierung für die kaufmännische Domäne. Springer Gabler, Wiesbaden.  https://doi.org/10.1007/978-3-658-10681-2CrossRefGoogle Scholar
  23. Klotz VK, Winther E (2017) On improving current assessment practices – competence measurement in the domain of business and commerce. In: Leutner D, Fleischer J, Grünkorn J, Klieme E (eds) Competence assessment in education: research, models and instruments. Springer, Heidelberg, pp 221–243CrossRefGoogle Scholar
  24. Klotz VK, Winther E, Festner D (2015) Modeling the development of vocational competence: a psychometric model for economic domains. Vocat Learn 8(3):247–268.  https://doi.org/10.1007/s12186-015-9139-yCrossRefGoogle Scholar
  25. Lempert W (2009) Berufliche Sozialisation: Persönlichkeitsentwicklung in der betrieblichen Ausbildung und Arbeit, 2nd edn. Schneider-Verlag Hohengehren, BaltmannsweilerGoogle Scholar
  26. Meiser T (2007) Rasch models for longitudinal data. In: von Davier M, Carstensen CH (eds) Multivariate and mixture distribution Rasch models extensions and applications. Springer, New York, pp 191–199CrossRefGoogle Scholar
  27. Michaelis C (2017) Kompetenzentwicklung zum nachhaltigen Wirtschaften: Eine Längsschnittstudie in der kaufmännischen Ausbildung. Peter-Lang, Frankfurt am Main.  https://doi.org/10.3726/b10896CrossRefGoogle Scholar
  28. Mislevy RJ, Haertel GD (2006) Implications of evidence-centered design for educational testing SRI International and University of Maryland Ravenswood. https://padi.sri.com/downloads/TR17_EMIP.pdf. Accessed 14 Dec 2017
  29. Mislevy RJ, Riconscente MM (eds) (2005) Evidence-centered assessment design: layers structures and terminology, PADI technical report 9. SRI International, Menlo ParkGoogle Scholar
  30. Nickolaus R, Gschwendtner T, Geißel B (2008) Entwicklung und Modellierung beruflicher Fachkompetenz in der gewerblich-technischen Grundbildung. Z Berufs- Wirtschaftspädagogik 104(1):48–73Google Scholar
  31. Nickolaus R, Geißel B, Abele S, Nitzschke A (2011) Fachkompetenzmodellierung und Fachkompetenzentwicklung bei Elektronikern für Energie- und Gebäudetechnik im Verlauf der Ausbildung – Ausgewählte Ergebnisse einer Längsschnittstudie. In: Nickolaus R, Pätzold G (eds) Lehr-Lernforschung in der gewerblich-technischen Berufsbildung, Zeitschrift für Berufs- und Wirtschaftspädagogik – Beiheft 2. Franz Steiner Verlag, Stuttgart, pp 78–94Google Scholar
  32. Pohl S, Carstensen CH (2012) NEPS technical report. Scaling the data of the competence test. NEPS working paper no 14. Nationales Bildungspanel, Universität Bamberg. https://www.neps-data.de/Portals/0/Working%20Papers/WP_XIV.pdf. Accessed 14 Dec 2017
  33. Rasch G (1960) Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Nielsen & Lydiche, OxfordGoogle Scholar
  34. Rausch A (2011) Erleben und Lernen am Arbeitsplatz in der betrieblichen Ausbildung. VS Verlag für Sozialwissenschaften, WiesbadenCrossRefGoogle Scholar
  35. Rausch A, Seifried J, Wuttke E, Kögler K, Brandt S (2016) Reliability and validity of a computer-based assessment of cognitive and non-cognitive facets of problem-solving competence in the business domain. Empir Res Vocat Educ Train 8:9.  https://doi.org/10.1186/s40461-016-0035-yCrossRefGoogle Scholar
  36. Rosendahl J, Straka GA (2011a) Kompetenzmodellierungen zur wirtschaftlichen Fachkompetenz angehender Bankkaufleute. Z Berufs- Wirtschaftspädagogik 107(2):190–217Google Scholar
  37. Rosendahl J, Straka GA (2011b) Effekte personaler schulischer und betrieblicher Bedingungen auf berufliche Kompetenzen von Bankkaufleuten während der dualen Ausbildung Ergebnisse einer dreijährigen Längsschnittstudie. https://elib.suub.uni-bremen.de/edocs/00102039-1.pdf. Accessed 14 Dec 2017
  38. Rost J (2004) Lehrbuch Testtheorie – Testkonstruktion, 2nd edn. Hans Huber, BernGoogle Scholar
  39. Seeber S (2017) Economic competencies and situation specific commercial competencies: reflections on conceptualization and measurement. J Citizenship Soc Econ Educ 15(3):162–182.  https://doi.org/10.1177/2047173417695275CrossRefGoogle Scholar
  40. Seeber S, Michaelis C (2014) Development of a model of competencies required for sustainable economic performance among apprentices in business education. AERA, Washington, DC. http://www.aera.net/Publications/Online-Paper-Repository. Accessed 14 Dec 2017Google Scholar
  41. Seeber S, Hartig J, Dierkes S, Schumann M (2016) Ko-NaMa – simulation-based measurement and validation of a competence model for sustainability management. In: Pant HA, Zlatkin-Troitschanskaia O, Lautenbach C, Toepper, M, Molerov, D (eds) Modeling and measuring competencies in higher education – validation and methodological innovations (KoKoHs) – overview of the research projects (KoKoHs working papers, 11). Humboldt University, Berlin; Johannes Gutenberg University, Mainz, pp 53–56. https://www.kompetenzen-im-hochschulsektor.de/Dateien/KoKoHs_Working_Papers_No.11_Final_04.07.pdf. Accessed 14 Dec 2017
  42. Seeber S, Baethge M, Baas M, Richter M, Busse R, Michaelis C (2017) Ländermonitor berufliche Bildung 2017: Leistungsfähigkeit und Chancengerechtigkeit – ein Vergleich zwischen den Bundesländern. wbv, Bielefeld.  https://doi.org/10.3278/6004634wCrossRefGoogle Scholar
  43. Sembill D (2008) Führung und Zeit – institutionelle und unterrichtliche Perspektiven. In: Warwas J, Sembill D (eds) Zeitgemäße Führung – zeitgemäßer Unterricht. Schneider, Hohengehren, pp 81–98Google Scholar
  44. Shavelson RJ, Ruiz-Primo MA, Wiley EW (2005) Windows into the mind. High Educ 49(4):413–430.  https://doi.org/10.1007/s10734-004-9448-9CrossRefGoogle Scholar
  45. von Davier AA, Carstensen CH, von Davier M (2008) Linking competencies in horizontal vertical and longitudinal settings and measuring growth. In: Hartig J, Klieme E, Leutner D (eds) Assessment of competencies in educational contexts. Hogrefe, Göttingen, pp 53–80Google Scholar
  46. von Davier M, Xu X, Carstensen CH (2011) Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika 76(2):318–336.  https://doi.org/10.1007/s11336-011-9202-zCrossRefGoogle Scholar
  47. Warm TA (1989) Weighted likelihood estimation of ability in item response theory. Psychometrika 54:427–450.  https://doi.org/10.1007/BF02294627CrossRefGoogle Scholar
  48. Weinert FE (1999) Concepts of competence. Definition and selection of competencies: theoretical and conceptual foundations (DeSeCo). OECD, Paris. https://pdfs.semanticscholar.org/8b88/efa9dd5e0a4b605aea6e5e3b9ec640beb089.pdf. Accessed 14 Dec 2017Google Scholar
  49. Weißeno S, Seeber S, Kosanke J, Stange C (2016) Development of mathematical competency in different German pre-vocational training programmes of the transition system. Empir Res Vocat Educ Train 8(1):14.  https://doi.org/10.1177/2047173417695275CrossRefGoogle Scholar
  50. Winther E (2010) Kompetenzmessung in der beruflichen Bildung. Bertelsmann, Bielefeld.  https://doi.org/10.3278/6004148wCrossRefGoogle Scholar
  51. Winther E, Festner D, Sangmeister J, Klotz VK (2016) Facing commercial competence: modeling domain-linked and domain-specific competence as key elements of vocational development. In: Wuttke E, Schumann S, Seifried J (eds) Economic competence and financial literacy of young adults – status and challenges. Barbara Budrich, Opladen, pp 149–164Google Scholar
  52. Wu ML, Adams RJ, Wilson MR, Haldane SA (2007) ACERConQuest Version 2: Generalised item response modeling software. Australian Council for Educational Research, CamberwellGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Economic SciencesGeorg-August-University of GöttingenGöttingenGermany

Section editors and affiliations

  • Esther Winther
    • 1
  1. 1.Vocational Education and TrainingUniversity of Duisburg-EssenDuisburgGermany

Personalised recommendations