Competency-Based Exams in Professional Education
- 6.1k Downloads
Exams and evaluations are components of an essential didactic method that confirm a definable performance. In professional education final exams and an official certificate of performance are important prerequisites for entry into the labour market. In order to comply with recent employers’ requirements, the exams should be competency-based. The article gives insights into the legal provisions for carrying out examinations and requirements towards examiners in the Central Asian countries of Kazakhstan, Kyrgyzstan and Tajikistan and allows a comparison with examples from Germany. Furthermore, the article provides practical cases of respective rules’ implementation and challenges for the examination systems in the named three countries.
KeywordsProfessional education Examination Competency-based examination Examiner Examination committee
15.1 Competency-Based Exams
When providing education services, educational institutions’ primary aim should be to enhance the quality of their services and also the level of knowledge students acquire during this process. Average performance levels indicate the quality of education not only in a particular institution but also in the whole educational system. In the context of general education, this principle serves large-scale international comparisons like the PISA and the TIMMS study. Therefore it is worth exploring this aspect further in the context of professional education.
Indeed one of the forms to ensure education quality is by examination. In educational institutions in Germany and in the Central Asian countries of Kazakhstan, Kyrgyzstan and Tajikistan, exams are used as tools to evaluate interim performance, as well as a final assessment. While the interim performance evaluation serves to provide feedback and recommendations for students’ further development as well as for teachers’ professional activities, the main aim of the final exam is to some extent to assess skills and competences and to provide an overview of a graduate’s abilities that can be used on the labour market. In other words, the exam results provide a short, unified and clear description of a potential employee’ skills and comparability. To ensure better quality and labour market orientation, exams should be competency-based.
In literature, ‘competences’ are discussed differently, and there are big differences between European and Anglo-American understandings. The following definition meets the understanding of the authors and will be used throughout this article. According to Weinert (2001), ‘competences are inherent or developed by a person cognitive abilities and skills to solve particular problems and connected with this motivational, volitional and social readiness and ability to responsibly and successfully use these solutions in different situations’1 (id. 2001, p. 271). In this way, the main objective of competency-based examinations is not only to define the level of knowledge students have but also their ability to apply this knowledge in solving different types of problems, mainly in their professional life.
The three abovementioned Central Asian countries recognize the importance of linkages between the educational system and the labour market (Stehling 2015). Thus, the National Sustainable Development Strategy for the Kyrgyz Republic for 2013–2017 underlines that VET institutions should adequately collaborate with the private sector. The Strategy ‘Kazakhstan-2050’ states that curricula should have more practical content and should focus on labour market needs. The Tajik National Education Development Strategy also admits that the VET education does not meet labour market requirements and companies should take part in educational processes. So, all three countries recognize that integrating elements of dual education into the national VET systems is crucial for increasing the quality of vocational education. It is also stated in these documents that employers should be involved in assessing graduates’ skills, as national legal frameworks contain further details on this issue.
15.2 Research Methods and Data Collection
In this research, interviews have been carried out. These interviews are based on literature and study of examination protocols. First, in the framework of a GIZ project in Tajikistan, a series of examinations for motorcar mechanics under the guidance of the German experts has been carried out. Observation criteria such as planning and organizing examination process and follow-up were agreed upon in advance and served as a basis for observations during preparing and conducting examinations. Then, the observations were documented in the form of protocols. Based on the latter, face-to-face interviews with teachers from Kazakhstan and Kyrgyzstan have been carried out. Another method used in this research is to study and to compare legal acts in the sphere of vocational education and training in the given countries. Results are presented in the form of recommendations provided in this article.
15.3 Legal Frameworks for Carrying Out Examinations in Professional Education in Central Asia
Since officially certified performance is a precondition for labour market entry, the performance should be assessed accordingly and final exams are strictly regulated. Taking into consideration that these requirements are of a compulsory nature, only relevant state bodies can be responsible for providing the main rules and regulations. In Kazakhstan, Kyrgyzstan and Tajikistan, these are ministries of education and, partially, ministries of labour (in the field of adult education). Germany follows the principle of the dual system in professional education. Based on this principle, professional associations such as the Chamber of Crafts (Handwerkskammer) or the Chamber of Industry and Commerce (Industrie- und Handelskammer) are responsible for the content of educational plans and examinations. Thus, as members of these associations, private companies are involved in the professional education and examination processes. In turn, every association has its own examination regulations.
For example, if one compares the examination regulations of the Chamber of Crafts Kassel, a city in the Federal German State of Hessen, with the national assessment regulations in VET in Kazakhstan, Kyrgyzstan and Tajikistan, it stands out that the regulations of Central Asian countries are more general. Rules on the appointment of the examination committee members and preparation of the examinations are quite vague. Not all aspects of conducting exams (for instance, participation in exams of persons with special needs) are covered, and assessment criteria are not clearly provided. In this way, regulations leave much room for interpretation, which transfers added responsibility onto examination committees and makes the results of assessments more difficult to compare.
Nevertheless, forms of final assessment and requirements of different examination committees can be more or less compared.
Written exams: This is usually the most common examination type in Central Asian countries, as it ideally ensures unbiased assessment and is easy for documentation. If students have complaints about their assessment, the exam results can also be easily reviewed. Apart from closed questions and multiple choice questionnaires (tests – this type is considered in this article as a separate type of examination), questions can be used, which require thinking out of the box, analytical skills and explanations of conclusions. However, there is hardly any complex task-setting. Most written exams test students’ knowledge in a particular subject but not necessarily their abilities to find solutions to complex problems.
Tests: As a form of the written examination is mostly used for continuous monitoring of students’ performance, this method has gained increasing popularity among teachers. First of all, it allows assessing large groups of students in a finite time period as well as opportunities for comparison. Secondly, it limits the ability of external influences to compromise the quality of the assessment. Tests also help evaluate students’ knowledge across topics of the training course. On the other hand, the nature of this method does not allow for a comprehensive measurement of acquired knowledge or an evaluation of the ability to use this knowledge and bring it to implementation (Furs 2005).
Oral exams: These are used for interim performance evaluations and presentations of the diploma project. The standard form involves students answering questions posed directly by examiners who then ask clarifying or control questions. In the three Central Asian countries, this form is used most often. Other forms of oral exams such as case-related debates, discussions of case studies and role games are barely used at all. One reason could be that teachers lack skills in assessing students’ performance in such an ‘unstructured’ way. Or, overwhelmed by reports and other paper-work, they simply do not have the time to invest in more sophisticated forms of examination. Support from managerial personnel should play an important role here, but it is not always forthcoming.
Practical exams: Most of these exam types are used in technical vocational education. This type of examination requires more time for preparation which entails both the design of tasks and the testing of equipment that should be ready to use on the day of the exam. Observations from Tajikistan, for instance, show that this rule is often neglected and equipment is in nonworking condition (and is sometimes even being repaired) as the examinations take place.
Forms of final assessment and compositions of examination committees at VET institutions in Kyrgyzstan, Kazakhstan, Tajikistan and Germany
Model regulations on interim evaluation and final assessment in VET institutionsa
Regulations on final assessment of graduates of VET institutionsb
Regulations on final assessment of graduates of VET institutions
Law for vocational trainingc
Form of assessment
Diploma project, incl. oral presentation and/or written examinations
Examinations based on theoretical and practical testing procedures. Oral examination could be included
Teachers, production masters and experts from companies
Teachers from the given and other VET institution, experts from companies and other institutions
Teachers from the given and other VET institution, experts from companies and other educational institutions
Minimum one teacher, an employer representative, an employee representative (based on the dual education system structure)
Expert in a particular field
Expert in a particular field, approved by Ministry of Education
Representative of the examination committee
As different types of exams assess various types of competences, a combination of examination forms is often used. For instance, skills evaluated during oral examinations (presentation skills, ability to explain the way of thinking) cannot be fully assessed by purely practical exercises and vice versa. When using the practice-oriented approach, not only knowledge and skills but also competences are evaluated with a focus on the labour market. It is the role of the private sector to evaluate whether skills acquired during the training are applicable in the labour market. Especially crucial is this when national qualification frameworks are missing (among the three countries, only Kazakhstan has developed this framework).
Based on an observation of examination processes in the three countries, it follows that students lack experience in handling comprehensive tasks and have insufficient problem-solving skills. One reason could be that tasks are usually given based on particular parts of a training course and comprehensive tasks that require extensive problem-solving skills are the exception rather than the rule. This is closely connected with the teaching methods. The value of understanding complex problems and interconnections between problems should be made integral to the whole training process.
15.4 Requirements of Examiners and Examination Committee
15.4.1 Examiners and Members of the Examination Committee
During interim performance evaluations, students’ knowledge and skills on specific topics are assessed by their teachers. However, for final exams or presentations of a diploma project, an examination committee consisting of a minimum of three members, and usually up to five or seven, is formed. The qualitative composition of the board is also considered. This is ensured by the principle of parity, which means that apart from the teaching and managerial personnel of the educational institution, representatives of potential employers should be included into the committee. As stated above, educational institutions are vital measurements for the private sector, since when employing graduates, companies rely on their grades and assessment given to them by educational bodies. The grades earned in final exams and the average final grade in most cases play a significant role in ensuring a young person’s entry into the labour market. Thus, the examination committee should embody not only specific knowledge but also practical experience in the topics to which the questions relate.
For this reason, a minimum of one employer representative should be present. This is provided in the legal provisions in all the Central Asian countries mentioned. However, the rule is not always followed: many representatives of companies (especially in the private sector) do not recognize the importance of being involved in examination processes and view participation in examinations as a distraction from their everyday work.
It is possible that it is precisely the fact that representatives of the private sector are only involved into the process as late as in a final exam which reduces their interest and trust in the results of assessments. For this reason, their involvement into the entire training process of a student or trainee is a key requirement to improve the VET system embedded in the national strategies.
15.4.2 Methodological and Didactical Skills of Examiners
Apart from specific knowledge, the examiners should possess methodological and didactical skills (a competence matrix for examiners can be developed) related to the topics of the exams. Examiners should follow the same rules and ensure a standard pattern of behaviour. Observations of several exams in VET institutions in Tajikistan, however, indicate that the quality of the examiners, or rather the quality of their procedures and methods, is not equal or consistent. This is especially true regarding their approach of asking questions: some ask interposed questions, others guiding questions and others still simply observe and take notes. In this regard measurements to develop consistency should be taken, for instance, conducting exam-related trainings for teachers.
Examiners should know how to ask appropriate guiding questions, provide pedagogical or even psychological support to students and remain unbiased. Neutral and bias-free treatment of students during examinations is one of the main requirements for competent examiners. Sometimes it is not easy to meet this requirement, however, since some factors can lead to a distortion of an examiner’s perception and, consequently, result in subjective judgments over students’ competences. German scientist Sebastian Walzik names the following factors: first impression, prejudices, stereotypes, contrast effect and context effect (Walzik 2015). Examiners cannot adopt an appropriate unbiased position if they have a particular relationship with students (relatives, colleagues, etc.). Hence, this fact should be excluded during the formation of the examination committee, although it is not clearly mentioned in the examination regulations of either Kazakhstan, Kyrgyzstan or Tajikistan.
Which of these two tasks helps assess students’ competences?
Although the right answer in both cases is the same, the first task (1) allows only assessing the ability of a student to fulfil simple arithmetic operations. The second one (2) helps students show not only knowledge of arithmetic but also their ability to apply math rules.
Moreover, examiners should know how to handle non-standard or even emergency situations. What to do if a participant is late due to objective or subjective reasons? As a common rule in Central Asian countries, latecomers are not allowed to enter the examination room and should take the exam later (upon decision of the examination committee). However, this leads to increasing examination costs borne by educational institutions. Therefore it would be reasonable to introduce a solution such as late arrivals – up to 15 min can be accepted (at least for written exams) – without adding extra time after the end of the exam. This goodwill regulation could meet requirements on examination effectiveness and simultaneously reduce stresses connected with repeat exams.
Another non-standard situation is a power blackout or breakdown of equipment used during the examination. On the one hand, it is of course impossible to foresee all non-standard situations and provide description of steps to follow. Therefore training should be made available for examiners on ways to handle unforeseen circumstances depending on the educational institution’s capacities and the preparations made by examiners for such situations.
15.5 Training of the Examiners
As mentioned above, the methodological and didactical preparation of examiners is one of the most vulnerable components of the educational systems in Kazakhstan, Kyrgyzstan and Tajikistan, which requires additional testing. Some teachers, who also act as examiners in their institutions, report a lack of instructions providing clear guidance for examination processes and competences assessment. There are also no guidelines on the way students should be treated (how to provide necessary support, while remaining bias-free, how not to be distracted by external factors such as students’ looks or noises outside of the class room, etc.).
Taking all these arguments into consideration, it is advisable to carry out not only singular trainings but also regular further trainings for examiners. Systematic feedback should be provided to aid quality control, and possible improvements should be discussed. Then, the lessons learnt should be applied in preparation for and during examinations (Weiß 2011, p. 45). Crucial support is also provided to examiners in the form of general information, practical advice and platforms for the exchange of experience. Measures to explore these opportunities in the Central Asian countries should be taken.
Experience has shown that the legal framework for competency-based evaluation is in place in all the aforementioned Central Asian countries. However, the frameworks, regulating examinations in Central Asia, are rather general compared to the ones in Germany (Bundesministerium für Wirtschaft und Energie 2017). For this reason a member of the examination committee has a much higher responsibility regarding the implementation of comparable standards into examinations as well as comparable assessment criteria. Neither Kyrgyzstan nor Tajikistan has implemented a National Qualification Framework yet, leaving Kazakhstan as the only country that has met this requirement. Even more important is the participation of the private sector in defining the content of trainings and apprenticeships as well as being part of examinations to evaluate competences acquired.
To fulfil this weighty responsibility, members of the examination committees require all possible support in the form of ongoing trainings and facilitating the exchange of experiences between committees. Taking into account that the national strategies in these countries focus on the implementation of dual elements in professional education in order to meet the needs of a rapidly changing labour market as well as on tackling challenges inherent in economic globalization, examiners have to be both highly skilled and experienced. There are encouraging examples of responsible cooperation with the private sector. Clear descriptions of job profiles on the part of the private sector will help train young people according to key requirements and consequently improve their perspectives in the labour market. Experience exchange of best practices in VET networks can be a focal point for continuous improvement in the area of evaluation of competences, as well as the ongoing training of examiners.
Translation by authors’.
- Berufsbildungsgesetz, Bundesministerium der Justiz und für Verbraucherschutz 31.8.2015.Google Scholar
- Bretschneider, M., Gutschow, K., & Lorig, B. (2014). Kompetenzorientiert prüfen – Prüfungspraxis und weiterer Entwicklungsbedarf. Berufsbildung in Wissenschaft und Praxis, 43(3), 26–29. Retrieved from https://www.bibb.de/veroeffentlichungen/de/bwp/show/id/7273
- Bundesministerium für Wirtschaft und Energie. (2017). Das duale Berufsausbildungssystem in Deutschland. Berlin. Retrieved from http://www.bmwi.de/Redaktion/DE/Downloads/C-D/duales-berufsausbildungssystem-in-deutschland.pdf?__blob=publicationFile&v=9
- Furs, V. V. (2005). Testirovanie kak forma provedenia ekzamena po gumanitarnym disciplinam. In I. L. Zelenkova & A. A. Legchilin (Eds.), Kontroliruemaia samostoiatelnaia rabota studentov: sostoianie i perspektivy. Materialy nauchnometodicheskoi konferencii (pp. 76–80). Minsk. Retrieved from http://elib.bsu.by/handle/123456789/44522
- Government of the Kyrgyz Republic. (2012). Polojenie ob itogovoi gosudarstvennoi attestacii vypusknikov obrazovatelnoi organizacii srednego professionalnogo obrazovania Kyrgyzskoi Respubliki, Pravitelstvo Kyrgyzskoi Respubliki 4.7.2012.Google Scholar
- Government of the Republic of Tajikistan (2015). Zakon Respubliki Tadjikistan “O srednem professionalnom obrazovanii.” Tajikistan. 8.8.2015.Google Scholar
- Ministry of Education and Science of the Republic of Kazakhstan. (2008). Tipovye pravila provedenia tekushego kontrolia uspevaemosti, promejutochnoi i itogovoi attestacii obuchaushihsia v vyshih uchebnyh zavedeniah, Ministerstvo Obrazovania i Nauki Respubliki Kazahstan 18.3.2008.Google Scholar
- Rummler, M., Nehls, T., Jordan, P., & Heitmann, G. (2012). Innovative Lehrformen: Projektarbeit in der Hochschule: Projektbasiertes und problemorientiertes Lehren und Lernen (1. Aufl.). Hochschuldidaktik. Weinheim: Julius Beltz.Google Scholar
- Schenk, H.-O. (2005). Die Examensarbeit: Ein Leitfaden für Wirtschafts- und Sozialwissenschaftler. Uni-Taschenbücher Arbeitshilfen Wirtschaftswissenschaften (Vol. 2657). Göttingen: Vandenhoeck & Ruprecht.Google Scholar
- Stehling, C. (2015). Vom Frontalunterricht zu mehr praktischer Anwendung. Herausforderungen der Berufsbildung in Zentralasien. Zentralasien-Analysen (93), 2–6. Retrieved from http://www.laender-analysen.de/zentralasien/pdf/ZentralasienAnalysen93.pdf
- Walzik, S. (2015). kompetenzorientiert prüfen an hochschulen: kriterien für kompetenz-orientierte leistungsnachweise. weiterbildungshalbtag an der phlu 21. august 2015. Luzern. Retrieved from PH Luzern website: www.phlu.ch/fileadmin/media/phlu.ch/uu/hd/handout_walzik_phlu_20150821.pdf
- Weinert, F. E. (2001). Leistungsmessungen in Schulen (2., unveränd. Aufl., Dr. nach Typoskript.). Beltz Pädagogik. Weinheim: Beltz-Verl.Google Scholar
- Weiß, R. (2011). Prüfungen in der beruflichen Bildung - ein vernachlässigter Forschungsgegenstand. In E. Severing & R. Weiß (Eds.), Schriftenreihe des Bundesinstituts für Berufsbildung, Bonn: Vol. 10. Prüfungen und Zertifizierungen in der beruflichen Bildung. Anforderungen – Instrumente – Forschungsbedarf (pp. 37–52). Bielefeld: Bertelsmann.Google Scholar
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.