Keywords

1 Competency-Based Exams

When providing education services, educational institutions’ primary aim should be to enhance the quality of their services and also the level of knowledge students acquire during this process. Average performance levels indicate the quality of education not only in a particular institution but also in the whole educational system. In the context of general education, this principle serves large-scale international comparisons like the PISA and the TIMMS study. Therefore it is worth exploring this aspect further in the context of professional education.

Indeed one of the forms to ensure education quality is by examination. In educational institutions in Germany and in the Central Asian countries of Kazakhstan, Kyrgyzstan and Tajikistan, exams are used as tools to evaluate interim performance, as well as a final assessment. While the interim performance evaluation serves to provide feedback and recommendations for students’ further development as well as for teachers’ professional activities, the main aim of the final exam is to some extent to assess skills and competences and to provide an overview of a graduate’s abilities that can be used on the labour market. In other words, the exam results provide a short, unified and clear description of a potential employee’ skills and comparability. To ensure better quality and labour market orientation, exams should be competency-based.

In literature, ‘competences’ are discussed differently, and there are big differences between European and Anglo-American understandings. The following definition meets the understanding of the authors and will be used throughout this article. According to Weinert (2001), ‘competences are inherent or developed by a person cognitive abilities and skills to solve particular problems and connected with this motivational, volitional and social readiness and ability to responsibly and successfully use these solutions in different situations’Footnote 1 (id. 2001, p. 271). In this way, the main objective of competency-based examinations is not only to define the level of knowledge students have but also their ability to apply this knowledge in solving different types of problems, mainly in their professional life.

The three abovementioned Central Asian countries recognize the importance of linkages between the educational system and the labour market (Stehling 2015). Thus, the National Sustainable Development Strategy for the Kyrgyz Republic for 2013–2017 underlines that VET institutions should adequately collaborate with the private sector. The Strategy ‘Kazakhstan-2050’ states that curricula should have more practical content and should focus on labour market needs. The Tajik National Education Development Strategy also admits that the VET education does not meet labour market requirements and companies should take part in educational processes. So, all three countries recognize that integrating elements of dual education into the national VET systems is crucial for increasing the quality of vocational education. It is also stated in these documents that employers should be involved in assessing graduates’ skills, as national legal frameworks contain further details on this issue.

2 Research Methods and Data Collection

In this research, interviews have been carried out. These interviews are based on literature and study of examination protocols. First, in the framework of a GIZ project in Tajikistan, a series of examinations for motorcar mechanics under the guidance of the German experts has been carried out. Observation criteria such as planning and organizing examination process and follow-up were agreed upon in advance and served as a basis for observations during preparing and conducting examinations. Then, the observations were documented in the form of protocols. Based on the latter, face-to-face interviews with teachers from Kazakhstan and Kyrgyzstan have been carried out. Another method used in this research is to study and to compare legal acts in the sphere of vocational education and training in the given countries. Results are presented in the form of recommendations provided in this article.

3 Legal Frameworks for Carrying Out Examinations in Professional Education in Central Asia

Since officially certified performance is a precondition for labour market entry, the performance should be assessed accordingly and final exams are strictly regulated. Taking into consideration that these requirements are of a compulsory nature, only relevant state bodies can be responsible for providing the main rules and regulations. In Kazakhstan, Kyrgyzstan and Tajikistan, these are ministries of education and, partially, ministries of labour (in the field of adult education). Germany follows the principle of the dual system in professional education. Based on this principle, professional associations such as the Chamber of Crafts (Handwerkskammer) or the Chamber of Industry and Commerce (Industrie- und Handelskammer) are responsible for the content of educational plans and examinations. Thus, as members of these associations, private companies are involved in the professional education and examination processes. In turn, every association has its own examination regulations.

For example, if one compares the examination regulations of the Chamber of Crafts Kassel, a city in the Federal German State of Hessen, with the national assessment regulations in VET in Kazakhstan, Kyrgyzstan and Tajikistan, it stands out that the regulations of Central Asian countries are more general. Rules on the appointment of the examination committee members and preparation of the examinations are quite vague. Not all aspects of conducting exams (for instance, participation in exams of persons with special needs) are covered, and assessment criteria are not clearly provided. In this way, regulations leave much room for interpretation, which transfers added responsibility onto examination committees and makes the results of assessments more difficult to compare.

Nevertheless, forms of final assessment and requirements of different examination committees can be more or less compared.

From Table 15.1, one may conclude that the legal provisions in Central Asian countries and Germany specify various types of examinations:

  • Written exams: This is usually the most common examination type in Central Asian countries, as it ideally ensures unbiased assessment and is easy for documentation. If students have complaints about their assessment, the exam results can also be easily reviewed. Apart from closed questions and multiple choice questionnaires (tests – this type is considered in this article as a separate type of examination), questions can be used, which require thinking out of the box, analytical skills and explanations of conclusions. However, there is hardly any complex task-setting. Most written exams test students’ knowledge in a particular subject but not necessarily their abilities to find solutions to complex problems.

  • Tests: As a form of the written examination is mostly used for continuous monitoring of students’ performance, this method has gained increasing popularity among teachers. First of all, it allows assessing large groups of students in a finite time period as well as opportunities for comparison. Secondly, it limits the ability of external influences to compromise the quality of the assessment. Tests also help evaluate students’ knowledge across topics of the training course. On the other hand, the nature of this method does not allow for a comprehensive measurement of acquired knowledge or an evaluation of the ability to use this knowledge and bring it to implementation (Furs 2005).

  • Oral exams: These are used for interim performance evaluations and presentations of the diploma project. The standard form involves students answering questions posed directly by examiners who then ask clarifying or control questions. In the three Central Asian countries, this form is used most often. Other forms of oral exams such as case-related debates, discussions of case studies and role games are barely used at all. One reason could be that teachers lack skills in assessing students’ performance in such an ‘unstructured’ way. Or, overwhelmed by reports and other paper-work, they simply do not have the time to invest in more sophisticated forms of examination. Support from managerial personnel should play an important role here, but it is not always forthcoming.

  • Practical exams: Most of these exam types are used in technical vocational education. This type of examination requires more time for preparation which entails both the design of tasks and the testing of equipment that should be ready to use on the day of the exam. Observations from Tajikistan, for instance, show that this rule is often neglected and equipment is in nonworking condition (and is sometimes even being repaired) as the examinations take place.

Table 15.1 Forms of final assessment and compositions of examination committees at VET institutions in Kyrgyzstan, Kazakhstan, Tajikistan and Germany
Full size table

As different types of exams assess various types of competences, a combination of examination forms is often used. For instance, skills evaluated during oral examinations (presentation skills, ability to explain the way of thinking) cannot be fully assessed by purely practical exercises and vice versa. When using the practice-oriented approach, not only knowledge and skills but also competences are evaluated with a focus on the labour market. It is the role of the private sector to evaluate whether skills acquired during the training are applicable in the labour market. Especially crucial is this when national qualification frameworks are missing (among the three countries, only Kazakhstan has developed this framework).

Based on an observation of examination processes in the three countries, it follows that students lack experience in handling comprehensive tasks and have insufficient problem-solving skills. One reason could be that tasks are usually given based on particular parts of a training course and comprehensive tasks that require extensive problem-solving skills are the exception rather than the rule. This is closely connected with the teaching methods. The value of understanding complex problems and interconnections between problems should be made integral to the whole training process.

4 Requirements of Examiners and Examination Committee

4.1 Examiners and Members of the Examination Committee

During interim performance evaluations, students’ knowledge and skills on specific topics are assessed by their teachers. However, for final exams or presentations of a diploma project, an examination committee consisting of a minimum of three members, and usually up to five or seven, is formed. The qualitative composition of the board is also considered. This is ensured by the principle of parity, which means that apart from the teaching and managerial personnel of the educational institution, representatives of potential employers should be included into the committee. As stated above, educational institutions are vital measurements for the private sector, since when employing graduates, companies rely on their grades and assessment given to them by educational bodies. The grades earned in final exams and the average final grade in most cases play a significant role in ensuring a young person’s entry into the labour market. Thus, the examination committee should embody not only specific knowledge but also practical experience in the topics to which the questions relate.

For this reason, a minimum of one employer representative should be present. This is provided in the legal provisions in all the Central Asian countries mentioned. However, the rule is not always followed: many representatives of companies (especially in the private sector) do not recognize the importance of being involved in examination processes and view participation in examinations as a distraction from their everyday work.

It is possible that it is precisely the fact that representatives of the private sector are only involved into the process as late as in a final exam which reduces their interest and trust in the results of assessments. For this reason, their involvement into the entire training process of a student or trainee is a key requirement to improve the VET system embedded in the national strategies.

4.2 Methodological and Didactical Skills of Examiners

Apart from specific knowledge, the examiners should possess methodological and didactical skills (a competence matrix for examiners can be developed) related to the topics of the exams. Examiners should follow the same rules and ensure a standard pattern of behaviour. Observations of several exams in VET institutions in Tajikistan, however, indicate that the quality of the examiners, or rather the quality of their procedures and methods, is not equal or consistent. This is especially true regarding their approach of asking questions: some ask interposed questions, others guiding questions and others still simply observe and take notes. In this regard measurements to develop consistency should be taken, for instance, conducting exam-related trainings for teachers.

Examiners should know how to ask appropriate guiding questions, provide pedagogical or even psychological support to students and remain unbiased. Neutral and bias-free treatment of students during examinations is one of the main requirements for competent examiners. Sometimes it is not easy to meet this requirement, however, since some factors can lead to a distortion of an examiner’s perception and, consequently, result in subjective judgments over students’ competences. German scientist Sebastian Walzik names the following factors: first impression, prejudices, stereotypes, contrast effect and context effect (Walzik 2015). Examiners cannot adopt an appropriate unbiased position if they have a particular relationship with students (relatives, colleagues, etc.). Hence, this fact should be excluded during the formation of the examination committee, although it is not clearly mentioned in the examination regulations of either Kazakhstan, Kyrgyzstan or Tajikistan.

Another critical task for an examiner in competency-based examination is to distinguish competences from performance. For instance, a student can find a solution of a task by good luck (performance) or by applying his or her skills (competence). In order to make sure that the student can really apply and implement his or her skills, the teacher should ask at least one additional question or ask the student to explain their way of thinking and how they came to a solution. Or, in other cases, due to some unanticipated factors (excitement, confusion, or non-confidence) young people can underperform, but a guiding question or a sign of support from the examiner’s side can help them to relax and apply their real potential. The given example relates to oral exams. In written exams it is not feasible to use these methods (provoking, supporting, guiding questions), but other methods can be used in order to make students use their potential. One is to develop an appropriate way of formulating questions or problems. A very easy and telling illustration of this is provided by Sebastian Walzik 2015):

$$ (1)\kern0.50em 3\times 2+1=?(2)\kern0.50em 1+2\times 3=? $$

  • Which of these two tasks helps assess students’ competences?

Although the right answer in both cases is the same, the first task (1) allows only assessing the ability of a student to fulfil simple arithmetic operations. The second one (2) helps students show not only knowledge of arithmetic but also their ability to apply math rules.

Moreover, examiners should know how to handle non-standard or even emergency situations. What to do if a participant is late due to objective or subjective reasons? As a common rule in Central Asian countries, latecomers are not allowed to enter the examination room and should take the exam later (upon decision of the examination committee). However, this leads to increasing examination costs borne by educational institutions. Therefore it would be reasonable to introduce a solution such as late arrivals – up to 15 min can be accepted (at least for written exams) – without adding extra time after the end of the exam. This goodwill regulation could meet requirements on examination effectiveness and simultaneously reduce stresses connected with repeat exams.

Another non-standard situation is a power blackout or breakdown of equipment used during the examination. On the one hand, it is of course impossible to foresee all non-standard situations and provide description of steps to follow. Therefore training should be made available for examiners on ways to handle unforeseen circumstances depending on the educational institution’s capacities and the preparations made by examiners for such situations.

5 Training of the Examiners

As mentioned above, the methodological and didactical preparation of examiners is one of the most vulnerable components of the educational systems in Kazakhstan, Kyrgyzstan and Tajikistan, which requires additional testing. Some teachers, who also act as examiners in their institutions, report a lack of instructions providing clear guidance for examination processes and competences assessment. There are also no guidelines on the way students should be treated (how to provide necessary support, while remaining bias-free, how not to be distracted by external factors such as students’ looks or noises outside of the class room, etc.).

Taking all these arguments into consideration, it is advisable to carry out not only singular trainings but also regular further trainings for examiners. Systematic feedback should be provided to aid quality control, and possible improvements should be discussed. Then, the lessons learnt should be applied in preparation for and during examinations (Weiß 2011, p. 45). Crucial support is also provided to examiners in the form of general information, practical advice and platforms for the exchange of experience. Measures to explore these opportunities in the Central Asian countries should be taken.

6 Conclusion

Experience has shown that the legal framework for competency-based evaluation is in place in all the aforementioned Central Asian countries. However, the frameworks, regulating examinations in Central Asia, are rather general compared to the ones in Germany (Bundesministerium für Wirtschaft und Energie 2017). For this reason a member of the examination committee has a much higher responsibility regarding the implementation of comparable standards into examinations as well as comparable assessment criteria. Neither Kyrgyzstan nor Tajikistan has implemented a National Qualification Framework yet, leaving Kazakhstan as the only country that has met this requirement. Even more important is the participation of the private sector in defining the content of trainings and apprenticeships as well as being part of examinations to evaluate competences acquired.

To fulfil this weighty responsibility, members of the examination committees require all possible support in the form of ongoing trainings and facilitating the exchange of experiences between committees. Taking into account that the national strategies in these countries focus on the implementation of dual elements in professional education in order to meet the needs of a rapidly changing labour market as well as on tackling challenges inherent in economic globalization, examiners have to be both highly skilled and experienced. There are encouraging examples of responsible cooperation with the private sector. Clear descriptions of job profiles on the part of the private sector will help train young people according to key requirements and consequently improve their perspectives in the labour market. Experience exchange of best practices in VET networks can be a focal point for continuous improvement in the area of evaluation of competences, as well as the ongoing training of examiners.