International Quality Reviews with an EQAR-Registered Agency

Abstract

The present study provides a first analysis of higher education institutions’ experience with a crossborder quality review (evaluation/audit/accreditation, at programme or institutional level). The paper, based on the results of 13 case-study interviews, paints a comparative picture of the rationale as well as of the benefits and challenges experienced by higher education institutions that had their external quality assessment carried out by a quality assurance agency from another country. The study explores whether these reviews are embedded in the internationalisation strategy of the institution, how they contribute to the organisation’s quality culture and the role, played by the national legislative framework as inhibiting or facilitating factors for such reviews. The paper complements these case studies with a macro level analysis on the national openness of European Higher Education Area (EHEA) countries to quality assurance agencies working across borders, based on substantial compliance with the European Standards and Guidelines for Quality Assurance (ESG) attested by its listing in the European Quality Assurance Register for higher education (EQAR). The macro level analysis looks at the commitment taken by EHEA Ministers to “allow EQAR-registered agencies to perform their activities across the EHEA, while complying with national requirements”.

1 Introduction

Quality assurance of higher education is arguably the most successful “action line” set out by ministers within the Bologna Process. One reason for its successful development is that it was put at the heart of the efforts to build a European Higher Education Area (EHEA) (Bologna Process Stocktaking, 2005, p. 16). An extra drive was given even before the Bologna Declaration by the European Council’s recommendation from 1998, and then followed by a joint recommendation with the European Parliament in 2006 (European Parliament and Council Recommendation 2006). The support received from European Commission, but also from association of higher education institutions, from a number of national governments, students and quality assurance agencies have further strengthened the development of a European dimension in quality assurance (Sursock 2012, p. 247).

The adoption of the “Standards and Guidelines for Quality Assurance in the European Higher Education Area” (commonly referred to as ESG) in 2005 following the proposal of the E4 Group¹ was the result of a major commitment of Bologna countries to quality assurance. This has provided the basis for European QA developments and discussions and has paved the way for the establishment of the European Quality Assurance Register for Higher Education (EQAR 2011).

EQAR is the first organisation with legal entity to have emerged directly from the Bologna Process. It was set in place with the main purpose of allowing stakeholders and the general public open access to trustworthy quality assurance agencies (list of registered agencies) working in line with the European Standards and Guidelines for Quality Assurance (ESG).² Six years after its establishment, the Register includes 32³ quality assurance agencies from 16 different EHEA member countries.

Since its’ founding in 2008, EQAR has developed and gained credibility among governments, stakeholders and the general public as shown in its external review in 2011 (EQAR External Evaluation Report, p. 9) and by the interest of agencies seeking registration. Moreover, with the Bucharest Communiqué (2012), EHEA ministers committed further to ‘allow EQAR-registered agencies to perform their activities across the EHEA, while complying with national requirements’ and to ‘recognise quality assurance decisions of EQAR-registered agencies on joint and double degree programmes.’ This commitment would provide higher education institutions with the possibility to choose among registered accreditation or quality assurance agencies that fit their needs and profile for their external quality assurance review. Using the Register as a proxy for cross-border external quality assurance was supported by the European Parliament and the Council of the European Union in 2006 to enhance European cooperation, and emphasized again in the European Commission’s Report on Progress in Quality Assurance of Higher Education (2009, p. 21).

The Bologna developments in quality assurance have recorded a fast progress when it comes to the external QA dimension, e.g. 22 QA agencies established between 2000 and 2010 (Eurydice 2010, p. 25). At grass-root level, the QA developments have followed a slower pace, encouraged however by the setup of national QA frameworks and QA agencies. EUA’s Examining Quality Culture survey (Loukkola and Thérèse 2010, p. 22) and EURASHE’s study (Voldánová et al. 2012, p. 30) indicate that most institutions developed or changed their internal QA under the influence of Quality Assurance Agencies (QAAs) and national regulations. The differences in focus and philosophy of QAAs have shaped the internal processes of institutions differently and they have not always been linked to the introduction of the ESG (Sursock and Smidt 2010, pp. 21–22). It becomes apparent that the maturing QA “Bologna infrastructure” has not followed a similar script across national systems, and the implementation of the ESG (particularly when referring to internal QA) has often lacked consistency or congruence with the European agenda.

The Bucharest commitment aimed at supporting higher education institutions in choosing among any EQAR-registered agency for assessing their internal quality or for accrediting their programmes. These external quality assurance (EQA) reviews would foster both the implementation of the ESG and the development of institutional QA frameworks in line with the ESG. To understand the extent to which this aim has transpired within the diverse landscape of EHEA national systems, it is important to gauge into the institutional frameworks that ensure the quality of degrees across Europe. So far, the knowledge on the institutional experience with a cross-border quality assurance agency is rudimentary, mostly exemplified in the form of single case study description (see EQAF 2012 paper on VDTK & evalag).⁴ To widen the understanding of the dimension of cross-border external QA and the institutional experience with a cross border quality assurance agency, EQAR has designed and carried out a research project.⁵ The current paper extends on the analysis carried out as part of this project, addressing the higher education institutional experience with a cross-border quality review (evaluation/audit/accreditation, at programme or institutional level).

1.1 National Quality Assurance Infrastructure

Before undertaking the case study interviews, desk research was carried out to map national legal frameworks and their openness to external QA. This mapping enabled the selection of institutions from countries where the cross-border EQA of an EQAR-listed agency is recognised as part of the periodic external review and where this can only be done on a voluntary basis (in addition to the periodic review). The resulting analysis⁶ showed that there are different levels to which a higher education institution is able to discharge their obligatory external quality assurance through review by any EQAR-registered QA agency. Some countries allow all HEIs to choose a registered agency for all types of external quality assurance obligations they are subject to. In other countries, the ability to choose a quality assurance agency is limited to a certain group of HEIs (e.g. full universities in Austria) or to certain types of external QA (e.g. only for programme accreditation, but not for institutional accreditation; or not for initial accreditation). Certain countries recognise reviews by foreign QA agencies only for joint degrees, transnational provision or other specific circumstances, while others use different requirements than EQAR registration for allowing QA agencies from other countries to carry out reviews (Fig. 1).

Fig. 1

Mapping the openness to EQAR-registered QA agencies within EHEA (as of September 2014). Dark blue Countries recognising EQAR-registered agencies as part of the national requirements for external QA (Albania, Armenia, Austria, Belgium: Flemish Community, Bulgaria, Denmark, Germany, Kazakhstan, Liechtenstein, Lithuania, Poland, Romania). Light blue Countries recognising foreign agencies as part of the national requirements for external QA. Gray Countries not open to external QA evaluation by a foreign QA agency (colour figure online)

1.2 Case Study Methodology

To understand the dynamics of internal quality assurance in a pan-European setting, a multiple-case study research was developed. This research method facilitates the exploration of similarities and contrasting results when looking at different institutional environments.

By “institutional experience” this paper refers to the perspective from inside a higher education institution (as opposed to the perspective of a quality assurance agency or an outside stakeholder), and the term is used irrespective of whether the review was carried out at the level of the entire higher education institution, a faculty or at the programme level.

1.3 Sampling Countries and Higher Education Institutions

In selecting the case studies the diversity of the external quality assurance (EQA) frameworks and approaches was taken into account.

The countries (in the case of Flanders: community) were selected with the aim to represent a geographically balanced sample and to provide a relevant mix of quality assurance frameworks across the EHEA.⁷ To achieve this heterogeneity, national higher education frameworks were selected based on the differing characteristics of their QA systems (openness to cross-border reviews, stage of development of the external QA system, diversity in terms of outcomes of QA reviews) and geographical balance.

Two groups of countries differing in one main characteristic of their national higher education setting were selected as follows:

• 8 case-study interviews in four countries that recognise EQAR-registered QA agencies as eligible to satisfy the official requirements for external QA and

• 4 case-study interviews in four countries where cross-border external QA (EQA) is being carried out on a voluntary basis, in addition to the periodic obligatory external review.

Similarly, the selection of HEIs was made taking into account the diversity of EQA experience with a cross-border review (i.e. programme and institutional reviews, as well as joint or double degree programmes) and the representativeness of the HEI within the national QA system (whenever possible). The case study interviews were conducted at institutions where EQA has been carried out with an EQAR listed agency. There were two distinct exceptions: EFMD’s institutional accreditation under the EQUIS label carried out in 2014 at the University of Lund (Sweden) and the IEP review carried out in 2007 at the University of Aveiro (Portugal). IEP was later included in the Register but at the time of the review, EQAR was not operational. The EFMD review was chosen to provide a different perspective for carrying out a review across borders with an EQAR-registered agency.

The final results of the selection are presented below⁸:

• 1. Higher education institutions from 4 countries that recognise reviews of foreign EQAR-registered agencies as part of the national requirements for external QA (National setting I):

  Austria: University of Vienna (OAQ, quality audit 2013), University of Graz (FINHEEC, institutional audit, 2013)

  Belgium: Flemish Community (BE-NL): Ghent University (AQAS, joint degree accreditation 2012/2013), Belgium: Royal Military School (CTI & NVAO joint review, 2011)

  Lithuania: VTDK University (evalag, programme accreditation 2011); Mykolas Romeris University in Vilnius (AHPGS, programme accreditation 2011)

  Romania: University of Bucharest (IEP, institutional evaluation 2012) & Dimitrie Cantemir University from Targu Mures (AHPGS, programme accreditation 2012);

• 2. Higher Education institutions from 4 countries that do not recognise (or are in progress of recognising) cross-border EQA reviews (National setting II):

  Croatia: University of Zagreb (ASIIN, Faculty of Electrical Engineering and Computing, accreditation 2013 and Faculty of Civil Engineering, programme accreditation 2013);

  France: Centre d’Etudes Supérieures Européennes (CESEM) at NEOMA Business School (FIBAA, accreditation of a double degree 2011);

  Sweden: University of Lund (Lund School of Economics and Management—LUSEM, EFMD accreditation 2014);

  Portugal: University of Aveiro (IEP, institutional evaluation 2007).

1.4 Design of the Study and Conceptual Framework

A conceptual framework was designed to guide the case-study research questions on the institutional experience with a cross-border EQA (Fig. 2). The research questions focus on the rationale for a cross border review, the specificities of the review process, the institution’s experience and the impact of the review. The same general line of questioning was used in both national settings, to gather a comparative perspective.

Fig. 2

Case-study conceptual framework

1.5 Data Collection Methods and Instruments

Semi-structured interviews have been carried out with key representatives and stakeholders of each higher education institution (i.e. leadership, coordinator of the institutional/programme EQA, representative of the QA department, QA council, student representatives, management). Following the interview, a report summarising the main findings for each case study was prepared. The reports of the review were checked for factual inaccuracy or possible misinterpretation by each of the interviewed institutions.

The confidentiality of the specific information provided was ensured to interviewees so as to allow disclosure of possible critiques and to increase openness.

1.6 Case-Study Research Questions

In order to facilitate a systematic comparison of cases, a common set of research questions was developed. The interviews are nevertheless contextualised within their different national settings (NS1 & NS2). In addition, specific questions have been added according to the particularity of the review (e.g. double/joint degree programmes, joint QA review etc.).

The main research questions for the case studies are presented in Table 1. The questions are not intended to be a pre-set checklist but, rather, a set of thematic guidelines. To some extent the sections might overlap due to the similarities among the researched elements.

Table 1 Case-study research questions

1.7 Overview of Case-Studies

The analysis of institutional experiences with a cross-border external quality assurance (EQA) (evaluation/accreditation/audit at institutional level or programme level) has been portrayed in the cross-case synthesis presented below. The contacted institutions have been asked whether they have carried out any additional cross-border EQA activities with an EQAR or non-EQAR registered agency. The additional cross-border reviews have been considered within the initial analysis as they complement the general findings and provide a more comprehensive overview of the cross-border experience within the institution. A Table 2 with the overview of all cross-border EQA activities reported upon by the 12 interviewed institutions are presented below.

Table 2 Overview of cross-border EQA activities within the selected case-studies

1.8 The National Context for the Selected Case Studies

The reviews were carried out with the purpose of a programme accreditation (Romania, Lithuania), an institutional audit (Austria) or as part of a joint programme⁹ accreditation (Belgium). The following reviews were all recognised as part of the initial or periodic EQA requirements,¹⁰ Higher education institutions in the Flemish Community of Belgium can have the review for their programme accreditation carried out by foreign agencies. This review will be the basis for accreditation of the study programmes by the NVAO. All EQAR-registered agencies are entitled to carry out the assessment reviews, but have to agree to the Terms of Reference with NVAO beforehand.

To carry out a review with a foreign QA agency in Lithuania, the institution must launch a public call for tender, in conformity with the “Procedure for the external evaluation and accreditation of study programmes”, issued by the Minister of Education and Science. The call includes a short description of the study programme(s) to be accredited (e.g. cycle, study area, field), the requirements and criteria for the external evaluation (i.e. evaluation scale), the time frame of the review process, as well as some specific requirements from experts (e.g. qualified specialists in the area of study). The law specifies that the external evaluation may be performed by a foreign agency included in EQAR, while the national QA agency will take an accreditation decision on the reviewed study programme. The two universities selected as case studies are the only two Lithuanian institutions that had programmes reviewed by a foreign agency.

In Austria, there is no requirement for a public procurement procedure, unless the cost of the review process would exceed EUR 50 000. Public universities have to undergo an audit of their internal quality assurance system periodically every seven years and can choose to have that audit carried out by the national agency (AQ Austria), a suitable EQAR-registered agency or another agency recognised by the ministry responsible for higher education. The universities of Graz and Vienna are two of the other public universities that opted for a review by a foreign agency (Uni Wien, Uni Graz, WU Wien, VetMed Wien, Innsbruck, Leoben).

Romanian higher education institutions can choose for their programme accreditation and periodic institutional evaluations the national agency (ARACIS) or another suitable EQAR-registered agency. The external review body must however comply with the national regulation and other international field related standards.¹¹ Although the procedure for a cross-border EQA with an EQAR-registered agency seems straightforward, there has been some uncertainty as to the practical application of that legal provision and there is currently only one example of such a review carried out in Romania.

In the following countries, reviews were undertaken “voluntarily”, i.e. in addition to the obligatory national QA reviews. However, having access to specific funding streams to cover the cost of an international accreditation, the institution was more likely to choose an EQAR-registered agency (i.e. VDTK, Lithuania) or by a QA agency with considerable international experience (IEP, EFMD). The “voluntary” type of reviews carried out in both national settings (NS1 & NS2) is most often improvement-led as it has no consequence for the public funding or accreditation of a study programme.

Croatian higher education institutions are subject to different types of external quality assurance organised by the national Agency for Science and Higher Education (ASHE, Croatian acronym: AZVO). Public universities are self-accrediting as regards their study programmes, but subject to an institutional audit and reaccreditation. Even though called “re-accreditation of higher education institutions”, these actually refer to separate faculties. Evaluations, accreditations and audits carried out by foreign quality assurance agencies are done in addition to the obligatory national reviews, but are not recognised to replace or form part of the national external quality assurance framework.

In France, the authority to confer degrees is granted and renewed by the Ministry of Higher Education and Research. The certification (“habilitation”) is offered after reviewing the application presented by the institute in question. The review is usually done by the national quality assurance body, AERES which is in charge of institutional evaluation, research unit evaluation and bachelor, master and doctoral programme evaluation. However, when it comes to evaluating engineering programmes, the review is carried out by CTI (Commission des Titres d’Ingénieurs).

The quality assurance system for higher education in Portugal was set up by the Rectors’ Council in 1990 and comprised different coordination councils who assessed the quality of the public, private and polytechnic sectors. After the system was reformed in 2007, the national Assessment and Accreditation agency, A3ES was assigned to accredit study programmes in a five-year cycle. In addition, since 2012 A3ES has started quality audit procedures as a pilot exercise. The institutional audit is to be fully implemented in 2016, after the first accreditation cycle is completed. The results of the assessment or accreditation procedures requested by Portuguese higher education institutions from other national or foreign quality assurance bodies may be recognised depending on the protocols of agreement and decision of the Executive Board of A3ES.

In Sweden, the authorisation for public institutions to carry out programmes leading to a degree-level award is provided by the national agency (with the authority of the Ministry) following a review. The national agency (HSV) also carries out the periodic evaluations of programmes. For a positive programme accreditation, the decision is valid for four years. Reviews carried out by foreign quality assurance agencies (QAAs) cannot replace the periodic reviews of HSV and are carried out on a voluntary basis.

2 Case-Study Analysis

2.1 The Rationale Behind a Cross-Border EQA

Higher education institutions turn to an agency active across borders not only to fulfil their periodic EQA requirements, but also to enhance their reputation, increase the employability of their graduate or to develop their own internal quality culture (Fig. 3).

Fig. 3

Rationale for EQA considering the type of cross-border EQA

Institutions referred to the following main reasons when asked about their rationale for a cross-border review:

2.1.1 Increasing the International Visibility and Reputation

Considering the international profile of business schools and research-oriented institutions, the decision to carry out a review is to enhance the institution’s profile internationally and, as a result, extend its partnerships and collaborations within the network. A high number of international partnerships (e.g. bilateral agreements, educational and research programmes, networks and research collaboration) are supported with this type of accreditation.

Small and regionally established higher education institutions consider the recognition provided by an international accreditation body as a way of attracting more students not only from the national pool, but also from outside the country.

2.1.2 Achieving “Bologna-Compatible” Degrees

The decision to carry out the review was set in the context of the Bologna Process reforms. A few of the interviewed institutions saw the international accreditation agency as a way to make their studies more attractive for the labour market by having their programmes and qualifications recognised/certified as “Bologna-compatible” and in line with European standards. This was mainly the case where the national QAAs were not yet established or had not yet been reviewed against the ESG.

2.1.3 Quality Culture

More than half of the selected institutions have carried out at least two external reviews with a QAA active in cross-border QA, either at the programme or institutional level. The interviewees saw these external evaluations/audits as contributing to the development of their internal quality culture. Building upon the experience of previous reviews, the institutions found that they had improved their internal quality arrangements (better developed structures and processes) and approach to internal quality.

2.1.4 Development of Institution’s Management and Organisation

One institution sought external expertise to develop the strategic goals of the institution (e.g. assess less developed areas of the university, enhance its research infrastructure, regional development etc.). This institution placed a high emphasis on the recommendations of the external review panel, as essential in consolidating the institutional development goals.

2.1.5 In the Context of National Reforms

EQA activities were at times commissioned by ministries to implement a larger higher education reform agenda. One of the selected case studies was reviewed as part of a national EQA exercise (2012–2014), co-financed by the European Union’s structural funds and implemented by UEFISCDI¹² and EUA’s Institutional Evaluation Programme (IEP). Within the framework of the project, IEP carried out 70 reviews of public higher education institutions during three rounds of evaluations (between 2011 and 2014). The project was set to improve the management and quality assurance within the Romanian higher education system by strengthening the strategic capacity and autonomy of universities. Additionally, the project sought to provide the Ministry with an independent international opinion on which to base its future strategic decisions regarding institutional development.

2.1.6 Regulated Professions

Although these cases were not the focus of the case studies, some institutions reported that some of their programmes had undergone an international accreditation to fulfil the required educational standards for specific EU regulated professions. For instance, the Faculty of Veterinary Medicine (University of Gent) reported that it has to comply with a number of standards in the preparation of veterinary surgeons since the profession is regulated by the European Union (Directive 2013/55/EU). The EAEVE review provides recognition for veterinary education establishments if they have achieved the EU minimum compulsory requirements for this profession.

2.1.7 A Second Opinion

Some institutions may decide to commission a non-national QA review to try out a different approach or methodology that the one provided by the national QA agency. This might be due to the institution’s belief that that agency that carried out the review(s) in the past did not understood the institution and its work sufficiently well.

2.2 Selection of a Suitable QAA

Institutions considered a number of criteria that played a role in the final selection decision (international reputation, country of origin, expertise in a particular field, affordability etc.). In some cases, no initial decision was taken to choose a foreign or a national QA agency, and both options were considered equally.

The choice of a suitable agency usually involves considerable desk research for higher education institutions, for which they use information provided on the EQAR website and on the agencies’ own websites (e.g. expertise in different methodologies of external quality assurance, countries where agencies have worked). Institutions often find themselves in the situation that only a small number of the (currently) 32 EQAR-registered agencies would at all be suitable to undertake the review that is needed.

2.2.1 Fulfilling the Legal Requirements

A pre-condition for choosing a QAA in the case of higher education institutions seeking to discharge their external QA obligations was to only consider QAAs that fulfilled the national legal provisions. QAAs active in cross borders QA were required to have expertise in a certain type of EQA (audit/accreditation/evaluation). In many cases, agencies are also required to use a set of national criteria. This gave an advantage to those agencies with prior experience in the country or those that could articulate clearly how they would carry out the review in the specific country.

2.2.2 Language

Among the selected institutions the common language of the international QA review was English. In a few cases, institutions requested German, French and Dutch as the main language of the QA review process. The requirement of carrying out the EQA in the official language of the country was either an internal decision (in case of bilingual institutions), or it was requested to ensure a more efficient review process, i.e. to reduce the possibility of misinterpretation in the use of technical terms and to increase acceptance of the review among internal or external stakeholders.

2.2.3 International Experience and Expertise

Due to the international dimension of the degree programmes, in particular in the case of Erasmus Mundus and other joint and double degrees programmes, institutions were considering QAAs that could have an international-led approach in reviewing the quality of the programme. Institutions also mentioned they looked at the portfolios of international activities and at the presentation of QAAs’ procedures and cross-border review policies.

2.2.4 Peers

The interviewees mentioned they preferred QAAs with a wider pool of experts and had asked for panel members with expertise in a given field. In two cases, the institutions requested the cross-border QAA not to include experts from within their country. Due to a limited pool of national reviewers, the higher education institutions feared the biased view of experts coming from one of the ‘competing’ higher education institutions.

One of the institutions mentioned they were dissatisfied with the lack of training requirements for specialists used by one of the reviewing agencies (a non-EQAR registered agency).

2.2.5 Costs

For some of the interviewed institutions (large and mature HEIs) costs did not play a major role, even though the institution relied on its own budget to cover the review costs.

These institutions noted that the internal costs (preparation, self-evaluation, etc.) were anyway significantly higher than the cost of the review as such. The situation was different for those institutions that had to launch a public call for tender.

Most institutions recognised that the costs of a cross border review were higher than the costs of a review by the national QA agency, which might deter institutions in seeking a cross-border EQA. The choice of a cross border EQA is taken because the (long term) benefits are considered to outweigh the higher costs of such a review.

2.3 Benefits of a Cross Border EQA

2.3.1 Internationalisation and Recognition of Degrees

Institutions felt that a review by a foreign international agency was a more genuine international experience, even if the national QA agency would include international peers on its panels. This is mainly based on the perception that the chosen agency has a broad pool of international peers and would be clearly seen as international by their stakeholders. Also, international review teams are considered to have a more developed understanding of the programme complexities and the institution’s experience with international students.

The most widely acknowledged impact of a cross-border EQA review was the strengthening of internationalisation policies and development of institutions collaboration with other foreign institutions. The internationalisation effect also extended to the academic formation practices (teaching and learning) of the institution and the development of mobility programmes.

The cross-border EQA reviews seem to also support the implementation of Bologna tools. For instance, to meet the requirements of a programme accreditation, the reviewed institution reported to have aligned its study programmes to the European Qualifications Framework (EQF), making the transition to the national qualification framework (which was later established) more easily.

In comparison with the national reviews, some of the interviewees from small higher education systems considered the international/foreign panel of experts to be more balanced in its judgments and review of their programmes.

2.3.2 Stakeholder Engagement

In preparing the review process, the institutions engaged in a wide consultation with their academic communities. This has enhanced the participation of stakeholders in the development plans of the institution and reviewed programmes. The cross-border EQA review is sometimes seen to have enhanced the voice of students within the institution as well, increasing the recognition of their input. One of the institutions reported that it decided to include a student representative within the self-evaluation committee for the first time.

Interviewees also reported an increased external acknowledgement of the institutions’ efforts to improve from the local community and enhanced collaborations with their alumni and social partners.

2.3.3 Development of QA Practices and Procedures

Following these reviews, institutions stated that they also developed or enhanced their internal QA system. This usually included development of the internal quality management system, integrated information system, quality system for curricular units, development of procedures for the monitoring of the quality and teaching, the launch of the first alumni survey, enhancement of the student feedback system, increased the number of regional partnerships etc.

The institutions appreciated the reviewers approach to quality as development and not punishment, setting a positive incentive in taking in the outcomes of the review and allowing the institution to take forward ideas/plans for change.

2.3.4 Strengthening the Institution’s Own Responsibility for Quality

Some of the institutions stated that they valued the opportunity to choose an agency that can promote more autonomy and underscore the responsibility of the institution for its own internal quality assurance processes.

The preparations have also fostered the self-reflection process, allowing the institution to identify possible problems (e.g. areas where universities’ internal processes were uncoordinated) and providing an impetus to challenge the status-quo (e.g. review out-dated procedures and practices). This has often helped the top-management to reconsider its current working methods but it has also provided a stronger basis or an external pressure to follow-up on the external recommendations.

2.3.5 A Positive Add-on to the Regular EQA

Cross-border EQA is also considered by one of the institutions as a positive add-on to the regular, obligatory external quality assurance exercise carried out by the national QAA. In cases where both the national and cross-border QAAs followed the European Standards and Guidelines, the review process was not significantly different. However, when it comes to differences, the standards and criteria used by the international/foreign reviewers are perceived to be less rigid, more wide-ranging and also more outcome-oriented. If the international review was done in addition to the obligatory EQA, it was often considered helpful in the preparation for the national accreditation.

2.4 Challenges of a Cross-Border EQA

2.4.1 Extensive Preparation Phase

Considering the novelty of these reviews for some institutions, the preparation phase was very demanding. The biggest challenge was the extensive documentation. The preparation for the review entailed a long and laborious work for the institutions, and in particular for the self-evaluation steering groups (weekly meetings, several months of collecting data, consulting stakeholders, writing and redrafting chapters etc.). The length of the preparation depended on the type and complexity of the review. Single programme accreditation required a few months up to half a year, while institutional evaluations or institutional audit extended to a whole year. The documentation sometimes entailed extra effort in the case of joint and double degree programmes due to the need to coordinate the review process with partnering countries.

2.4.2 Understanding of the National Educational System

Since most QAAs were carrying out a cross-border EQA for the first time in the reviewed country, the institution was tasked not only with the self-evaluation report, but also with providing additional materials and explanation about the national context, background and specificities. The institutions invested considerable time and effort in supporting these preparations (e.g. translating documents, clarifying different understandings related to the national QA terminology for domain specific area).

To ensure an efficient and accurate review within the higher education system, institutions stated that they found the foreign QAA’s preparation to be particularly helpful in improving the quality of the review (e.g. one QAA sent a list of open questions before the on-site visit inquiring about the specificities of the national legislation with regards to the reviewed programme). In some cases, a technical preparation meeting was set up with the QAA before the site visit to assist with the preparation process.

2.4.3 Legislative Context

The set up and coordination of the EQA of joint programmes presented particular challenges, as it entailed overcoming national legislative barriers, national quality assurance frameworks and specific institutional regulations (e.g. taking into account expiry of programme accreditation, equivalence for grading systems etc.).

The recommendations were in some cases difficult to follow due to the legislative framework (e.g. the development of new specialisations required a royal decree).

A changing legislative context also created difficulty for the institution and the reviewers, altering the focus of the external review from the status of the current governance to the potential changes that would take place in the governance structure.

2.4.4 Language Barriers

One additional challenge encountered by most institutions was related to the lack of available documentation in English (legislation, university strategy and documents etc.). The translation of the required documentation presented a number of difficulties due to the specific national terminology used in higher education and quality-related matters.

However, for some of the interviewed institutions, the need to translate documents into English was not a specific challenge, as they were anyway required for reviews by the national QA agency involving international experts.

In preparing for the site-visit interviews, some institutions stated that they found it difficult for some of their staff to meet and discuss with the international review panel in English. To overcome this challenge, an interpreter was often provided.

2.4.5 Complexity of the Review

The institutions sometimes experienced two different approaches to quality assurance that meant being assessed on similar standards that were differently defined and had a different weight for each of the reviewing agency. Internalising the recommendations while taking into account two different perspectives on the results of the EQA, was considered rather confusing.

The bilingual approach of some reviews entailed double sets of self evaluation reports. The interviewees admitted that a lengthier preparation was required due to this approach.

Quality assurance of joint programmes (leading to joint, double or multiple degrees) often involves a number of complexities. Whereas institutions that are subject to external quality assurance only at the institutional level (e.g. in a regular audit) tend to have fewer difficulties, institutions from countries with obligatory accreditation or evaluation at the level of study programmes often find themselves unable to have one joint programme evaluated/accredited in one joint procedure, due to different (and sometimes even conflicting) formal requirements in the countries involved.

3 Discussion on Findings

3.1 Why Turn to a Cross-Border EQA?

Generally, a review by a foreign QA agency requires more time and effort than a review by the national QA agency. In terms of challenges, higher education institutions consider the extra effort invested in explaining “their” system and context to a foreign agency and peers. The issue of costs can be an inhibiting factor where a review by the national QA agency is free of charge, while a review by another EQAR-registered agency might be at the higher education institution’s own expense.

The choice of a suitable agency usually involves considerable desk research undertaken by higher education institutions, for which they have used information provided on the EQAR website (e.g. expertise in different methodologies of external quality assurance, countries where agencies have worked) and on the agencies’ own websites. Institutions often find themselves in the situation that only a small number of the (currently) 32 EQAR-registered agencies¹³ would at all be suitable to undertake the review that is needed.

Nevertheless, institutions that choose to be reviewed by a foreign quality assurance agency see important advantages and opportunities that justify the extra effort required: receiving the best feedback, a review best suited to their own needs, or improved recognition of their qualifications.

3.2 Internationalisation as a Driver for EQA

Most often, higher education institutions turn to a cross-border EQA (at institutional or programme level) to increase their international reputation. Institutions felt that a review by a foreign/international agency was a more genuine international experience, even if the national QA agency would include international peers on its panels. The impact of such an international accreditation is recognized as having a stronger connection with the labour market, benefiting students when they graduate, in terms of finding employment or continuing their academic career abroad more easily.

Institutions (especially small or regional ones) consider the review to strengthen their international profile and international partnerships. Related to this, the reputation and image of the agency chosen plays a certain role in the selection process.

Internationalisation of higher education institution has become a driver for cross-border quality assurance processes. The current developments suggest a new approach in the role and purposes fulfilled by quality assurance, a role that might be part of the discourse on the “modernisation” of quality assurance in Europe.

3.3 ESG as a Proxy for Trust Within EHEA

HEIs interest towards cross-border EQA might also prompt the traditional QA establishments to reconsider their current trust arrangements, where the national seal of an accreditation guarantees funding of the higher education and the recognition of qualifications for the labour market. Stensaker and Maasen¹⁴ observe that the bilateral trust relationship between state authorities and higher education institutions has been shifting towards a multilateral type of trust relationship. With the implementation of the ESGs, a foundation was created for increased trust in the quality assurance of higher education in each of the EHEA countries. Similarly, according to the Map ESG project, the ESGs are perceived to be a useful instrument that should maintain generic standards to respond to the many purposes of higher education.

Confidence in external QA across the continent is not an end in itself, but it can serve as a proxy to increase mutual trust in higher education institution and their study programmes.

However, the examples of country openness to cross-border external quality assurance seem to be few and far between. The desk research of the project showed that most countries are reluctant to devolve any type of responsibility (and thus trust) on external QA, while the responses to the QAA survey carried out by EQAR show that cross-border EQA is taking place almost in all EHEA member countries (39 out of 47).¹⁵ In these closed system, the cross-border EQA leads to a duplication of efforts. It remains to see whether the recognition of higher education institutions commitment to be responsible for their own quality assurance and the use of ESGs as a “common denominator” for quality assurance agencies (whereas EQAR is used as a proxy for ESG compliance) would provide a cross-country foundation for the development of trust within EHEA.

4 Acronyms and Glossary

“E4 Group”	The E4 group refers to four European stakeholders in higher education: The European Association for Quality Assurance in Higher Education (ENQA), the European Students’ Union (ESU), the European University Association (EUA) and the European Association of Institutions in Higher Education (EURASHE)
European Higher Education Area (EHEA)	Launched along with the Bologna Process’ decade anniversary during the Budapest-Vienna Ministerial Conference in March 2010 by 47 states, together with the European Commission, and the consultative members, namely the Council of Europe, UNESCO, EUA, ESU, EURASHE, ENQA, Education International and BUSINESSEUROPE. EHEA was meant to ensure more comparable, compatible and coherent systems of higher education in Europe
European Quality Assurance Register for Higher Education (EQAR)	The Register aims at increasing transparency of quality assurance in higher education across Europe. It has been founded in 2008 by the European Association for Quality Assurance in Higher Education (ENQA), the European Students’ Union (ESU), the European University Association and the European Association of Institutions in Higher Education (EURASHE). EQAR publishes and manages a list of quality assurance agencies that substantially comply with the European Standards and Guidelines for Quality Assurance (ESG) to provide clear and reliable information on quality assurance agencies operating in EHEA
European Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)	European Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) are an agreed set of standards and guidelines for quality assurance in European higher education. They were developed by the “E4 Group” and adopted by the ministers in Bergen in (2005)
External Quality Assurance (EQA)	External quality assurance refers to the process of evaluation or audit of a higher education programme or institution undertaken by a specialised body outside the institution. Typically, the body may be a quality assurance or accreditation agency, or an ad hoc panel of experts and peers constituted by the responsible Ministry. The evaluation will involve the collection of data, information and evidence for assessment against agreed standards
Higher education institution (HEI)	Officially recognised public and private higher education institutions that offer programmes at ISCED levels 5 and 6 and are provided for under the legislation of the country concerned
Joint programmes	Programmes that are developed and implemented jointly by several institutions in different countries
Joint degree	A higher education qualification issued jointly by at least two or more higher education institutions or jointly by one or more higher education institutions and other awarding bodies
Quality assurance (QA)	An all-embracing term referring to processes of evaluating (assessing, monitoring, guaranteeing, maintaining and improving) the quality of a higher education system, institution or programme
Quality assurance agency (QAA)	A body established by public authorities with responsibility for external quality assurance. Agencies are intended to play a strong role in ensuring accountability of higher education institutions and may have specific objectives and developmental roles regarding enhancing quality
Quality culture	Quality culture is a set of group values that guide how improvements are made to everyday working practices and consequent outputs
National Qualifications Frameworks (NQF)	National qualifications frameworks describe qualifications in terms of level, workload, learning outcomes and profile. They relate qualifications and other learning achievements in higher education coherently and are internationally understood
Polytechnic institute	A technical higher education establishment offering instruction in many industrial arts and applied sciences
Regulation	A law, decree or any other officially binding document, issued by the top-level education authorities