Qualifications, Skills, and Workforce Effectiveness: The Relevance of the OECD’s Survey of Adult Skills to Asia

Abstract

The OECD Survey of Adult Skills combines a direct assessment of key cognitive skills with the collection of information on the use of cognitive skills and other generic skills in the workplace (and other contexts) and information concerning the factors associated with the development, maintenance and loss of skills as well as the information on the outcomes of these skills. Results from the first round of the assessment which involved 24 countries will be released in October 2013. This chapter provides an overview of the main features of survey and describes how the data from the survey is relevant to education and training and labour market policy in the countries of the Asia-Pacific region.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

Improving the process of the development, maintenance and use of skills is increasingly recognised as a core element of the policy package necessary to support sustainable long-term growth and employment creation and contribute to a fairer distribution of income and opportunities. The recently released OECD Skills Strategy (OECD 2012b) identifies three key areas for action by governments.

• Developing relevant skills: Ensuring that the supply of skills is sufficient in both quantity and quality to meet current and emerging needs is a central goal of skills policies. Supply can be ensured by developing the right mix of skills through education and training and influencing the flow of skills by attracting and retaining talent. Importantly, supply is not only responsive to demand; it can also have an important influence on demand.

• Activating skills: People may have skills, but for a variety of reasons may decide not to offer them to the labour market. Individuals withdraw from the labour force for a range of reasons, including personal preferences, life circumstances or the lack of financial incentives to work. Encouraging inactive individuals to enter or re-enter the labour force can increase the skills base of an economy. This requires identifying inactive individuals, possibly retraining them, ensuring that the benefit system offers them financial incentives to enter or return to the labour market and removing demand-side barriers to hiring.

• Putting skills to effective use: Investment in skills development by individuals and governments needs to be accompanied by policies that ensure that these skills are used effectively. Moreover, the match between the skills demanded in a job and the skills of the person doing the job has an impact on further skills development: unused skills tend to atrophy, while new skills are, to a large extent, developed informally, often through work experience.

A focus of the work of the OECD in supporting governments in dealing with issues of skill formation is that of the enhancement of the evidence base. Areas in which there are major information gaps include measures of skills endowment of the population which go beyond information about educational qualifications, the processes by which skills are gained, maintained and lost over the lifecycle and the effectiveness of the utilisation of skills. The OECD Survey of Adult Skills¹ represents an initiative of the OECD that will provide governments, policy makers and researchers with some of the essential information needed to review and develop policies and interventions relating to the development, maintenance and use of skills. The survey has as its objectives to:

• Identify and measure differences between individuals and across countries in key competencies believed to both underlie personal success and respond to labour market requirements.

• Assess the impact of competencies on a range of economic and social outcomes.

• Assess the performance of education and training systems, workplace practices as well as labour market policies, in generating competencies at the levels required by social and economic demands.

• Help identify policy levers to reduce ‘deficiencies’ in key competencies.

To this end, the OECD Survey of Adult Skills combines a direct assessment of cognitive skills that provide a foundation for effective participation in the societies and economies of the twenty-first century, the collection of information on the use of cognitive skills and other generic skills in the workplace (and other contexts) and information concerning the factors associated with the development, maintenance and loss of skills as well as the information on the outcomes of skills.

Currently, the OECD Survey of Adult Skills is being implemented by over 30 countries in Europe, the Americas and the Asia/Pacific region (see Annex A for the full list). Results from the first round of the assessment involving 24 countries which started in 2008 will be released in October 2013. A second round of the assessment began in late 2011 with results being available in mid-2016.

This chapter provides an overview of the main features of the OECD Survey of Adult Skills and describes how the data provided by the assessment is relevant to many of the issues concerning the development and operation of education and training systems and the relationship of these systems to the labour market which preoccupy policy makers and governments in the countries of the Asia-Pacific region.² Some additional information about participation in the survey and its operational aspects is presented in Annex A.

What Is the OECD Survey of Adult Skills?

The OECD Survey of Adult Skills involves the administration of a questionnaire and self-completed assessment of key foundation skills to nationally representative samples of adults (16–65 year olds).

As implemented in the first cycle of the programme, the survey contains the following components:

• A direct assessment of the proficiency of adults (16–65 year olds) in the domains of literacy, numeracy and problem solving in technology-rich environments (TRE);

• A module of questions regarding the use of a range of generic skills at work;

• A background questionnaire which contains questions regarding demographic characteristics, social and linguistic background, educational attainment and participation, training participation, employment status and income, use of ICTs and literacy and numeracy practices.

The Direct Assessment

The direct assessment component of the OECD Survey of Adult Skills evaluates the skills of adults in three domains, those of literacy, numeracy and problem solving in TRE. Before outlining the way in which each of these domains is conceptualised in the OECD Survey of Adult Skills, the question of why these particular skills are assessed is addressed and some important general features of the way in which these skills are conceptualised in the assessment are discussed.

Literacy, numeracy and problem solving in TRE are assessed in the OECD Survey of Adult Skills as they are considered to constitute ‘key’ competencies important for achieving a successful life and a well-functioning society (Rychen and Salganik 2003). They are ‘key’, firstly, in the sense that they provide a foundation for the development of other higher-order cognitive skills as well as constituting a precondition for gaining access to and understanding of specific domains of knowledge. Being able to read, manage mathematical and numerical information and solve problems are essential for the development of higher-order analytic and communication skills, for example. Secondly, they are relevant in an extremely broad range of life contexts, from education through work to everyday life. In information-rich societies in which information in text format (whether print-based or digital) is ubiquitous, a capacity to effectively read and react appropriately to text-based information is essential, whether in terms of understanding the user information on a packet of medicine or responding appropriately to a memo from a colleague or superior at work. Similarly, numerical and problem-solving skills are essential in most areas of life (e.g. to undertake monetary transactions, to plan a holiday).

Two features of the approach to the assessment of literacy, numeracy and problem solving in the OECD Survey of Adult Skills are important to mention before looking at the definition of the individual skills domains.

The skills measured in the OECD Survey of Adult Skills are viewed as functional skills. Reading, numerate behaviour and solving problems are conceived as purposive activities in which people engage, not as ends in themselves but to achieve various goals in a variety of real world contexts (e.g. in the workplace, in educational settings or at home). From this it follows that the focus of the OECD Survey of Adult Skills is not on the measurement of skills associated with the ability to read or understand mathematical concepts, for example, but on the application of these skills for specific purposes in specific contexts.

In addition, the OECD Survey of Adult Skills defines the skills assessed in terms of a continuum of proficiency. The skills measured are seen as something a person has more or less of rather than as something one either has or does not have. In other words, there is no threshold on the skill continuum which separates those who have a particular skill from those who do not (e.g. distinguishes illiterates from literates). Individuals with higher levels of proficiency have greater chances of undertaking more complex information processing tasks than those with lesser proficiency. At the highest levels of proficiency, individuals are able to successfully complete tasks that demand the use of complex cognitive operations.

It should also be noted that the objective of the OECD Survey of Adult Skills is to gain an accurate picture of the entire spectrum of proficiency of the adult population. In other words, it seeks to provide information regarding adults with high levels of skills as well as those with low skills. While skills such as literacy and numeracy are sometimes presented as ‘basic’ skills (in the sense of providing a ‘foundation’ for the development of other competencies), the OECD Survey of Adult Skills is not an evaluation which focuses on assessing the achievement of a basic level of skill.

The way in which literacy and numeracy and problem solving in TRE are defined in the OECD Survey of Adult Skills is briefly described below. More detailed information on the definition of these constructs is contained in Annex B.

Literacy

Literacy is defined in the OECD Survey of Adult Skills as understanding, evaluating, using and engaging with written texts to participate in society, to achieve one’s goals, and to develop one’s knowledge and potential (OECD 2012a: 19). ‘Literacy’ in the OECD Survey of Adult Skills does not include the ability to write or to produce text, skills commonly falling within the definition of literacy.³ However, at the same time, ‘literacy’ is a broader construct than ‘reading’, narrowly understood as a set of strategies for decoding written text. It is intended to encompass the range of cognitive strategies (including decoding) that adults must bring into play to respond appropriately to a variety of texts of different formats and types in the range of situations or contexts in which they read. A unique feature of the assessment of literacy in the OECD Survey of Adult Skills is that it assesses adults’ ability to read digital texts (e.g. texts containing hypertext and navigation features such as scrolling or clicking on links) as well as traditional print-based texts.

To provide more detailed information about adults with poor literacy, the assessment of literacy in the OECD Survey of Adult Skills is complemented by a test of ‘reading component’ skills. Reading components represent the basic set of decoding skills which provide necessary preconditions for gaining meaning from written text – knowledge of vocabulary, ability to process meaning at the level of the sentence processing and fluency in the reading of passages of text.

Numeracy

Numeracy is defined in the OECD Survey of Adult Skills as the ability to access, use, interpret and communicate mathematical information and ideas, in order to engage in and manage the mathematical demands of a range of situations in adult life (OECD 2012a: 34). Numeracy is further specified through the definition of ‘numerate behaviour’ which involves managing a situation or solving a problem in a real context by responding to mathematical information and content represented in multiple ways.

It is recognised that literacy skills such as reading and writing constitute an enabling factor for numerate behaviour and that when mathematical representations involve text, performance on numeracy tasks is, in part, dependent on the ability to read and understand text. However, numeracy in the OECD Survey of Adult Skills involves more than applying arithmetical skills to information embedded in text. In particular, numeracy relates to a wide range of skills and knowledge (not just arithmetic knowledge and computation), a range of responses (which may involve more than numbers) and responses to a range of representations (not just numbers in texts).

Problem Solving

In the OECD Survey of Adult Skills, problem solving in technology-rich environments is defined as using digital technology, communication tools and networks to acquire and evaluate information, communicate with others and perform practical tasks. The first wave of the OECD Survey of Adult Skills focuses on the abilities to solve problems for personal, work and civic purposes by setting up appropriate goals and plans, and accessing and making use of information through computers and computer networks (OECD 2012a: 47).

The PS-TRE (problem solving in technology-rich environments) domain of the OECD Survey of Adult Skills covers the specific class of problems people deal with when using ICT. These problems share the following characteristics:

• The existence of the problem is primarily a consequence of the availability of new technologies.

• The solution to the problem requires the use of computer-based artefacts (applications, representational formats, computational procedures).

• The problems are related to the handling and maintenance of technology-rich environments themselves (e.g. how to operate a computer, how to fix a settings problem, how to use the Internet browser in a technical sense).

Problem solving in TRE represents a domain of competence which involves the intersection of the set of skills that are sometimes described as ‘computer literacy’ (i.e. the capacity to use ICT tools and applications) and the cognitive skills required to solve problems. Some knowledge of how to use basic ICT input devices (e.g. use of a keyboard and mouse and screen displays), file management tools, applications (word processing, email) and graphic interfaces is essential in order to be able to undertake assessment tasks. However, the objective is not to test the use of ICT tools and applications in isolation, but rather to assess the capacity of adults to use these tools to access, process, evaluate and analyse information effectively.

Other Information on Skills

Literacy, numeracy and problem solving in TRE constitute a subset of the skills and competences that are demanded in the labour market and/or mediate access to resources and services more generally in society. Along with specific technical and professional skills, other generic skills such as communication, interaction skills (such as the capacity to relate to others and work cooperatively), skills related to learning and the transmission of knowledge as well as physical skills are valued to a greater or lesser extent in the labour market. Most of these skills are difficult, if not impossible, to assess directly, either in an international comparative context or using surveys methods.⁴ The OECD Survey of Adult Skills collects a considerable amount of information on the skills possessed and used by adults in addition to the measures of proficiency in literacy, numeracy and problem solving in TRE.

Qualifications and Work Experience

Educational qualifications and work experience are commonly used proxies for individuals’ skill endowments. The OECD Survey of Adult Skills collects information on respondents’ highest level of educational attainment as well as information regarding the duration of work experience and mobility. This is complemented with information on their perceptions regarding the educational qualifications and work experience they believe are normally necessary to get the job they currently occupy as well as the qualifications needed to perform this job satisfactorily.

Use of Skills at Work

Information is collected from respondents regarding four broad categories of generic work skills: cognitive skills, interaction and social skills, physical skills and learning skills.⁵ Cognitive skills encompass reading, writing, mathematics and the use of ICTs. Interaction and social skills cover collaboration and cooperation, planning the work and time of one’s self and others, communication and negotiation, and customer contact (e.g. selling products and services and advising). Physical skills involve the use of gross and fine motor skills. Learning skills cover activities such as the instruction of others, learning (formally or informally) and keeping up to date with developments in one’s field of professional activity.

The approach used in the OECD Survey of Adult Skills owes much to the Jobs Requirements Approach (JRA) pioneered in the UK Skills Survey (Felstead et al. 2007). The JRA method consists of asking individuals about the importance of different types of tasks performed at work and subsequently inferring the types of skills that are required from their answers. By focusing on job tasks, this approach is considered to provide a more objective description of these skills than an approach relying on subjective self-assessments by individuals of the type and level of skills they possess.

Respondents are also asked about the extent to which they believe that their skills (considered globally) match the requirements of the job in which they are currently working.

Work-Related Training

Given the importance of work-related training as a potential source of skills and as an element of a strategy for the maintenance and upgrading of workforce skills, information is collected on participation by respondents in training of both a formal and informal nature over the 12 months prior to the interview.

Personal Characteristics, Background and Outcomes

The OECD Survey of Adult Skills background questionnaire includes a range of information regarding the factors which influence the development and maintenance of skills such as education, social background, engagement with literacy and numeracy and ICTs, and language, as well as information on outcomes which may be related to skills. Information is collected on the current activity, employment status and income of respondents. In terms of noneconomic outcomes, the OECD Survey of Adult Skills includes questions on health status, volunteering, political efficacy and social trust.

What Can the OECD Survey of Adult Skills Tell Us?

As can be seen from the description of its content, the OECD Survey of Adult Skills will provide researchers and policy makers with comprehensive information regarding the ‘supply’ of key cognitive skills, the use of cognitive and other generic skills at work, engagement with literacy, numeracy and ICTs outside work as well as background factors, education and labour market and other outcomes. The information provided will allow analysis of a range of issues concerning the level and distribution of these skills in the population, the factors that contribute to the development and maintenance of these skills and their potential influence on a range of outcomes.

An idea of the range of information that will be available from the OECD Survey of Adult Skills and its relevance to policy concerns can be gained from the literature using data from the two previous international adult literacy and skills surveys, the International Adult Literacy Survey (IALS) and the Adult Literacy and Life Skills Survey (ALL).⁶ A survey of this literature which covers themes such as the distribution of proficiency, the impact of literacy on labour market outcomes, the factors affecting the development of literacy, skills match and mismatch is available in Thorn (2009). In this section, the contribution of the OECD Survey of Adult Skills in three areas is discussed.

The Level and Distribution of Proficiency in Key Skills

Illiteracy and poor literacy (as well as poor numeracy) represent a considerable hurdle to the full and fulfilling participation of individuals in the economy and society. Other things being equal, low performance in literacy and numeracy is associated with higher chances of unemployment, lower earnings, lower participation in continuing education and training and poor health (OECD and Statistics Canada 2000, 2011; Statistics Canada and OECD 2005). At the macro-level, poor literacy and low levels of cognitive performance in the population more generally may constrain the potential for growth.

For many countries in the Asia-Pacific region, illiteracy and poor literacy among adults and young people represent a reality and challenge for governments. Literacy rates for adults remain low particularly in South and West Asia where Bangladesh, Bhutan, India, Iran, Nepal and Pakistan have literacy rates for adults of between of 53 and 82% as well as in Cambodia, the Lao Republic and Papua New Guinea. Literacy rates are in most cases lower for women than men (UNESCO 2011: table 2). Literacy rates for youths (15–24 year olds) are higher than for their older compatriots. However, the evidence from PISA suggests that for a number of countries in the Asian region, even where illiteracy per se has been eradicated, poor literacy will remain a problem for some years to come. For example, in Azerbaijan, Indonesia, Kazakhstan, Kyrgyz Republic and Thailand, countries in which youth have literacy rates of around 100%, there are significant proportions of 15-year-old school students performing at the lowest levels on the PISA reading literacy scale (OECD 2010b).

Even in countries with developed economies, negligible rates of adult illiteracy and high levels of performance in PISA, poor literacy among adults constitutes a potential policy concern. IALS revealed that a significant proportion of adults in developed countries have low levels of literacy and numeracy and may have difficulty in responding to many of the literacy and numeracy demands they face in their work and everyday life (OECD and Statistics Canada 2000), findings that were confirmed by the ALL survey (Statistics Canada and OECD 2005; OECD and Statistics Canada 2011). For example, in both Australia and New Zealand,⁷ around 40% of adults were found to perform at the two lowest levels of literacy proficiency⁸ in IALS and ALL (ABS 2008 and Satherley et al. 2008). Improvement of the levels of literacy and numeracy among adults was identified as one of the major areas in which action was needed in a national Workforce Development Strategy developed recently in Australia (Skills Australia 2010). It is likely that these types of findings would be replicated to a greater or lesser degree in other countries in the Asia-Pacific region with high per capita GDP and/or high performing school systems.

The OECD Survey of Adult Skills will offer policy makers and researchers a snapshot of the level and distribution or variability of proficiency in key foundation skills in the adult population. It will be possible to look at the performance of key subgroups of the population and, depending on the sample size, by geographic region. The estimates of proficiency will be able to be compared with those of other participating countries.

The information provided by the OECD Survey of Adult Skills will go well beyond knowing the proportions of the population above and below a threshold defining a desirable minimum level of proficiency (e.g. literate or illiterate). It will be possible to identify the proportion of the population, for example, who are fully equipped to handle complex text-based information processing tasks. At the other end of the scale, it will be possible to identify the proportion of adults who possess basic skills but struggle with coping with many of the reading tasks required to function effectively in modern society.

The OECD Survey of Adult Skills will offer the most comprehensive set of information available on the capacity of the adult population to access and manage information and display higher-order cognitive skills in digital environments. This encompasses not only the reading of digital texts but also the capacity to integrate information from multiple sources and appropriately use digital applications and tools to solve the problems. The OECD Survey of Adult Skills will thus allow policy makers to have a far deeper understanding of the digital competence of the population than is available from usage statistics.

In the case of the poorest readers, the reading components assessment will provide a wealth of information on their strengths and weaknesses, for example, in terms of basic vocabulary, basic comprehension and fluency. The OECD Survey of Adult Skills measures will aid policy makers in identifying and understanding the extent and dimensions of illiteracy and poor literacy. Having a detailed picture of the spectrum of ability will assist policy makers target and design programmes, not only to eradicate illiteracy but also to improve the skills of adults with some basic literacy. The latter, in particular, is an area which will become increasingly important as a component of a comprehensive strategy to develop the human resources necessary to underpin continued economic growth and development and raise productivity.

The Use of Generic Skills at Work

There is a broad consensus regarding the importance of generic skills in the modern workplace and also regarding the importance of ensuring that education and training systems develop these skills among young people in preparation for entry to the labour market. Shifts in employment towards services (which place a premium on interaction with clients), rapid technological change, especially computerisation (which has been argued to have led to a shift towards employment in occupations involving complex communication and expert thinking),⁹ changes in work organisation emphasising teamwork, flexible production and multi-skilling and increasing competition and rapid change in consumer tastes have all been argued to increase demand for workers with broad-based thinking and interaction skills as well as technical competence and knowledge.

In this context, there have been a number of exercises that have sought to identify and group the key generic competencies needed in the modern workplace and that should be developed alongside occupational specific skills, particularly in preparing young people for entry to the labour market (see, for example, SCANS 1991; Mayer 1992; DEEWR 2012). Most frameworks identify similar clusters of competencies: basic or foundation skills such as literacy and numeracy, higher-level cognitive skills such as problem solving and analytical reasoning, interpersonal skills such as communication, working in teams and ability to negotiate, ability to use technology particularly ICTs and learning skills such as knowing how to learn. These types of findings seem generalisable. A recent study of graduate employability in three Asian countries¹⁰ found that employers placed considerable emphasis on the possession of soft skills by university level graduates (UNESCO Bangkok 2012). The World Bank (2010) has also recently emphasised the importance of behavioural skills such as teamwork, diligence, creativity and entrepreneurship in its STEP skills development framework.

Despite the importance attributed to generic skills, there is little hard evidence regarding either the demand for or the supply of such skills, especially from a comparative perspective. The OECD Survey of Adult Skills will fill a considerable void in this area. On the supply side, the OECD Survey of Adult Skills will provide information regarding the proficiency of the workforce in the domains of literacy, numeracy and problem solving in TRE. While these cognitive skills represent a subset of the bundle of generic skills that are valued on the labour market, they constitute key indicators of the ability to manage the information processing tasks that are of growing importance in a knowledge-based economy such as ‘complex communication’ and ‘expert thinking’ (see Levy 2010). On the demand side, the OECD Survey of Adult Skills will offer a comprehensive picture of the incidence and intensity of use of a range of generic skills at work for the countries participating in the study and provide a baseline for looking at change in the use of these skills over time. It will also allow analysis of the relationship between the generic skill content of jobs and the occupational and industry structure of employment in different countries as well as the impact of the use of generic skills on earnings and productivity.

Matching the Supply of and Demand for Skills

Ensuring a better match between the supply of and the demand for skills has emerged as a key issue for policy across the world. Concerns include the existence of persistent skill shortages, high rates of unemployment among particular population groups such as youth (which sometimes coexist with labour shortages), the oversupply of individuals with certain skills and types of qualification, and the failure of employers to make the most of the skills their employees possess. Optimising the use of skills has been identified as a central theme of the OECD Skills Strategy (OECD 2011b) and by the World Bank in its framework for skills development (STEP) (World Bank 2010).

In the Asia-Pacific region, there is evidence of the existence of mismatches of various types in many countries. On the one hand, a recent study by the Manpower Group (2011) found that employers in the Asia-Pacific region¹¹ had more difficulty than employers elsewhere in filling positions and the proportion of employers reporting such difficulties had increased since 2008. Skills shortages, particularly of higher-level vocational skills, are the focus of policy action in a number of countries such as India, the People's Republic of China and the Republic of Korea. On the other side, an oversupply of graduates with higher education qualifications is seen as a problem in the Republic of Korea. Youth unemployment is high in economies such as Indonesia, the Republic of Korea, Taipei,China and Hong Kong, China. (ILO 2012). Making better use of the skills of the workforce has been identified as a central element of a proposed National Workforce Development Strategy in Australia (Skills Australia 2010).

A useful framework for examining the question of matching the supply of, and demand for, skills is provided by Fig. 8.1. In addition to the phenomena of shortages (where job vacancies remain unfilled for lack of supply of suitable applicants) and unemployment (when individuals cannot find work due to lack of demand for their skills), there are also phenomena of mismatch in which workers are employed in jobs which either do not make full use of the skills that they possess (underemployment or over-skilling) or alternatively require skills which they do not have to the required level (skill gaps, under-qualification or under-skilling). The latter phenomena may arise for a number of reasons. These include individual preferences (e.g. for jobs that provide flexibility in hours of work), labour market rigidities which provide disincentives for mobility, inadequate labour market information, over- or undersupply of certain types of labour, technological change and poor human resource practices in firms (poor recruitment processes, under-provision of training or poor job design and task allocation).

Fig. 8.1

Components of labour supply and demand (Source: UKCES 2010: 40)

Mismatch in the form of skills gaps and underemployment has become a topic of increasing interest to researchers and policy makers over recent years (CEDEFOP 2010; Desjardins and Rubenson 2011; OECD 2011a). In fact, skills gaps have been argued to be a more serious problem than shortages (UKCES 2009).

In looking at these issues, it is important to distinguish between qualifications mismatch and skills mismatch. Qualifications mismatch refers to a discrepancy between the highest qualification held by a worker and the qualification required by his/her job. Skills mismatch refers to a discrepancy between the skills – both specific and general – possessed by a worker and the skills required by his/her job (OECD 2011a). Skills mismatch has been much less studied than qualification mismatch (largely because of data availability), and when it has been, it has been done so largely on the basis of subjective information.¹²

The OECD Survey of Adult Skills will provide one of the most comprehensive sources of information available for the analysis of mismatch both in terms of the range of information it collects and the number of country observations. At the same time as providing data on subjective appreciations (self-reports) of over- or under-qualification and over- and under-skilling, it will represent one of the few available sources of information which permits the exploration of skills mismatch using ‘objective’ measures of the skills of workers in the form of proficiency in literacy, numeracy and problem solving in TRE.¹³ The combination of measures of proficiency with information on the use of reading, numerical skills and computer use at work will provide a powerful tool for looking at mismatches between skills and job requirements for these key cognitive skills. In addition, the OECD Survey of Adult Skills contains information that allows examination of the impact of mismatch on factors such as wages and job satisfaction and access to training as well as the relationship between mismatch and factors such as age, gender, educational attainment and immigration status.

A recent OECD working paper offers an example of what type of analyses will be able to be accomplished using the OECD Survey of Adult Skills. Desjardins and Rubenson (2011) use data from ALL to examine the extent of match between the literacy and numeracy proficiency of workers and the literacy and numeracy tasks they undertake at work and the impact of mismatch on earnings and access to training.

Literacy and numeracy match and mismatch are defined by Desjardins and Rubenson (2011) on the basis of reported engagement in literacy-related tasks at work and direct measures of the literacy skills of workers. Persons with reading engagement scores below the median are assigned to the ‘low- to medium-low-engagement’ category (low-skill job), and those scoring above to the ‘medium-high- to high-engagement’ category (high-skill job). Similarly, persons scoring at skills Levels 1 and 2 on the prose literacy scale are assigned to the ‘low-skills’ category, and those scoring at Levels 3 and 4/5 assigned to the ‘high-skills’ category. Four match and mismatch categories are identified:

• Low-skill, low to medium-low engagement = LOW-SKILL MATCH

• Medium- to high-skill, medium-high to high engagement = HIGH-SKILL MATCH

• Low-skill, medium-high to high engagement = DEFICIT MISMATCH

• Medium- to high-skill, low to medium-low engagement = SURPLUS MISMATCH

In the nine¹⁴ countries included in the analysis, the proportion of literacy and numeracy matches is about 59–69% and 48–65%, respectively, depending on the country. Literacy and numeracy mismatch is a widespread phenomenon with 31–41% and 35–52%, respectively, of workers having skills that do not match the requirements of their job, depending on the country. Skill deficits are apparent in every country with 9–29% (literacy) and 6–20% (numeracy) of the workforce falling into this category, depending on the country. The level of surplus of literacy and numeracy skills also varies substantially by country, ranging from 12–32% to 17–46%, respectively.

In addition to examining the incidence of literacy and numeracy match/mismatch, Desjardins and Rubenson look at the distribution of mismatch by age, gender and immigration status and the impact of mismatch on earnings and access to training. Literacy surpluses are found to be greater among younger (16–35 year olds) workers, women and immigrants. In terms of earnings, the characteristics of jobs relating to literacy use are found to have a substantial effect on earnings. Being in a situation of high- skills match and a situation of deficit mismatch are associated with earnings premiums whereas being in a situation of surplus mismatch is associated with a small earnings penalty relative to individuals in situations of low-skills match. Access to employer supported training is also found to be closely linked to job requirements. As in the case of earnings, workers in jobs which entail high-literacy requirements have higher probabilities of access to training than those in jobs with low-literacy requirements irrespective of their level of literacy proficiency.

These findings give weight to approaches which emphasise the need for action on the demand side of the labour market (e.g. in terms of job design, human resource management) as well as on the supply side (e.g. influencing the output from the education system) in responding to the existence of skills mismatch.

The release of the OECD Survey of Adult Skills data will provide an opportunity to deepen analysis of the phenomenon of mismatch. The data available from the OECD Survey of Adult Skills will allow issues of qualification mismatch to be examined in conjunction with skills mismatch as well as to combine examination of self-reported mismatch with objective measures. In addition, analysis will be able to cover a far wider range of countries than is possible in ALL, for example.

Conclusion

Results and data from the first round of the OECD Survey of Adult Skills will be released in October 2013 with further results for additional countries becoming available in 2016. As a result, this chapter has only been able to describe the content of the OECD Survey of Adult Skills and give some idea of the types of questions that will enable policy makers and researchers to explore and, hopefully, answer. A number of countries in the Asia-Pacific region are participating in the OECD Survey of Adult Skills and will be able to benefit directly from the study. However, the information from the OECD Survey of Adult Skills will be relevant to other countries in this region. The OECD Survey of Adult Skills database will constitute one of the most comprehensive sources of information for analysing issues of the distribution of key cognitive skills as well the factors that are linked to their development and their relationship to economic and social success.

Appendices

Annex A: How Is the OECD Survey of Adult Skills Conducted?

The OECD Survey of Adult Skills is delivered as a computer-based assessment. The test application (including the background questionnaire and the direct assessment) is loaded on a laptop computer. The background questionnaire is administered by the interviewer. The majority of respondents complete the assessment on the laptop computer under the supervision of the interviewer. Respondents who have little or no familiarity with computers complete a pencil-and-paper version of the assessment that tests skills in the domains of literacy and numeracy only. All respondents, irrespective of whether they take the assessment in the computer or pencil-and-paper format, first take a ‘core test’ to assess their capacity to undertake the full assessment. Those who ‘fail’ the core test are directed to the assessment of reading components. Those who ‘pass’ the core test proceed to the full assessment.

An enhanced pencil-and-paper version of the assessment is currently being developed for use in countries in which the incidence of familiarity with computers is low. This version of the assessment (which covers literacy, reading components and numeracy only) will be administered in Indonesia as part of the second round of the OECD Survey of Adult Skills (see below).

A minimum of 4,500–5,000 adults (16–65 year olds) is tested in each participating country.

Countries implementing the OECD Survey of Adult Skills are required to meet certain technical standards covering all aspects of the study such as sampling, translation, interviewer training, response rates, contact with respondents, scoring and coding and database preparation. Compliance with these standards is monitored. Only data which meets the necessary quality standards is released.

Two ‘rounds’ of the OECD Survey of Adult Skills are currently being conducted. The first, which started in 2008 and will be finalised in October 2013 with the release of a comparative report and public use data set, involves 24 countries. The second round started in 2012 and will report in 2016 and will involve 10 countries.

Countries participating in the OECD Survey of Adult Skills

Round 1	Round 2
Australia	Chile
Austria	Greece
Belgium	Indonesia
Canada	Israel
Cyprus	Lithuania
Czech Republic	New Zealand
Denmark	Portugal
Estonia	Singapore
Germany	Slovenia
Finland	Turkey
France
Ireland
Italy
Japan
Republic of Korea
Netherlands
Norway
Poland
Russian Federation
Slovak Republic
Spain
Sweden
United Kingdom
United States

Annex B: Defining Literacy, Numeracy and Problem Solving in Technology-Rich Environments

Literacy

The domain of literacy is structured in terms of three main dimensions, those of texts, contexts and cognitive operations. The texts that adults are required to read are categorised in the OECD Survey of Adult Skills along three main axes – those of ‘medium’, ‘format’ and ‘type’. In terms of medium, texts are classified as either ‘print-based’ or ‘digital’ texts. Format involves the distinction between continuous texts (in which information is presented in the form of sentences and paragraphs) and noncontinuous texts (in which information is presented in a matrix format, such as tables, charts and forms). Type covers the rhetorical stance of the text, for example, narration, description, instruction and record.

Four broad categories of ‘context’ are defined in the OECD Survey of Adult Skills: ‘work and occupation’, ‘personal’ (which covers home and family, health and safety, consumer economics, and leisure and recreation), ‘community and citizenship’, and ‘education and training’.

Three broad cognitive strategies identified as essential for achieving a full understanding of texts are identified – those of accessing and identifying (locating information in texts), integrating and interpreting (relating different parts of a text to one another) and evaluating and reflecting (linking the content of a text to other, extratextual knowledge and content).

Numeracy

The domain of numeracy is defined in terms of four dimensions: (1) context; (2) response; (3) mathematical content, information and ideas; and (4) representations.

Three categories of context are identified: (1) work, (2) society, and (3) further learning.

Four categories of responses to mathematical information content and representations are identified: (1) identify, locate or access; (2) act upon, use; (3) order, count, estimate, compute, measure, model; (4) interpret, evaluate/analyse, and communicate.

Four areas of content are defined: (1) quantity and number; (2) dimension and shape; (3) pattern, relationships, change; and (4) data and chance.

Six types of representation of mathematical content are identified: (1) objects and pictures; (2) numbers and mathematical symbols; (3) formulae; (4) diagrams and maps, graphs, tables; (5) texts; and (6) technology-based displays.

Numerate behaviour is founded on the activation of several enabling factors and processes: mathematical knowledge and conceptual understanding; adaptive reasoning and mathematical problem-solving skills; literacy skills; beliefs and attitudes; numeracy-related practices and experience; and context/world knowledge.

Problem Solving in TRE

Problem solving in TRE is conceived in terms of three dimensions: cognitive processes, technologies and tasks.

The ‘cognitive dimension’ includes the mental structures and processes involved when a person solves a problem defined as a situation where a person cannot immediately and routinely achieve his or her goals due to some kind of obstacle or challenge. These include setting goals and monitoring progress; planning; locating, selecting and evaluating information; and organising and transforming information.

The ‘technology dimension’ covers the devices, applications and functionalities through which problem solving is conducted. These include hardware devices (laptop computers in the case of the OECD Survey of Adult Skills), simulated software applications, commands and functions, and representations (text, graphics, etc.).

The ‘tasks dimension’ includes the circumstances that trigger a person’s awareness and understanding of the problem and determine the actions needed to be taken in order to solve the problem. Ordinarily, a wide range of conditions can initiate problem solving. For instance, a computer user may realise that his or her mailbox is crowded and that a new schema is needed for classifying e-mails. Alternatively, he or she may be faced with a complex issue (such as finding out more about a medical treatment) and decide to look for relevant information on the Web. In test-taking contexts, tasks are more explicitly assigned to participants. They include the question and task instructions presented to test takers, as well as the specific materials and time constraints associated with the test.

Three categories of task are identified: the purpose and the context in which each task is performed, the intrinsic complexity of the problem and the explicitness of the problem statement and task directions given to the test taker.