Advertisement

The Australian Educational Researcher

, Volume 45, Issue 1, pp 103–124 | Cite as

Designing out barriers to student access and participation in secondary school assessment

  • Linda J. Graham
  • Haley Tancredi
  • Jill Willis
  • Kelli McGraw
Open Access
Article

Abstract

In an effort to support student agency in assessment, teachers seek to provide detailed instructions and advice in associated assessment task sheets. In this paper, we analyse a sample English assessment task to consider how such assessment design practices might inadvertently create barriers to access and participation. To make our case, we highlight the learning characteristics of students in two of the most prevalent disability groups: students with Attention Deficit Hyperactivity Disorder (ADHD) and students with Developmental Language Disorder (DLD). The paper links student agency to the problem of equity by analysing the conditions and criteria of access that are built into assessment design. The article concludes with design recommendations to help improve access for all students, including students in these two highly prevalent disability groups.

Keywords

Accessible assessment design Attention Deficit Hyperactivity Disorder Developmental Language Disorder Inclusive practice 

Introduction

Assessment task sheets are invitations to create performances that will be judged or assessed by others. By being aware of the criteria against which they will be judged, teachers hope students will grow their capacity for self-assessment and active participation. This capacity to evaluate options in order to make choices and have greater personal control over quality outcomes is at the heart of the definition of student agency (Bandura 2006). It is well established that students develop their understanding of quality and their ability to make choices in assessment by drawing from descriptions of shared criteria and standards, alongside social support such as discussion and multiple exemplars, to ensure their assessment responses amount to more than criteria compliance (Sadler 1987; Torrance 2017). Ideally, assessment task sheets are structural resources that help students engage in practical evaluative judgments as they check their developing work against the specifications, while drawing on habits developed over time and their creative imagination. These are practices that reflect Emirbayer and Mische’s (1998) threefold definition of human agency. In practice, however, assessment task sheets also serve multiple purposes for teachers, such as providing evidence of complying with the requirements of a syllabus as part of teacher quality assurance processes, and to help justify assessment decisions to students and parents. These multiple purposes can add to the complexity of task sheet designs, which can then present barriers for students to access and understand the task requirements, so that they can actively participate in the assessment. Even though teachers may attempt to support student agency by demystifying some of the ‘secret teacher business’ of assessment by giving detailed instructions and information in the task sheet, we contend that overly detailed instructions can complicate tasks and create additional barriers for students with disability. Student agency is therefore inextricably linked to accessibility and conceptions of equity.

Equity in summative assessment is a concept that is experiencing radical and rapid change. Where once altering any part of the assessment for any student would be seen as a threat to the reliability of the instrument, teachers are now required to adjust conditions to enable all students to participate in the same task. However, retrospective adjustments can be time-consuming and therefore are typically only made for a small number of students with severe disabilities. As assessment specialists increasingly recognise that consequential validity applies to a much wider range of students, the new challenge for teachers is to design inclusive assessment tasks from the outset (Thurlow and Kopriva 2015; Rajagopalan and Gordon 2016).

Inclusive assessment is a move away from discourses of assessment that originated in the early part of the 20th century and which served an historical function of selection for high status occupations, where the failure of a percentage of students was essential for ranking and sorting purposes (Gipps 1999). Instead of operating as a technology of exclusion, assessment is now expected to operate as a technology of inclusion, where all students have an opportunity to learn and to demonstrate what they have learned (Torrance 2017; Moss et al. 2008). Inclusive assessment designs have contributed alternative approaches for teachers, such as narrative assessment (Bourke and Mentis 2013; Morton et al. 2012) and Universal Design for Learning where greater choice for students in assessment modes is associated with increased agency (Rose and Meyer 2002; Van Haren 2010).

Despite recent advances, the underpinning values from a century ago remain deeply embedded in most classroom assessment traditions (Shepard 2000). Revisiting these dilemmas through the analysis of contemporary assessment task sheets is an opportunity to bring some of those underpinning values to the surface to critically evaluate how assessment task sheets can better support students as active agents who can confidently use the conditions of assessment to shape their assessment performance.

Agency and equity depend on access

When considering the concept of equal educational opportunity almost four decades ago, Burbules et al. (1982) drew on Aristotelean principles of equity and fairness to re-define equal educational opportunity as “access”, which they then split into two domains. These two domains were
  1. (i)

    Conditions of access or external/environmental factors (in this case, an assessment task sheet), and

     
  2. (ii)

    Criteria of access or the personal factors (e.g. traits, skills and abilities) necessary to complete the task.

     
While these may appear to represent a dichotomy between structure/environment and agency/personal capacity, conditions and criteria of access are integrated and mutually informing concepts. Critically, as Burbules et al. (1982) emphasise in the excerpt below, the conditions of access dictate the necessary criteria of access.

Imagine that a book in an elementary school library is on a shelf 6-feet above the ground (with no ladders or other assistance available). In order to have an opportunity to read the book, a student must have access to the book. The fact that the book is on a shelf so high is a ‘condition of access’; it is an environmental and external factor. This condition of access determines that a necessary ‘criterion of access’ to the book is the ability to reach a height of 6-feet; a child 4-feet tall would not (without assistance) have access to the book, hence would have no opportunity to read it. The criteria of access in this case, height or jumping ability, are personal factors, but they are not internal; they are criteria of access precisely because of certain external conditions (Burbules et al. 1982, p. 175, our emphasis).

Problems arise in the realm of assessment when the external conditions create criteria that make access more difficult for some individuals, unfairly disadvantaging them in the process. Students with a disability are those most likely to experience disadvantage in assessment, especially students with so-called “mild” disabilities, such as Attention Deficit Hyperactivity Disorder (ADHD) and Developmental Language Disorder (DLD), who are predominantly educated in mainstream schools and seldom receive adjustments (Mulholland 2017; Marshall et al. 2002). Students with a diagnosis of ADHD or DLD qualify as having a disability under Australian anti-discrimination legislation, which requires education providers to provide reasonable adjustments to ensure that people with disability can access and participate in education on the same basis as everyone else. The term ‘on the same basis’, however, is misleading and can be misinterpreted to mean students with disability must complete the same (unadjusted) assessment. This misinterpretation is compounded by the misperception that reasonable adjustments unfairly confer advantage on students with disability (Dickson 2011).1 This, however, is where the concept of relevance becomes important and it is one that Burbules et al. (1982) emphasise in their consideration of equity.

Determining relevance

Most people intuitively understand that holding an examination in a second-storey assessment hall to which there is only stairway access discriminates against wheelchair users. Drawing on Burbules et al.'s (1982) concept of equity, the condition of access in this case is the stairs and the criterion they impose is the ability to walk. The key issue here is whether that is fair and the most important consideration in determining fairness is whether the criterion being imposed is relevant to the core objective of the assessment. Ensuring access to the examination hall for wheelchair users via a lift, for example, is an uncontroversial adjustment and no one would argue that this unfairly advantages the student with disability or that it disadvantages the majority without. However, there remains significant resistance to the provision of adjustments to students with cognitive or language disabilities because of enduring beliefs that these adjustments make the assessment “easier”, advantaging students with disability in the process (Poed 2016).

In the case of accessible assessment task sheets, such an advantage would only be true if the benefit was not universal and if students’ ability to navigate and interpret the task sheet was the core purpose of the assessment. This is what is meant by the first- and second-order purposes of assessment and this is a useful distinction to apply when considering relevance because the first-order purposes of an assessment task that relate to the core concepts being assessed can easily be overwhelmed by second-order priorities (Cumming and Maxwell 1999). If these second-order priorities complicate assessment task sheets to the extent that the task sheet itself creates barriers to student access and participation, then the result is not a true reflection of that student’s response to the (first-order) purpose of the assessment and the assessment is therefore both inequitable and invalid. Addressing barriers to ensure all students can access and participate in educational assessment is more than a professional responsibility, and it is supported by anti-discrimination legislation and education policy. However, teachers understanding of their legal obligations can also be a barrier to inclusive assessment design.

Educators’ obligations under Australian legislation

In Australia, it is a federally legislated requirement for reasonable adjustment to be made to support all students with disability to access their education on the same basis as students without disability, as described in the Disability Standards for Education (Australian Government 2005). The Standards are informed by the understanding that disability is socially mediated by barriers to access and participation and apply to any student with a disability as described by the Disability Discrimination Act (Australian Government 1992). As the provision of education is a state/territory government responsibility, it is up to each state and territory to interpret the Standards, communicate responsibilities to school staff, and allocate resources to enable the provision of appropriate supports through quality differentiated teaching practice and educational adjustment, as per the processes outlined in the Nationally Consistent Collection of Data on Students with Disability (NCCD).

This, however, is where the translation of theory to policy and practice can lead to confusion. A key area of contention is the concept of reasonable adjustment and which students are entitled to it (Poed 2016), followed closely by poor understanding of differentiation and the importance of proactive planning (Graham et al. 2017; Capp 2016). Knowledge of these concepts and the practices to enable them is important for educators to meet their obligations under the Standards, which are not dependent on state or sector-based disability verification procedures or the provision of support funding. Despite this, there remains enduring belief within Australian schools that students must have a ‘verified’ disability and be in receipt of individually targeted funding to qualify for additional support, even that which should be provided through the first NCCD level of quality differentiated teaching practice (Graham et al. 2017). This thinking sits at odds with the Standards but is influenced by the ways in which education systems have historically determined and distributed resourcing for students with disability (Graham and Jahnukainen 2011).

While the mechanisms differ between education systems across Australia, most students with disability receive support through a hierarchy of responses beginning with differentiated teaching practice and supplementary adjustments, such as extra time in exams. Students considered to have low to moderate support needs are funded through census-based resource allocation methods (Sigafoos et al. 2010), which include a mix of staffing (e.g. learning support teacher allocation) and funding based on school enrolment numbers and school/community need. This general or ‘base’ funding can be used to fund support programs, teacher release, professional development and the employment of support staff. Schools can also apply for individually targeted funding to support students whose needs cannot be fully met through this base-level resourcing.

To date, individually targeted (per student) funding has been tightly regulated by eligibility criteria that are tied to medical categories and departmental ‘verification’ of disability (Sigafoos et al. 2010).2 The verification process is highly bureaucratic, labour intensive and administratively expensive. The catch is that there are only a limited number of eligible disability categories and funding is apportioned in accordance with the level and frequency of support required. Queensland’s ‘Education Adjustment Program’ (EAP), for example, provides targeted funding for students who meet specific criteria and who require significant support to access and demonstrate their learning (DET 2015). Within Queensland state schools, there are five eligibility categories: Intellectual Disability (ID), Speech-Language Impairment (SLI), Autism Spectrum Disorder (ASD), Physical Impairment (PI), Vision Impairment (VI) or Hearing Impairment (HI).

Categorical resource allocation methods are problematic because linking funding to diagnoses is known to drive diagnosis in eligible categories and to encourage substitution of diagnoses from unfunded to funded categories, e.g. substituting a diagnosis of Attention Deficit Hyperactivity Disorder with Autism Spectrum Disorder (Graham 2015a; Skellern et al. 2005). Although education systems in Australia have attempted to blunt the perverse effects of categorical support allocation methods through the increased use of census-based approaches for students considered to have low to moderate support needs (Graham 2016a), and by developing and promoting adjustments-based models such as the NCCD, these programs are poorly communicated by state Departments of Education and are therefore not well understood in schools (Graham 2015a). As we noted earlier, the greatest misconception is that only students who receive individually targeted funding are entitled to reasonable adjustments. A related misconception is that all adjustments require individually targeted funding and a teacher’s aide or other staff to provide them (Graham 2015a).

Due to these misconceptions, students with a disability who have not been through the verification process and who do not receive individually targeted funding are vulnerable to not having their learning needs met. In this paper, we focus on two groups that are especially vulnerable: students with Attention Deficit Hyperactivity Disorder (ADHD) and students with Developmental Language Disorder (DLD). Students in these groups frequently miss out on support and do not receive adjustments, either because ADHD is not listed in the limited number of state-based categories of disability eligible for individually targeted funding, or because state-based eligibility thresholds capture only a small proportion of the students affected by DLD. For these students, it is critical that teachers plan proactively to address potential barriers to their access and participation, as even subtle barriers can have significant functional impact on their learning opportunities and educational outcomes.

ADHD and DLD

Attention Deficit Hyperactivity Disorder (ADHD) is defined as a persistent pattern of inattention and/or hyperactivity–impulsivity that is atypical to other individuals at comparable levels of development and which interferes with functioning in at least two settings (American Psychiatric Association 2013). It is the most frequently diagnosed neurodevelopmental disorder in Australia, representing 26% of the outcome from all referrals to paediatricians (Wright et al. 2009). Of the seven mental disorders assessed by the second Australian Child and Adolescent Survey of Mental Health, ADHD was the most common with an estimated prevalence of 7.4% (Lawrence et al. 2016). Teachers with 30 students are likely to encounter at least two students with ADHD in their class (Ohan et al. 2008). This number may be much higher if the school is in a disadvantaged community given the higher prevalence of ADHD in low socioeconomic areas (Russell et al. 2016), or if the school streams lower-achieving students and students with disability into separate ‘support’ classes (Graham 2015b).

The classroom setting represents one of the most challenging environments for students with ADHD (DuPaul and Weyandt 2006). To be successful learners, children are required to concentrate, not be easily distracted or distract others, listen attentively, follow instructions, be organised and prepared, and complete assigned work accurately and in a timely manner (Prosser 2008). However, these everyday tasks place significant demand on neurocognitive functions that are known weaknesses for students with ADHD, including cognitive and interference control, inhibition, working memory, set-shifting, planning, fluency, aggregated executive function, attention and information processing (van Lieshout et al. 2013). Difficulties enacting these cognitive functions manifest in noticeable behaviours. For example, being easily distracted makes it hard to concentrate. Concentration difficulties, in conjunction with working memory limitations, affect listening and reading comprehension, making it difficult to follow complex verbal or written instructions. Not surprisingly, the most common reason for which these students report getting in trouble at school is “not following instructions” and “not doing work” (Graham 2016b).

Although most children with ADHD are of average or above average intelligence, some of the conditions often experienced when engaging in higher-order cognitive tasks—such as dealing with visual and linguistic complexity, distinguishing between important and unimportant information, and prioritising, organising and coordinating—require these students to expend significant effort, time and energy that is incommensurable with that intended by the task. Unnecessary complexity in the way that curriculum, pedagogy and assessment task expectations are communicated presents barriers to students with ADHD with the result being that many give up before they even start. As these students seldom qualify for individually targeted funding (Mulholland 2017), most are reliant on the provision of quality differentiated teaching practices that proactively address potential barriers to these students’ access and participation.

Developmental Language Disorder (DLD) is defined as persistent difficulties in the acquisition and use of language, impacting word and sentence structure, discourse skills, and comprehension and/or production of vocabulary (Bishop, Snowling, Thompson, Greenhalgh, CATALISE consortium 2016). These difficulties are evident in the spoken, written and/or signed language modes (American Psychiatric Association 2013). DLD has been described as one of the most common but poorly identified disabilities of childhood (Whitehouse 2012), with at least 7% of students presenting with DLD at school entry (Tomblin et al. 1997). This translates to around two students in an average sized classroom (Norbury et al. 2016). Most students with DLD have non-verbal cognitive skills within the average range; however, as most higher-order cognitive tasks—such as organising and structuring ideas and converting these to a written text, synthesising key ideas or themes from a written text, or deciphering important from unimportant information—are bound in language, these tasks are extremely difficult for students with DLD.

Clinically significant DLD—that is, DLD with functional impact—is indicated by two or more composite scores on standardised language assessment that are at least 1.25 standard deviations (SD) below the mean (Tomblin et al. 1997). As we noted earlier though, state-based categorical funding models are based on narrow criteria, including standard scores on standardised language assessments. In Queensland, the cut-off for verification is a standard score of 2.0 SD below the mean and, in Victoria, the cut-off is 3.0 SD below. In practice, this means that there may be many students with clinically significant DLD impacting on their ability to access and participate in learning who may not be eligible for individually targeted funding. These verification cut-offs also act as a disincentive for schools to assess for DLD and, as such, there are many students whose language needs have not been identified and who therefore receive no support, even through quality differentiated teaching practice.

As all aspects of curriculum, literacy and interpersonal engagement in the school context are underpinned by language (Ukrainetz and Fresquez 2003), the educational impact of DLD is significant, all pervasive and debilitating (Leonard 2014). Success in the classroom requires children to learn around 2000 new words each year and to map these within their existing language system. They also need to understand, learn and apply new concepts, store and retrieve information, understand and respond to instructions, comprehend written texts, create meaningful and well-structured texts, engage in discussions with others in the classroom, and give their perspective related to classroom content (Nippold 2016). These classroom tasks place significant demand on the linguistic and social language difficulties that are core to DLD, such as learning and using new vocabulary, understanding and appropriately using multiple word meanings and figurative language, using and comprehending morphological structures (for example, to indicate tense, as in walking), creating and structuring sentences and longer texts to convey ideas, and adjusting the communicative tone or style used to suit the audience or task (Leonard 2014). Considering potential language barriers and working to minimise these will thus facilitate participation for all students, but is essential for students with DLD to enable them to participate in and demonstrate their learning.

Analysis of a sample assessment task

While many teachers adjust classroom teaching to support students who experience difficulties in learning, including those with ADHD and DLD, adjustments are less common when it comes to assessment, especially in the secondary years of school (Cumming et al. 2013). This concern prompted a research project that investigated how secondary school English teachers design assessment to support student agency and equity (Willis et al. 2016). Analysis of the sample task sheet featured in this paper enabled the research team to engage in a process of collaborative assessment re-design with Senior English teachers from two schools participating in the project. Written permission to use the de-identified task sheet featuring in the following analysis was provided by the English Head of Department from “Happy Days High School”. It is acknowledged that the task was designed to be helpful and engaging for students. The analysis focuses on the way that the task is communicated to students, as this level of detailed instruction is typical for contemporary assessment task sheets.

In this paper, we use a multi-layer approach to analyse the sample assessment task from “Happy Days High School” to question whether some of the practices teachers use to support student agency may inadvertently affect access and therefore equity. For example, in the first layer of analysis, we consider how the provision of “helpful” information might contribute to visual complexity, making it harder for students to distinguish between important and unimportant information. In the second layer, we examine procedural complexity, as in the number of elements that students are required to incorporate to meet the assessment criteria, and consider how poor alignment between these elements may make the task more difficult than it needs to be. In the third layer, we review linguistic complexity, or the language that is being used in the task sheet, and contemplate the barriers that this language may create.

Layer 1: visual complexity

The sample Year 8 task sheet is two pages in length, totalling 1,110 words. The first page has 45 text boxes either holding information related to the task or providing spaces that require the student’s name, class and classroom teacher’s name (see Fig. 1). The top third of the first page includes information that is both important (e.g. conditions, dates due) and unimportant (e.g. Item ‘S2’) for students. The salience of some of the information provided (e.g. text type, mode) is unclear, without knowing the emphasis placed on explicitly teaching these in class. The text is left-justified with the use of five different font sizes (9, 10, 11, 12 and 14 point), plus bold lettering and bullet points for emphasis.
Fig. 1

Year 8 English assessment task sheet (p. 1)

The middle third of the page includes a long and densely written paragraph on context (e.g. what students have been doing in the unit of study leading up to this summative assessment), and a central text box outlining the student’s role, the assessment task and the aim of the assessment. This box is heavily outlined and the ‘task’ sentence at the centre is bolded and underlined, presumably to draw students’ attention to these aspects of the task sheet, indicating core requirements of the task. Unlike the previous section, the text is centre-justified with the use of another five different font sizes (10, 11, 12, 14 and 21 point), plus italic and bold lettering for emphasis.

The bottom third of the page contains a large amount of additional information presented with no consistent style. There are three main text box sections. The first includes information on task length, purpose and audience, as well as information relating to the language features (e.g. point of view, tense, field, tenor, mode and tone) relevant to the task. The second box contains information regarding ‘Common Curriculum Elements’; however, this information is highly technical and of questionable relevance to students. The final box contains a statement of the student’s contribution and spaces for students to sign and date to verify their authorship. Despite the requirement for students to cite any assistance received, there is no room to do so. The text ranges from centre-justified to left-justified with the use of another five different font sizes (8, 9, 10, 11 and 16 point), plus the use of bold lettering and underline for emphasis.

The second page of the assessment task sheet has 25 text boxes presenting a range of performance examples corresponding to A–E grades across two modes: receptive and productive. Although neatly set out, there is an abundance of text in 9-point Arial Narrow font with large, heavily indented bullet points further crowding the text. The formal reference framing the standard descriptors is that ‘the student work has the following characteristics’ indicating that the rubric has been written for teachers, rather than students. Rubrics, however, should ideally serve both and, if being provided to students, the rubric should help to clarify and not complicate, as this one might.

The emphasis in the way the task is described in different sections varies, which has led to some misalignment. For example the rubric emphasises students’ interpretation of the ‘poet’s use of poetic devices to construct a sense of place, supported by evidence and information from the poem and the poet’s background’. However, the rubric does not explicitly refer to the second aspect of the task that was highlighted on the first page (with the use of bold and underline) and which requires the student to ‘evaluate [their] personal response to the poem’, ‘consider how [they] connect to [their] sense of place’ and ‘how [their] sense of place has been influenced by the poem’. By contrast, the description of the task on the first page does not refer to the poet’s background and refers only obliquely to the use of specialised terms regarding poetic devices in the description of field in the ‘Language Features’ box. In other words, the task description does not exactly match the rubric, the danger being that students who take the instructions on the first—and arguably most important—page at face value may receive lower marks. There is also the potential for inequity in outcomes between (i) students who use the marking rubric as the definitive guide, (ii) students who use the highlighted instructions on the first page of the task sheet as their chief source of information, and (iii) students who attempt to satisfy the requirements of both. The presence of this conflicting information also has implications for procedural complexity (Figs. 1, 2).
Fig. 2

Year 8 English assessment rubric (p. 2)

Layer 2: procedural complexity

The presence of conflicting information in three different sections of the task sheet significantly increases the scope of the task by introducing new elements each time (see Fig. 3). For example, on the first page of the task sheet, the ‘Task’ is described as
Fig. 3

Breakdown of Year 8 assessment instructions

Choose a poem from the list given to you by your teacher and use it to:

Create and deliver a presentation explaining how the poem uses language in an emotive way to express a sense of place, and evaluate your personal response to the poem.

Although this statement is broken into two sections, four ideas are presented, and each constitutes an element required to complete the task. Further information is then provided in the ‘Aim’ section (see Fig. 3 below), adding another four elements to the task:

This should include an analysis of how the poem uses language in an emotive way to express a sense of place and belonging and how you connect to it and what you learn about your own ‘sense of place’ from the poem.

Students are therefore required to complete eight elements to complete this task. Figure 3 visually maps these eight task elements to illustrate the full complexity of and contradictions in what Year 8 students are being asked to do in the assessment task.

It is important to note here that the task sheet is only one source from which students receive assessment information. Teachers often clarify the task through student questions. However, this type of clarification can increase the potential for further misunderstanding as students attempt to interpret the task sheet and to reconcile it with their teacher’s verbal instructions. In other words, drawing conclusions about what is required in this task requires students to organise and prioritise both the information presented in the task sheet and the teacher’s verbal instructions in class. Requiring this level of analysis and synthesis places a heavy cognitive load on 13 to 14-year-old students, and the contradictions and irregularities within the task sheet present an unnecessary level of challenge for students with ADHD or DLD. This is at odds with the Disability Standards for Education (Australian Government 2005), which calls educators to engage in assessment practices that are accessible for all.

Another factor that contributes to the procedural complexity of this assessment task is the required word count of 600 words together with the allocated speech length of 2–3 min. In the rubric on the second page, emphasis is placed on the use of a range of ‘language features’ to present a ‘clear, highly engaging’, ‘cohesive’, ‘highly effective and reflective’ presentation at the A-standard level. One of the language features assessed is pace, however, as the average number of spoken words per minute is 130 words, a more appropriate time limit for a 600-word speech is 4–5 min. Alternatively, a more appropriate word count for a 2- to 3-min speech is 250–400 words. The incompatibility of the requirements in the assessment task may negatively affect the achievement of students who attempt to present their whole speech verbally within the prescribed time limits. Furthermore, a rigid approach, where assessment modalities are prescribed—as in the requirement for a ‘spoken multimodal presentation’—may result in discrimination against students with ADHD and DLD because of the nature of the language or cognitive difficulties faced by these students. Not only is this inflexibility at odds with the Disability Standards for Education (Australian Government 2005) but it is inconsistent with the Australian Curriculum, which was developed in accordance with the principles of Universal Design for Learning (Rose and Meyer 2002) precisely to enable flexibility and choice. At this point, we emphasise that we have only analysed visual and procedural complexity. The following analysis of the linguistic complexity of the task is designed to raise the awareness of assessment task designers so that they can address issues such as this in future designs.

Layer 3: linguistic complexity

For a reader to comprehend a text, both the vocabulary (that is, the words used within the text and the reader’s understanding of these) and the syntax (word order, sentence and paragraph structure) must be accessible and meaningful to the reader (Nippold 2016). In attempting to help students understand the connection between the English unit and the associated assessment task, this sample task sheet outlines the ‘Context’, stating:

In this unit, you have explored the representation of ‘sense of place’ through local, Indigenous poetry and analysed how the poems create a connection to a special place through the use of poetic techniques which create imagery and evoke emotions.

This 40-word, vocabulary-rich sentence contains four verb phrases and five prepositional phrases. The linguistic length, density and structure of this sentence is taxing on the reading comprehension system and presents multiple opportunities for students with language or attention difficulties to misinterpret the message or disengage from the content. The linguistic length of the sentence is in part due to the use of prolix or unnecessarily wordy sentence construction. Furthermore, terms such as ‘evoke’, a low-frequency word (Australian National Corpus 2012), may not be readily understood by all Year 8 students. Thus, while this contextual information may have been provided in an attempt to support students by setting the scene of the task and modelling complex vocabulary, the language used in the Context section has the potential to instead cause comprehension difficulties, thereby reducing students’ understanding of the task requirements.
Similar complexity is evident in the ‘Aim’ section, which utilises a complex, 42-word sentence comprising six verb phrases and five prepositional phrases:

This should include an analysis of how the poem uses language in an emotive way to express a sense of place and belonging and how you connect to it and what you learn about your own ‘sense of place’ from the poem.

Again, a complex syntactic structure is adopted in this sentence, where layers of meaning are built through a series of subordinated and coordinated clauses. To comprehend this sentence, a young adolescent reader must have superior working language memory, an understanding of the vocabulary used and the ability to synthesise several ideas, as we depicted earlier in Fig. 3.

In the rubric on the second page of the task sheet, the ‘language features’ section introduces technical, specialist discourse, relevant to the English curriculum. There is a significant difference, however, in the specialist discourse offered in the ‘A’ range standard versus the ‘B–E’ range standards in this area. For example, performances reaching an ‘A’ range standard are considered to demonstrate ‘informal language’, use of ‘specialised poetic terms’ and ‘inclusive language’, whereas tasks in the ‘B–E’ range demonstrate ‘formal language’, use ‘language of judgement’ and ‘rhetorical devices’. It is possible that a copy-and-paste error is the reason for this inconsistency or it may be that the use of formal language in the B–E range is being taken as a signal that students have not recognised the tone considered appropriate to adopt when speaking to a fellow student audience. Either way, reliance on students’ understanding of any of this technical terminology remains a concern, even if the document were consistent across the A–E standards. By including terms such as ‘cohesive ties’, ‘inclusive language’ and ‘reflective’ (a key term for understanding this task), the task sheet author assumes a shared meaning with the Year 8 student reader, which will depend on these terms being explicitly taught as part of classroom teaching experiences.

The tension of being transparent about marking criteria and attempting to embed fine-grained detail within the marking rubric is a challenging design issue. However, transparency can come at the cost of overloading and cluttering the rubric with too much information. If rubrics using specialist language are to be provided to students, students’ comprehension of key terms should ideally be supported by a glossary that uses student-friendly definitions to ensure that students with ADHD or DLD can have access to this language on the same basis as their peers. And, although we do not have the space to analyse the effect of “competing logics” (Wall 2017) in this paper, the complexity of and contradictions within this task sheet does raise the question as to whether pressure on teachers to try and meet too many competing agendas has played a role in corrupting the integrity of this assessment task.

Discussion

Close analysis of this typical high school assessment task sheet makes this common and everyday artefact seem unfamiliar. It reveals a visually dense, procedurally difficult and linguistically challenging document. For adolescents with ADHD or DLD, it would constitute an informational minefield with their chance of doing poorly increasing with every repetition, each contradiction and each new ‘re-framing’ of the task’s aim, purpose, mode and success criteria. Certainly, students with ADHD or DLD are significantly disadvantaged as the demands of complex task sheets can work in ways that privilege students with the capacity to closely read, identify and reconcile the incompatibilities within the document to successfully execute the task. As we described earlier, inherent to the features of ADHD and DLD are difficulties with working memory, comprehension and synthesis of information. If the complexity of this task sheet creates access barriers that affect these students’ ability to understand the task requirements, this will also prevent these students’ from demonstrating what they have learned about the unit of study being assessed. This brings us to a key conceptual point that is fundamental to the principle of equity, and that is the a priori importance of accessibility. The driving question with which we start our discussion of the accessibility of and equity in assessment is this: Is the purpose of the assessment to grade students on their ability to decipher the assessment task sheet OR is the purpose to provide an opportunity for students to relay what they know about the text and have learned in the relevant unit of study?

We ask this question knowing that there is an historical perception that successful interpretation of an assessment task sheet is a prerequisite to demonstrating superior achievement: in that higher achieving students will “do better” in the task because they have the skills to analyse, interpret and approximate the required performance. The problem, however, is that assessment task sheets—even one as detailed as the one we have analysed—are not equally inaccessible. Design features that unnecessarily contribute to the visual, linguistic and procedural complexity of the task discriminate against students who have difficulty concentrating, processing dense tracts of text, distinguishing between important and unimportant information, and prioritising, organising and coordinating. We have chosen to focus on students with ADHD and DLD in this paper, however, the same challenges apply to students with dyslexia, students with Autism Spectrum Disorder, students with intellectual disability, students from a Language Background Other than English, Indigenous students, students with vision impairment, and students from disadvantaged backgrounds. These same students can be successful under the right conditions but—as we explained earlier—assessment task sheets set what Burbules et al. (1982) refer to as ‘conditions of access’. These are conditions that dictate implicit skills known as ‘criteria of access’ that students must apply to successfully complete the task: some will be relevant to the first-order purpose of the assessment, but others may not be relevant.

A key issue in relation to the task that was closely analysed in this article is whether all the skills demanded by this task sheet are relevant to the central domain and performance being assessed. To determine this, we return to the theoretical framework informing our approach to analysis, and ask:
  1. 1.

    What are the conditions of access to this assessment and which criteria of access do they impose?

     
  2. 2.

    Which of these criteria are intrinsic to the task being assessed and which are functioning like stairs at entry to the assessment hall?

     
Our three layers of analysis have identified numerous areas in which the conditions of access set by the task design impose additional criteria of access that distract from the first-order purpose of the task and which inadvertently discriminate against students with ADHD and DLD by requiring students to
  • negotiate unnecessary visual complexity,

  • distinguish between important and unimportant information,

  • decipher what action is necessary from different sections of the task sheet which contradict,

  • reconcile competing purposes,

  • deliver a written speech in too short a timeframe,

  • engage with long, complex sentences and instructions that were most likely written for teachers, and

  • interpret specialist and technical language that may or may not have been explicitly taught in class.

The key question for teachers is whether the imposition of these additional criteria is fair and, to answer it, they must ask two more questions:
  1. 1.

    Are the additional criteria imposed by the assessment task sheet relevant to the core learning objectives of the curriculum and relevant unit of study (e.g. understanding literature in context, developing a personal response to literature, creating a multimodal spoken presentation)?

     
  2. 2.

    Might these additional criteria prevent students from effectively demonstrating their knowledge, either because they cannot understand what the task is asking of them or because the task itself does not allow sufficient time or space to enable students to do justice to each component?

     

These are important questions for teachers to ask when examining existing assessment tasks; however, the next and most critical step is to incorporate these equity principles into task design. Proactive planning and accessible task design is the most efficient and effective way to ensure that students with ADHD and DLD—not to mention students with dyslexia, students with Autism Spectrum Disorder, students with intellectual disability, students from a Language Background Other than English, Indigenous students, students with vision impairment and students from disadvantaged backgrounds—are provided with a genuine opportunity to demonstrate what they have learned and can do.

Conclusion and recommendations

While our analysis of a typical assessment task sheet highlighted some concerns about assessment design, it is not intended to undermine the important intellectual work that teachers do in designing authentic and valid assessment. Locally designed assessment tasks enable teachers to respond to the diversity of the learners in their classroom and consider how assessment design relates to the ongoing development of those learners. Teachers draw together information from curriculum documents, school priorities, pedagogic frameworks, and knowledge of students’ cultural and social communities, as well as specific learning abilities and disabilities as they design assessment tasks. They are supported by policy documents that provide general principles for the design of assessment (Queensland Studies Authority 2013; Australasian Curriculum, Assessment and Reporting Authority 1995). Existing literature also includes important advice to consider the assessment from the perspective of the student (Cumming et al. 2013).

Recommendations for designing accessible assessment therefore need to consider not only accessible design elements, but also the experiences of learners, which is central to the definition of fair assessment (Klenowski 2014). Assessment task sheets are invitations to students to demonstrate their mastery of new understandings; however, these invitations must be evaluated in terms of the affordances they provide for different learners and whether learners can transform those affordances into action (Gee 2008). To conclude, we provide some additional, specific design recommendations to support teachers and student reviewers (see Table 1), as they address the important issue of access for many students based on our three layers of analysis.
Table 1

Design recommendations for equity in access

Visual accessibility

The layout helps the students access the important elements of the task

  • The most important information is easy to find

  • White space is used to separate sections

  • Text size aids readability (11 or 12 point font with 1.5 line spacing)

  • Margins are left-justified

  • Visual cues direct student attention

  • Information that is irrelevant to students is not included

Procedural accessibility

 Consistency and clarity of instructions

  • Authentic context is relevant

  • Common access barriers have been addressed in the design

  • The task, objectives and criteria align

  • Students are able to respond within the prescribed conditions

  • Enough space and resources are provided for responses

  • The assessment is scheduled to give students the best opportunity for success

  • Processes for evaluating quality are clear

  • Authentication strategies are included

  • Student feedback on the draft task was sought

  • Teacher peer feedback on draft task was sought

Linguistic accessibility

 Directions are clear

  • Instructions are clear and direct

  • Sentences are short and simply structured

  • The language is free of bias

  • Specialist language is defined using student-friendly terms

  • Information is stated once only and if it needs to be referenced more than once, consistent terminology is used

Alongside principles such as these, professional learning about conditions of access can help teachers make adjustments to their assessment designs. When teachers come to understand the barriers that conditions of access can create, they can readily ‘design out’ the types of accessibility issues we have identified in our analysis of the sample task sheet. In our research with two participating secondary schools, we used a preliminary version of the above recommendations in the form of an “accessibility checklist” to support teachers in their design work. Importantly, participating teachers reported that students who had not previously found success were able to demonstrate their learning with new levels of confidence. Rather than making individual adjustments after students begin to struggle, this article seeks to support teachers to learn more about accessible assessment in initial designs. Further research is needed to test the contribution that accessible assessment design can make to the engagement and achievement of all students, in addition to students with ADHD and DLD, as well as research to determine how accessible assessment design might affect teacher assessment literacy and workload.

Footnotes

  1. 1.

    Other contributing factors are the knowledge and attitudes of teachers towards students with certain disabilities, especially ADHD and DLD (Mulholland et al. 2015; Marshall et al. 2002), and the perceived trade-off between reasonable adjustments for people with disability and rigorous academic standards (Riddell and Weedon 2006; Poed 2016).

  2. 2.

    It is anticipated that this will change in 2018 when data from the Nationally Consistent Collection of Data on Students with Disability (NCCD) will inform Australian Government disability loadings under “Gonski 2.0”. At present, it is not clear how this change will affect the ways in which the different states and sectors allocate funding and resources.

Notes

Acknowledgements

The authors would like to thank the English Head of Department from “Happy Days High School” for granting permission for this team to analyse their school’s English assessment task sheet in support of this research, and the Queensland Department of Education and Training who funded the research through their Education Horizons scheme.

References

  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders: DSM-5. Washington, D.C: American Psychiatric Association.CrossRefGoogle Scholar
  2. Australasian Curriculum, Assessment and Reporting Authority. (1995). Guidelines for assessment quality and equity. Retrieved from Queensland Curriculum and Assessment Authority website https://www.qcaa.qld.edu.au/downloads/approach2/acaca_equity_guidelines.pdf.
  3. Australian Government. (1992). Disability discrimination act. Retrieved from http://www.austlii.edu.au/au/legis/cth/consol_act/dda1992264/.
  4. Australian Government. (2005). Barton, ACT: Australian Government Publishing Service. Retrieved from http://www.deewr.gov.au/Schooling/Programs/Documents/Disability_Standards_for_Education_2005.pdf.
  5. Australian National Corpus. (2012). Australian National Corpus. Retrieved from https://www.ausnc.org.au/.
  6. Bandura, A. (2016). Toward a psychology of human agency. Perspectives on Psychological Science, 1(2), 164–180.CrossRefGoogle Scholar
  7. Bishop, D. V., Snowling, M. J., Thompson, P. A., & Greenhalgh, T. (2016). CATALISE: A multinational and multidisciplinary Delphi consensus study. Identifying language impairments in children. PLoS ONE, 11(7), e0158753.CrossRefGoogle Scholar
  8. Bourke, R., & Mentis, M. (2013). Self-assessment as a process for inclusion. International Journal of Inclusive Education, 17(8), 854–867.  https://doi.org/10.1080/13603116.2011.602288.CrossRefGoogle Scholar
  9. Burbules, N. C., Lord, B. T., & Sherman, A. L. (1982). Equity, equal opportunity, and education. Educational Evaluation and Policy Analysis, 4(2), 169–187.CrossRefGoogle Scholar
  10. Capp, M. (2016). Is your planning inclusive? Australian Educational Leader, 38(4), 44–46.Google Scholar
  11. Cumming, J., Dickson, E., & Webster, A. (2013). Reasonable adjustments in assessment: Putting law and policy into practice in Australia. International Journal of Disability, Development and Education, 60(4), 295–311.CrossRefGoogle Scholar
  12. Cumming, J., & Maxwell, G. S. (1999). Contextualising authentic assessment. Assessment in education: Principles, policy & practice, 6(2), 177–194.CrossRefGoogle Scholar
  13. Department of Education and Training. (2015). Education Adjustment Program (EAP) Handbook. Brisbane, Australia: Department of Education and Training.Google Scholar
  14. Dickson, E. (2011, October). Reasonable adjustment and the assessment of students with disabilities: Australian legal issues and trends. In Proceedings of Australia and New Zealand Education Law Association 20th Annual Conference (pp. 22–31). Australia and New Zealnad Education Law Association.Google Scholar
  15. DuPaul, G. J., & Weyandt, L. L. (2006). School-based intervention for children with attention deficit hyperactivity disorder: Effects on academic, social, and behavioural functioning. International journal of disability, development and education, 53(2), 161–176.CrossRefGoogle Scholar
  16. Emirbayer, M., & Mische, A. (1998). What is agency? American Journal of Sociology, 103(4), 962–1023.CrossRefGoogle Scholar
  17. Gee, J. P. (2008). A sociocultural perspective on opportunity to learn. In P. A. Moss, D. C. Pullin, J. P. Gee, E. H. Haertel, & L. J. Young (Eds.), Assessment, equity, and opportunity to learn (pp. 76–108). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  18. Gipps, C. (1999). Socio-cultural aspects of assessment. Review of research in education, 24(1), 355–392.CrossRefGoogle Scholar
  19. Graham, L. J. (2015a). A little learning is a dangerous thing: Factors influencing the increased identification of special educational needs from the perspective of education policy-makers and school practitioners. International Journal of Disability, Development and Education, 62(1), 116–132.CrossRefGoogle Scholar
  20. Graham, L. J. (2015b, February 11). The way schools cope with learning difficulties is doing more harm than good. The Conversation, Retrieved from: https://theconversation.com/the-way-schools-cope-with-learning-difficulties-is-doing-more-harm-than-good-36544.
  21. Graham, L. J. (2016a). Reconceptualising inclusion as participation: Neoliberal buck-passing or strategic by-passing? Discourse: Studies in the Cultural Politics of Education, 37(4), 563–581.Google Scholar
  22. Graham, L. J. (2016b). ‘Schoolwork’ and ‘teachers’: Disaffected boys talk about their problems with school. In Challenging Dominant Views on Student Behaviour at School (pp. 115–131). Springer Singapore.Google Scholar
  23. Graham, L. J., Davis, J., & Spandagou, I. (2017). What exactly is Differentiation and why is it so poorly understood? A systematic review. Inclusive education SIG symposium paper presented at the Australian Association for Research in Education (AARE) Annual Conference, November 26th–December 2nd, Canberra.Google Scholar
  24. Graham, L. J., & Jahnukainen, M. (2011). Wherefore art thou, inclusion? Analysing the development of inclusive education in New South Wales, Alberta and Finland. Journal of education policy, 26(2), 263–288.CrossRefGoogle Scholar
  25. Klenowski, V. (2014). Towards fairer assessment. The Australian Educational Researcher, 41(4), 445–470.CrossRefGoogle Scholar
  26. Lawrence, D., Hafekost, J., Johnson, S. E., Saw, S., Buckingham, W. J., Sawyer, M. G., et al. (2016). Key findings from the second Australian child and adolescent survey of mental health and wellbeing. Australian and New Zealand Journal of Psychiatry, 50(9), 876–886.CrossRefGoogle Scholar
  27. Leonard, L. B. (2014). Children with specific language impairment (2nd ed.). London, England: The MIT Press.Google Scholar
  28. Marshall, J., Stojanovik, V., & Ralph, S. (2002). ‘I never even gave it a second thought’: PGCE students’ attitudes towards the inclusion of children with speech and language impairments. International Journal of Language & Communication Disorders, 37(4), 475–489.  https://doi.org/10.1080/1368282021000008892.CrossRefGoogle Scholar
  29. Morton, M., McMenamin, T., Moore, G., & Molloy, S. (2012). Assessment that matters: The transformative potential of narrative assessment for students with special education needs. Assessment Matters, 4, 110–128.Google Scholar
  30. Moss, P. A., Pullin, D. C., Gee, J. P., Haertel, E. H., & Young, L. J. (Eds.). (2008). Assessment, equity, and opportunity to learn. Cambridge: Cambridge University Press.Google Scholar
  31. Mulholland, S. M. (2017). ADHD: The untold truths of the ADEP (Australian Deficit in Educational Policy). International Journal of Disability, Development and Education, 64(1), 1–18.CrossRefGoogle Scholar
  32. Mulholland, S. M., Cumming, T. M., & Jung, J. Y. (2015). Teacher attitudes towards students who exhibit ADHD-type behaviours. Australasian Journal of Special Education, 39(1), 15–36.CrossRefGoogle Scholar
  33. Nippold, M. A. (2016). Later language development: The school-age and adolescent years (4th ed.). Austin, TX: Pro-Ed.Google Scholar
  34. Norbury, C. F., Gooch, D., Wray, C., Baird, G., Charman, T., Simonoff, E., et al. (2016). The impact of nonverbal ability on prevalence and clinical presentation of language disorder: Evidence from a population study. Journal of Child Psychology and Psychiatry.  https://doi.org/10.1111/jcpp.12573.Google Scholar
  35. Ohan, J. L., Cormier, N., Hepp, S. L., Visser, T. A., & Strain, M. C. (2008). Does knowledge about attention-deficit/hyperactivity disorder impact teachers’ reported behaviors and perceptions? School Psychology Quarterly, 23(3), 436.CrossRefGoogle Scholar
  36. Poed, S. (2016). Adjustments to curriculum for Australian school-aged students with disabilities: What’s reasonable? Unpublished PhD thesis. Griffith University.Google Scholar
  37. Prosser, B. J. (2008). Beyond ADHD: A consideration of attention deficit hyperactivity disorder and pedagogy in Australian schools. International Journal of Inclusive Education, 12(1), 81–97.CrossRefGoogle Scholar
  38. Queensland Studies Authority. (2013). Year 8 English. Australian Curriculum in Queensland. Retrieved from Queensland Curriculum Assessment Authority website. https://www.qcaa.qld.edu.au/downloads/p_10/ac_english_yr8.pdf.
  39. Rajagopalan, K., & Gordon, E. W. (2016). The testing and learning revolution: The future of assessment in education. New York: Springer.Google Scholar
  40. Riddell, S., & Weedon, E. (2006). What counts as a reasonable adjustment? Dyslexic students and the concept of fair assessment. International Studies in Sociology of Education, 16(1), 57–73.CrossRefGoogle Scholar
  41. Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Alexandria: Association for Supervision and Curriculum Development.Google Scholar
  42. Russell, A. E., Ford, T., Williams, R., & Russell, G. (2016). The association between socioeconomic disadvantage and attention deficit/hyperactivity disorder (ADHD): A systematic review. Child Psychiatry and Human Development, 47(3), 440–458.CrossRefGoogle Scholar
  43. Sadler, D. R. (1987). Specifying and promulgating achievement standards. Oxford Review of Education, 13(2), 191–209.CrossRefGoogle Scholar
  44. Shepard, L. A. (2000). The role of assessment in a learning culture. Educational researcher, 29(7), 4–14.CrossRefGoogle Scholar
  45. Sigafoos, J., Moore, D., Brown, D., Green, V. A., O’Reilly, M. F., & Lancioni, G. E. (2010). Special education funding reform: A review of impact studies. Australasian Journal of Special Education, 34(01), 17–35.CrossRefGoogle Scholar
  46. Skellern, C., Schluter, P., & McDowell, M. (2005). From complexity to category: Responding to diagnostic uncertainties of autistic spectrum disorders. Journal of Paediatrics and Child Health, 41(8), 407–412.CrossRefGoogle Scholar
  47. Thurlow, M. L., & Kopriva, R. J. (2015). Advancing accessibility and accommodations in content assessments for students with disabilities and English learners. Review of Research in Education, 39(1), 331–369.CrossRefGoogle Scholar
  48. Tomblin, J. B., Records, N. L., Buckwalter, P., Zhang, X., Smith, E., & O’Brien, M. (1997). Prevalence of specific language impairment in kindergarten children. Journal of Speech, Language, and Hearing Research, 40(6), 1245–1260.  https://doi.org/10.1044/jslhr.4006.1245.CrossRefGoogle Scholar
  49. Torrance, H. (2017). Blaming the victim: assessment, examinations, and the responsibilisation of students and teachers in neo-liberal governance. Discourse, 38(1), 83–96.Google Scholar
  50. Ukrainetz, T. A., & Fresquez, E. F. (2003). What isn’t language? A qualitative study of the role of the school speech-language pathologist. Language, Speech, & Hearing Services in Schools, 34(4), 284–289.  https://doi.org/10.1044/0161-1461(2003/024).CrossRefGoogle Scholar
  51. Van Haren, R. (2010). Engaging learner diversity through learning by design. E-Learning and Digital Media, 7(3), 258–271.CrossRefGoogle Scholar
  52. van Lieshout, M., Luman, M., Buitelaar, J., Rommelse, N. N. J., & Oosterlaan, J. (2013). Does neurocognitive functioning predict future or persistence of ADHD? A systematic review. Clinical Psychology Review, 33(4), 539–560.CrossRefGoogle Scholar
  53. Wall, L. (2017). Institutional logics and curriculum decision making: enacting the Australian Curriculum English and NAPLAN literacy. The Australian Educational Researcher, 44, 391–407.CrossRefGoogle Scholar
  54. Whitehouse, A. (2012, April 23). Autisms hidden older brother specific language impairment. The Conversation. Retrieved from http://theconversation.com/autisms-hidden-older-brother-specific-language-impairment-6295.
  55. Willis, J., McGraw, K., & Graham, L. J. (2016). Strengthening senior curriculum and assessment pedagogies. Education Horizon Research Project. Available at: https://research.qut.edu.au/selb/projects/strengthening-senior-curriculum-and-assessment-pedagogies/.
  56. Wright, C., Shelton, D., & Wright, M. (2009). A contemporary review of the assessment, diagnosis and treatment of ADHD. Australian Journal of Learning Difficulties, 14(2), 199–214.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Linda J. Graham
    • 1
  • Haley Tancredi
    • 1
  • Jill Willis
    • 1
  • Kelli McGraw
    • 1
  1. 1.Faculty of EducationQueensland University of TechnologyBrisbaneAustralia

Personalised recommendations