Keywords

1 Introduction

On the flight deck, English as-a-second language (ESL) flight crewmembers (captain and first officer) use crew-alerting systems designed with an English language emphasis. Design and integration of written English language on crew alerting and information systems (e.g. Quick Reference Handbook (QRH)) should provide ESL flight crewmembers with information enabling them to read and comprehend information adequately. Written English language is the preferred language of aviation (Hutchins et al. 2006) and it is utilized by ESL flight crewmembers to read and understand normal and non-normal conditions that may occur during typical phases of flight. Therefore, ESL flight crewmember ability to read and comprehend written English language and their level of English language proficiency should be adequate. Government and academia have investigated fundamental challenges ESL flight crewmembers experience while reading and comprehending written English language on the flight deck. Some of these challenges were discussed in multiple studies conducted by the Federal Aviation Administration (FAA). These studies noted that cryptic messages on crew alerting systems have the potential to impact ESL flight crewmembers’ ability to read and comprehend information on displays (FAA 1996; 2013). It was also recommended that simplified technical English be used when designing crew-alerting systems. However, FAA studies did not elaborate on the impact written English language has (e.g. QRH information), on ESL flight crewmember performance. In other words, FAA studies should have provided evidence of written English language design and integration factors that effect ESL flight crewmembers’ reading comprehension. Aircraft accident reports reveal that written English language has contributed to several airline accidents. In 2014 an A-320 aircraft flown by ESL flight crewmembers crashed. According to the investigation report, one factor that could have contributed to the accident was flight crewmembers’ ability to read and understand written English language on a checklist. Flight crewmembers indicated they were challenged with information on the QRH checklist related to computer-reset functionality. The checklist confused flight crewmembers and did not provide them adequate understanding of the non-normal condition (KNKT 2015). In 2009, ESL flight crewmembers were involved in an A-330 aircraft accident. The accident report suggested that detailed written English language procedures may have negatively impacted flight crewmembers’ performance (BEA 2012). The Center for Investigation and Prevention of Accidents (CENIPA) investigated an aircraft accident in year 2011 related to ESL flight crewmember misunderstanding of written English language on the QRH checklist. The accident report indicated that there were many checklist inaccuracies that negatively impacted ESL flight crewmembers’ ability to read and comprehend information on the checklist (CENIPA 2013). Another aircraft accident involving a MD-83 in 2014 was related to ESL flight crewmembers inadequate response time and awareness to airspeed and written English language in the Flight Crew Operations Manual (FCOM). Information in the FCOM contained text related to anti-icing systems with ambiguous wording related to procedure protocol for responding to the aircraft condition (MCI 2016). In yet another aircraft accident involving ESL flight crewmembers, the investigation revealed that ESL flight crewmember written English language proficiency was a factor that influenced the accident. English as-a-second language flight crewmembers’ ability to understand written English language on technical documentation impacted their ability to use information to solve problems on the flight deck. Their background knowledge of the written English information on Western built aircraft negatively impacted their reading comprehension performance (IAC 2009). Each of the previous accidents discussed reveal that English language proficiency, ESL flight crewmember background knowledge, and design/integration of written English language on information systems are factors that negatively impact ESL flight crewmember performance.

Although the aforesaid accidents reveal the outcome of ESL flight crewmember linguistic challenges on the flight deck, other studies reveal factors related to ESL flight crewmember linguistic challenges. The Aerospace Industries Association (AIA) conducted a study regarding ESL flight crewmember response to propulsion failures. The AIA indicated that approximately 15% of National Transportation Safety Board (NTSB) investigations were related to ESL flight crewmember ability to adapt to written English language. The study concluded that more emphasis on improving written English language on propulsion system diagnostics is needed when utilized by ESL flight crewmembers (Sallie and Gibbons 1998). A university in China revealed that 80% of their aviation students have experienced various challenges related to written English language. Challenges were related to reading comprehension of written English language, with respect to vocabulary words on technical documentation (Wang 2011a). Likewise, Ho (1996) revealed similar issues in a study that focused on flight deck operations procedures manuals. In Ho’s (1996) study, 30% of ESL flight crewmembers did not understand written English language safety data on documentation that referenced non-normal conditions. Smith-Jackson and Wogalter (2000), revealed that warnings should be understood by individuals with different linguistic backgrounds. Wogalter et al. (1997) indicated safety information related to warnings should be read and comprehended adequately by ESL adults, especially when they perform in sociotechnical environments (e.g. flight deck). Each of the previously mentioned studies provides indication that individuals with different language backgrounds should understand crew alerts. It was also recommended that more research is needed to understand impact of ESL flight crewmember understanding of system diagnostics that contain written English language emphasis.

Other studies have indicated that ESL flight crewmembers translate information on crew alerting systems and checklists. However, flight crewmembers should not have to translate information back in to their native language, given their proficiency reading and comprehending English language is adequate. If the meaning of vocabulary words/text corpora is cognitively translated by flight crewmembers in to their native language, it could impact their understanding of the crew alert and subsequent procedures (Drury and Ma 2005). If flight crewmembers translate written English language into their native language and they misunderstand vocabulary words and/or syntax meaning, this may cause flight crewmembers to revert back to their native language and search for words compatible to read and understand written English language (Kobayashi and Rinnert 1992). These types of factors have the potential to impact ESL flight crewmember performance and flight safety. It is obvious that precursors to ESL flight crewmember misunderstanding of written English language can negatively impact their performance. These precursors are mainly design and integration of written English language, ESL flight crewmembers’ proficiency, background knowledge, and their performance related to use of written English language on the flight deck. Linguistic challenges that impact ESL flight crewmember performance on the flight deck need further investigation. Forthcoming literature provides more evidence of factors that influence ESL flight crewmember performance on the flight deck.

2 Literature Review

Discussion of previous aircraft accidents do not provide enough details to understand fundamental problems ESL flight crewmembers experienced on the flight deck. Government and academia have addressed symptoms leading to ESL flight crewmember/individual misunderstandings related to written English language. Following literature review provides and overview of many factors that influence ESL adult ability to read and comprehend written English language. Next studies provide a review of fundamental challenges ESL adults experience while reading and comprehending written English language related to vocabulary words, text corpora, translation of text, design and integration of text, and English language proficiency. It also reviews methods that ESL adults use to read and comprehend written English language such as metacognition and use of background knowledge. These methods will be further discussed in the forthcoming review of literature.

According to Hancock (1998), reading requires processing and understanding information. Lexical knowledge and skills are acquired as a result of reading information. Comprehension requires an individual to understand what they are reading and utilize strategies (metacognition) to understand syntax and apply vocabulary knowledge. (Baker and Brown 1984). Cognition is defined as ability to read and comprehend information and apply it to a contextual environment. These factors require an individual to have a level of proficiency regarding written English language. English as-a-second language adults experience difficulties reading and comprehending syntax (e.g. sentences) (Condelli and Wrigley 2006). Karbalaei (2010) indicated that ESL adult ability to read and understand written English language is predicated by strategies they use to process their ideas, and this aids in execution of their decision-making processes. There are various types of mental models that ESL adults utilize to read and understand written English language. Top down and Bottom up models (or a combination of thereof) aid ESL adults with reading and understanding written English language. Bottom up models help ESL adults understand flow of linear text. This requires ESL adults to understand letters, vocabulary words, and phrases, and then decode the sentence meaning. This model is highly dependent on ESL adult level of written English language proficiency. Top down model consists of ESL adult use of background knowledge to understand information they read. Parry (1991) studied effect of mental model use by ESL adults while they read text. Small population of ESL adults utilized bottom up strategy while reading text corpora that was complex (contained challenging vocabulary words). English as-a-second language adult English language proficiency was low or intermediate level. Goal of the study was to determine if ESL adults could identify complex words and understand word meaning. Results indicated that ESL adults were challenged by vocabulary words and often omitted words. It was also indicated that ESL adults need extra time to interpret words that they did not know. It was revealed that due to their English language proficiency levels, their ability to understand text was negatively impacted. Yildiz-Genc’s (2009) developed a study that focused on 15 ESL adults with intermediate English language proficiency. Adults read text without a time constraint and two strategies (bottom up and top down) were utilized to read text. It was noted that top down strategy was utilized by ESL adults due to their proficiency level. Adults indicated they were challenged with word meaning while reading text. They utilized sentence syntax to understand information (e.g. phrases) from previous sentences. They also utilized their background knowledge and metacognitive strategies to understand text. Use of background knowledge by an individual may be utilized to facilitate lexical inferring to understand word meaning. Parry (1991) and Yildiz-Genc’s (2009) studies reveal English language proficiency is a key factor that influences ESL adult ability to interpret words. However, their studies do not focus on vocabulary word type and the effect ESL adult ability to process and interpret word meaning. Both studies indicate metacognitive strategies are important methods ESL adults utilize to read and comprehend written English language, and some adults use these strategies based on background knowledge and English language proficiency.

Next studies review types of vocabulary words and how their characteristics impact ESL adult reading comprehension. These studies also reveal impact ESL adult background knowledge and written English language proficiency has on their reading comprehension performance. Wang (2011b) utilized 34 ESL adults with advanced English language proficiency. Wang’s study researched use of lexical inference strategy use by ESL adults when they read an article with 240 vocabulary words. Each of the adults had experience (i.e. background knowledge) with academic and technical written English language. Results indicated that ESL adults utilized collocation knowledge and word association (e.g. lexical inferencing) to read and interpret written English language. Some of the participants incorrectly interpreted words they were not accustomed to reading in the text. Dycus (1997) indicated that highly proficient adults with vast vocabulary knowledge are often correct when they make inferences on vocabulary words. Adults with low English language proficiency and low vocabulary knowledge often make incorrect inferences on vocabulary words in text. In this case, highly proficient adults experienced difficulties interpreting types vocabulary words. Wang’s (2011b) study does not corroborate Dycus (1997) finding that highly proficient adults should not make incorrect inferences on vocabulary words in text. However, in Wang’s study these results could have also been due to adult level of background knowledge. Dwaik and Shehadeh (2013) researched the impact of lexical inferencing strategy had on 60 adults with low and high written English language proficiency levels. Results indicated that adults with high written English language proficiency guessed more words correctly than adults with low written English language proficiency. Adults with low written English language proficiency were challenged by use of context clues in sentences to read and understand text. Dwaik and Shehadeh (2013) and Dycus (1997) indicated that low English language proficiency adults and low vocabulary depth often experience difficulties with their interpretation of vocabulary words. Wang’s (2011b) study revealed strategies are important factors that influence adult ability to interpret written English language and that adult English language proficiency varies and is influenced by strategy use and background knowledge.

Next studies provide an overview of how written English language design and integration has the potential to impact ESL adult reading comprehension. Simplified written English text and sentence length are factors that influence ESL adult reading comprehension performance. Mehrpour and Riazi (2004) utilized 100 adults that were proficient with their use of written English language. Half of the adults had a background in English language and the other half did not. Adults had approximately five years of English language experience. Each of the adults read technical and academic texts. The first text (i.e. medicine genre) had approximately 240 vocabulary words, while the other text (i.e. sociology genre) had 260 vocabulary words. Results indicated that shorter text was more difficult to read than longer lengths of text. Abdul-Hamid and Samuel (2012) researched the impact of ESL adult reading comprehension performance while reading two types of scientific texts and metacognitive strategy used. Adult English language proficiency was proficient or not proficient. Goal of the research study was to determine level of difficulty when ESL adults read the texts. In the first text, participants had background knowledge of half the text they read in their native language. English as-a-second language adults were familiar with 30% of the second text. First text contained approximately 590 vocabulary words, while the other text contained approximately 740 words. Results indicated long sentences in both text were difficult to read and led to re-reading text. Adult level of proficiency was a factor influencing their ability to read and comprehend each of the scientific texts. It was also indicated that ESL adults translated vocabulary words into their native language to understand texts. Kim (2006) conducted a study on the impact of ESL adult reading comprehension when they read text with abbreviations/acronyms. Adult English language proficiency was low and high levels. They had three years of English language background knowledge and translating written English language text into their native language was a common strategy used to read and comprehend English language. Results indicated that acronyms/abbreviations were difficult for ESL adults to read and interpret because of different word meanings that are commonly used in ESL adult native language. You (2009) investigated the impact of ESL adult ability to read information on computer screens and on paper format. There were 120 ESL adults that participated in the study that had background knowledge in using English language. Two texts were utilized for the experiment. One text was familiar to the adults, while the other text was not. Adult English language proficiency was high, medium, and low. Each text contained 340 words. Paper format text allowed for more space for lines of text versus the computer screen format, which allowed for less text. Results indicated that adults performed better while reading text from the paper than from the computer screen. This was likely due to ESL adult ability to highlight information on paper and use other strategies (e.g. re-reading text) to read written English language on paper format. Adults did not use many strategies to read information on computers screens. But, adults indicated they were comfortable with their background knowledge while reading and comprehending information on the computer screen. Adults with low English language proficiency level reading performance was not adequate when they read information from paper and computer screen formats. On the other hand, medium and high proficiency level adults performed well when they read and interpreted information in each of the text formats. Mehrpour and Riazi (2004) and Abdul-Hamid and Samuel (2012) studies revealed that the type of text adults read, English language proficiency level, level of background knowledge, strategy use, and short versus long strings of text influences adult reading comprehension performance. Kim (2006) and You (2009) indicated that acronyms and abbreviations have an impact on adult performance depending on their background knowledge of the long form (e.g. HYD versus Hydraulic). Both authors reveal that information on a screen versus information on paper has an impact on adult performance. This is due to type of metacognitive strategy utilized by adults, their English language proficiency, and background knowledge. Although vocabulary word type, text type, adult English language proficiency, background knowledge, and strategy use are factors that influence adult performance while reading and comprehending written English text, there are adults that cognitively translate information into their native language to have a better understanding of written English language. On the other hand, translation of text by a translator also reveals important details as well.

Forthcoming studies review translation of text and impact on ESL adult reading comprehension performance.

Translation of written English language can occur in two different ways: unilateral translation of written English language text or translation of written English language text into ESL native language by a translator). Each type of translation has the potential to impact ESL adult ability to read and understand written English language. According to Ogilvie (1984), due to various complexities in written English language, it may not be appropriate to translate text into an adults’ native language. Zhao (2015) conducted a research study with 15 ESL adults, and investigated the impact of translating written English language into adult native lexis. Results indicated that adults that have adequate background knowledge of their native language are better equipped to read and comprehend a second language. It was also indicated that ESL adult inability to comprehend vocabulary words that were translated was due to adult ability to understand and translate word meaning into ESL adult native language. Ynfiesta et al. (2013) developed a study to determine impact of translating written English language acronyms used in technical text into a different another language. An experienced translator performed the translation task. It was determined that background knowledge of the long form acronym in written English language is essential, so that there are no misunderstandings of word meaning in ESL adult native language. The translator often searched for words that had equivalent meaning in another native language. It was noted that aforesaid factors have the potential to impact on ESL adult reading comprehension performance. Barani and Karimnia (2014) studied the impact of written English language translated into 32 ESL adults’ native language (e.g. Persian language). Goal of the study was determine strategies used to read text that was translated from English language into their native language. Results indicated participants’ background knowledge in the text genre (i.e. scientific text) enabled them to understand text. They used several metacognitive strategies (i.e. unilateral translation) to understand the text that was translated from written English language into Persian lexis. Zhao (2015) and Ynfiesta et al. (2013) indicated that translation of written English language words into adult native language impact reading comprehension performance. Zhao (2015) provided evidence that background knowledge of vocabulary words in adult native language can help them understand a second language. However, in each study adult English language proficiency was not reviewed. In previous studies, English language proficiency was noted as a factor that impacts adult understanding of vocabulary words and their meanings. In Yinefista et al. (2013) study it was revealed that information translated by a translator can impact adult performance. This is due to background knowledge of the translator and his/her ability to connect the translated word meaning to ensure that the reader understood it. Barani and Karimnia (2014) corroborated Zhao (2015) and Ynfiesta et al. (2013) studies regarding the need for background knowledge to understand text translated from written English language text to adult native language.

Overall, the literature review provides evidence of factors that influence ESL adult ability to understand written English language. Vocabulary words, text genre, adult English language proficiency, background knowledge, and strategy use impact adult performance. Unilateral cognitive translation of written English language into adult native language reveals challenges. Likewise, having a translator translate information from written English language into adult native language requires background knowledge of different types of text, vocabulary words and their meanings. In the context of ESL flight crewmembers use of procedures and crew alerts utilization on the flight deck, do flight crewmembers experience performance challenges while reading and comprehending written English language on the flight deck? Do ESL flight crewmembers’ linguistic challenges impact flight safety? It is hypothesized that there will be a statistically significant difference and interaction between ESL flight crewmember reading comprehension proficiency and performance when they read and comprehend written English language on QRH checklists and ECAM system, and written English language on QRH checklists translated into ESL flight crewmembers’ native language.

3 Methods

Thirty male ESL flight crewmembers from Portugal with air transport pilot ratings that currently fly aircraft for an airline were utilized for the study. Each flight crewmember had experience flying several Airbus aircraft types (i.e. A319). Flight crewmembers’ native language was Portuguese. Flight crewmembers English language was learned through formal schooling (i.e. high school) and this was considered background knowledge. Flight crewmembers’ International Civil Aviation Organization (ICAO) English language proficiency levels were utilized as their background knowledge using English language. Flight crewmembers ICAO English language Proficiency Rating (ELPRs) met the minimum level four operational. Level four operational indicates that flight crewmembers have adequate use of English language (e.g. speaking/listening). The ELPRs provide the reader with an understanding of flight crewmembers background knowledge of English language. Flight crewmembers rated their Reading Comprehension Level (RCL). Ratings were utilized to describe flight crewmembers’ proficiency ratings. Questionnaires were provided to each flight crewmember asking them to rate their proficiency, when they read and comprehend written English language on crew alerting systems and QRH checklists. Proficiency ratings were determined to be high level or medium level. High-level proficiency rating indicated flight crewmembers understood written English language vocabulary words, while medium-level proficiency rating indicated flight crewmembers experienced difficulties with use of vocabulary words/word meaning. Proficiency levels were utilized to determine flight crewmember extent of reading and interpreting English language, and if differences exist between flight crewmembers proficiency levels. A within subjects experimental design was developed and contained independent variable (IV)-language and dependent variables (DV)-response time, and National Aeronautics Space Administration (NASA) Task Load Index (TLX) workload scores. English as-a-second language flight crewmember response time was measured with a stopwatch, starting with the outset of the alert and time was stopped when the trial was complete. Electrical and hydraulic alerts from the A-320 aircraft Electronic Centralized Aircraft Monitor (ECAM) were utilized in the experiment. The NASA TLX workload scores were recorded post task completion for each system fault. Experimental trials lasted for sixty minutes. Each flight crewmember piloted an A-320 flight deck for 30 min while the researcher injected faults (electrical and hydraulic) during cruise phase of flight. Last thirty minutes was allocated for post interview discussion with each flight crewmember. Prior to the start of the trials, the researcher evaluated written English language text on written English language ECAM and QRH checklists, to determine text genre and vocabulary word types. Written English language on the QRH checklists were translated into Portuguese language by translators at the airline. More details on the translation method will be provided in a forthcoming section.

Limitations.

Written English language vocabulary words and text genre from electrical and hydraulic ECAM and QRH checklists were utilized for the study. Flight crewmember use of different written English language on ECAM and QRH checklists (e.g. pneumatic system) may have impacted their performance differently. Translation of QRH checklists into a different lexis (e.g. Chinese) may have impacted flight crewmember performance. Flight crewmember background knowledge of information, English language proficiency, use of metacognitive strategies may have also impacted their performance while using different crew alerting and information systems.

Table 1 is a review four specific hypotheses. The format of the hypotheses is as follows: Hypothesis, condition, and null hypothesis.

Table 1. Listed and described hypotheses tested
Full size table

Text corpora on ECAM and QRH checklists were evaluated prior to the experimental trials. As the literature review indicated, several vocabulary word types and text genre can be found in text corpora. An evaluation of text prior to flight crewmembers participation in the experimental trials was conducted. Translators, with experience in translation methods translated written English language QRH checklists (i.e. hydraulic and electrical system) into Portuguese language. Abbreviations, phrases, and acronyms were not translated if there was no equivalent meaning in Portuguese language. Previously mentioned, it is important to be aware that translation can impact adult understanding of word meaning and cause interpretation issues. Written English language ECAM system and QRH checklists text were not altered. In other words, authentic text was utilized for the study, certified by the airline. Texts were not simplified, word tokens were unchanged, and sentence length was not manipulated. If the ECAM system and QRH checklists had been altered prior to the study, results may be different. Fonts, and word case tense was unchanged from its original format. Text genre on the ECAM system and QRH checklists contained technical information with several different vocabulary word types. Furthermore, text contained expository and instructional information. Researcher utilized authority references such General Service List of English Words (GSLEW), Academic Word List (AWL), and the A-320 Flight Crew Training Manual (FCTM), ECAM system manual to evaluate written English language ECAM texts and QRH checklists texts. Some of the authority references contained technical/scientific, sub-technical, non-technical, and acronyms/abbreviations/long form word types, which were also found on the ECAM and QRH checklists. Each word on the ECAM and QRH checklists was mapped to the authority references. Results indicated a high percentage of high frequency words and several occurrences of words from the AWL and GSLEW lists. There were a small number of low frequency words and many sub-technical and technical words found on the ECAM and QRH checklists. Since written English language (in general) contains many words found on GSLEW and AWL lists Coxhead (1998) and West (1953), participants with a background in written English language and adequate English language proficiency may benefit from such words found on the ECAM and QRH checklists. Regarding technical words/acronyms/phrases, there were many of these types of words on each of texts. As Coady and Huckin (1997), Chung and Nation (2004) indicated, technical words are required to be known by ESL adults based on their training and background knowledge of the technical field. Technical vocabulary has the potential to cause difficulties with ESL adult interpretation when reading text that is considered technical. It was also indicated that their proficiency is a key factor that influences their ability to read and interpret technical information. Regarding text layout, ECAM and QRH checklists had different layouts with respect to data presentation. As previously stated, abbreviations, acronyms, and phrases appeared differently in format, with respect to ECAM and QRH checklists. As indicated by Hartley (1994), abbreviations and acronyms should be designed adequately so that technical information on checklists may be followed by ESL flight crewmembers, and thus allowing them to respond effectively to an alert. According to Dyson (2004), configuration of data may impact reading comprehension of information on paper. Configuration of data can also impact ESL flight crewmembers information processing on displayed crew alerts. Inter-rater reliability analyses were conducted to ensure there was no bias with categorizing the previously mentioned vocabulary words on the written English language ECAM and QRH checklists. Cohen’s Kappa coefficient was k = 0.57 for the ECAM electrical system and k = 1 for the electrical system QRH checklist. Cohen’s Kappa coefficient was k = .55 ECAM hydraulic system and k = 1 QRH hydraulic system checklist.

As the literature review indicated, it is essential to follow a methodical approach when translating information from one language to another. Authentic written English language selected QRH checklists (electrical and hydraulics) were translated from written English language into Portuguese lexis. Translation process lasted for one week and was conducted with two experienced translators. Both translators were ESL senior airline flight crewmembers whose first language was Portuguese. Each of the flight crewmembers rated their English language proficiency as high level. Following 14-step process was utilized to translate the texts (Table 2).

Table 2. Translation process
Full size table

4 Results

Descriptive statistics indicated that the average age of flight crewmembers was 47 years and the minimum age was 27 years. Flight crewmembers’ average airline years of experience was 24 years. Paired samples correlation test indicated mean response times from the written English language ECAM/written English language QRH checklists score was faster (M = 8.75; SD = 3.811) than participant response time on the Portuguese checklists (M = 14.4; SD = 4.730). The paired samples correlation value indicated a negative correlation (−.075), inverse relationship between participant response times when they utilize written English language ECAM/written English language QRH checklist and written English language ECAM/written Portuguese language QRH checklist. In other words, when participants use written English language ECAM/written Portuguese language QRH checklist to respond to hydraulic and electrical system malfunctions, they tend to have longer response times than with use of written English language ECAM/written English language QRH checklists. Significance value was (Sig p = .695). Since p > .05, this is an indication of no significant correlation. Paired samples t-test found a significant difference between participant response times when they use written English language ECAM/written English language QRH checklists and written English language ECAM/written Portuguese language QRH checklists. The results indicated t(29) = −4.947; Sig 1-tailed p = 0 and Sig 2-tailed p = .000; p < .05, d = −.132 (means are insufficient), the researcher accepts the alternative hypothesis (HA) that there is a significant difference between participant response times when they use written English language ECAM/written English language QRH checklists, and written English language ECAM/written Portuguese language QRH checklists when participants respond to electrical and hydraulic system malfunctions. Participant response times with use of written English language ECAM/written Portuguese language QRH checklists was slow and their response time using written English ECAM/written English language QRH checklists was fast. A paired samples correlation was performed to determine if there would be a correlation between participants NASA TLX workload scores when they utilize the written English language ECAM/written English language QRH checklists/written English language ECAM/written Portuguese language QRH checklists. Results indicated that mean participant NASA TLX workload score from the written English language ECAM/written English language QRH checklists score was (M = 34; SD = 17.777), which was lower than participants NASA TLX workload score on the Portuguese checklists (M = 50; SD = 23.163). The correlation value was .362, indicating a positive correlation between the two variables (English language/Portuguese language). This is an indication that when participants utilized written ECAM written English language/written English language QRH checklists their NASA TLX workload scores tend to move in a positive direction, and when participants utilized ECAM written English language/written Portuguese language QRH checklists their NASA TLX workload scores tends to move in the positive direction. The paired samples correlation test indicated a significant correlation between participant NASA TLX workload scores when they use written English language ECAM/written English language QRH checklist and written English language ECAM/written Portuguese language QRH checklist. The significance value for this analysis was p = .049, (p < .05) and the means are insufficient. This is an indication that there is a significant relationship between the aforesaid variables (English language/Portuguese language). Regarding the paired samples t-test, the researcher performed a one tailed and two-tailed test and found a significant difference (both tests) between participant NASA TLX workload scores when they use written English language ECAM/written English language QRH checklists, and their NASA TLX workload scores when they use written English language ECAM/written Portuguese language QRH checklists. The values are as follows: t (29) = −3.803, (Sig. 1-tailed = .0005; 2-tailed p = .001) (p < .05), d = −0.78. Therefore, researcher accepts the alternative hypothesis (HA) that there is a significant difference between participant written English language workload scores and Portuguese language workload scores, when participants respond to electrical and hydraulic system malfunctions. Participant use of written English language ECAM/written Portuguese language QRH checklists was more difficult than using written English ECAM/written English language QRH checklists. A Pearson product moment (Pearson’s r) correlation test was performed to determine if a significant positive correlation exists between participant NASA TLX workload scores (ECAM written English language/written Portuguese language QRH checklists) and participant response time (ECAM written English language/written Portuguese language QRH checklists). Recall, participant NASA TLX workload scores were (M = 50; SD = 23.163) and response time was (M = 14; SD = 4.730) (higher workload scores and response times were observed when participants utilized ECAM written English language/Portuguese language QRH checklists, compared to their use of ECAM written English language/English language QRH checklists. Pearson correlation value was r = .158 which indicates a minimal positive correlation. This result indicates as participant NASA TLX workload scores increase so does their response time to hydraulic and electrical system malfunctions. The significance value was p = .404 (p > .05), d = 2.15. These results indicated no significant correlation between participant NASA TLX workload scores (ECAM written English language/written Portuguese language QRH checklists) and participant response time (ECAM written English language/written Portuguese language QRH checklists). The evidence suggests that the correlation observed is not generalizable to the population of ESL flight crewmembers. The researcher accepts the null hypothesis (HO) that no significant positive correlation exists between participant NASA TLX workload scores (ECAM written English language/written Portuguese language QRH checklists) and participant response time (ECAM written English language/written Portuguese language QRH checklists).

A Pearson product moment (Pearson’s r) correlation test was performed to determine if a correlation exists between participant use of ECAM written English language/written English language QRH checklists and their NASA TLX workload scores, and their use of ECAM/written English language/written English language QRH checklists response times. Recall, participant ECAM written English language/written English language QRH checklists NASA TLX workload scores mean was (M = 34; SD = 17.777) and ECAM written English language/written QRH checklists response times was (M = 8.75; SD = 3.811) (lower workload and lower response time observed when participant utilized ECAM written English language/written English language QRH checklists, compared to their use of ECAM written English language/Portuguese language QRH checklists). The Pearson correlation value was r = .150 which indicates a minimal positive correlation. This result indicates as participant NASA TLX workload scores decrease so does their response time to electrical and hydraulic system faults. However, the significance value was p = .428 (p > .05), d = 1.96. These results indicated no significant positive correlation between participant NASA TLX workload scores (ECAM written English language/written English language QRH checklists) and participant response time (ECAM written English language/written English language QRH checklists). The evidence suggests that the correlation observed is not generalizable to the population of ESL flight crewmembers. The researcher accepts the null hypothesis (HO) that no significant positive correlation exists between participant NASA TLX workload scores (ECAM written English language/written English language QRH checklists) and participant response time (ECAM written English language/written English language QRH checklists). Researcher developed hypotheses and corresponding two-way ANOVAs (between- subjects design) to determine effect of participant English language proficiency, airline years of experience, and impact on their reaction time/NASA TLX workload scores (Table 3).

Table 3. Two-way ANOVAs between subjects hypotheses
Full size table

No significant main effect and interaction were observed between participant airline experience, proficiency, and reaction time when they read and comprehend the written English language on crew alerting systems and QRH checklists. Results indicated F (1, 26) = .003, p > .05, partial η2 = .000. Participant airline experience less than 20 years, high level proficiency participants reaction time mean was M = 7.63; SD = 2.26. Participant reaction time mean for medium level proficiency participants was M = 9.00; SD = 0. Participants with high-level proficiency reaction time were faster than medium level proficiency participants. Results also indicated F (1, 26) = .046, p > .05; partial η2 = .002. Participant airline years of experience 20 years or greater and high level proficiency revealed their reaction time was M = 9.62; SD = 4.66. Participants with medium level proficiency indicated M = 7.25; SD = 1.32. Participants with high-level proficiency had a longer reaction time than participants with medium proficiency level. Researcher accepts the null hypothesis. No significant main effect and interaction were observed between participant years of experience, proficiency, and NASA TLX workload scores when they read and comprehend written English language on crew alerting systems and QRH checklists. Results indicated F (1, 26) = .028, p > .05, partial η2 = .001. Participants with less than 20 years of experience high level proficiency NASA TLX workload scores indicated M = 40.26; SD = 18.96. Medium level proficiency participants NASA TLX workload scores were M = 15.00; SD = 0. Participants with less than 20 airline years of experience high-level proficiency had higher NASA TLX workload scores than medium level proficiency participants. Results also indicated F (1, 26) = 2.86, p > .05; η2 = .099. Participant airline experience 20 years or greater and high level proficiency indicated their NASA TLX workload scores M = 34.66; SD = 17.21. Participants medium level proficiency participants, M = 24.15; SD = 16.7. High-level proficiency participants with 20 years of experience or greater had higher workload scores than participants with medium level proficiency. The researcher accepts the null hypothesis.

5 Discussion

With respect to written English language on the ECAM and QRH checklists, participants’ mean response times revealed they responded more quickly to electrical and hydraulic system faults than when they utilized English language translated into Portuguese language on QRH checklists. All participants had background knowledge reading and interpreting written English language. They also had experience with use of technical information on the flight deck while responding to non-normal conditions (i.e. system faults). Participants had experience with reading and comprehending information on different ECAM systems and QRH checklists. This enabled them to have an understanding of how written English language text was designed and integrated on the ECAM and QRH checklists. Participants indicated they responded quickly to alerts and use of written English language checklist because they were accustomed to the English language. It was noted that participants are trained on how to use technical information while responding to a system fault. Many of them indicated they have encountered non-normal conditions while flying aircrafts at their airline, and they are trained to understand written English language logic on ECAM and QRH checklists to ensure their response time is effective. During experimental trials, the researcher observed most of the participants responding to the system faults very quickly and with precision, with respect to following published QRH checklist procedures. Moreover, participants did not indicate issues with their use of written English language on the ECAM system. Technical information on the ECAM system and QRH checklists (abbreviations and acronyms) were familiar to many participants. Park’s et al. (2014) study revealed that less time is utilized to read and comprehend acronyms, if ESL adults have sufficient amount of background knowledge of the acronyms in text. If longer response times are needed to process information such as acronyms/abbreviations on a display, it could impact their ability to solve time critical system/aircraft problems. As the researcher did not regulate a time limit to complete each task, this could also be a reason that participant response time was fast when they responded to electrical and hydraulic system faults. Park’s et al. (2014) study also provided an indication that temporal demand on ESL adults was not regulated when they read written English language text. Regarding participant English language proficiency and metacognitive strategy use in the researcher’s study, participants had high and medium levels of English language proficiency and they used QRH checklist references (published FCOM procedure text) to assist them with responding to electrical and hydraulic system faults. As Park’s et al. (2014) revealed, metacognitive strategy use such as referencing other sources is typical of ESL adults that have high level of English language proficiency. The researcher’s findings support Park’s et al. (2014) study. It was noted in the profiling of text exercise, there were many high frequency words (GSLEW) as well as academic words (AWL), small number of low frequency words, and sub technical/scientific acronyms/abbreviations. Previously discussed, written English language contains many high frequency words and they are more comprehensible due to their frequency in text (Nation 2001). Academic words were developed to catalog most frequently occurring words in academic text, and they assist learners of English-a-second language, with respect to their reading comprehension (Coxhead 1998). As participants had background knowledge of English language through different types of instructional learning, this could have prepared them for reading and understanding written English language. It should also be noted that the participants received written English language training in classes where there were different pedagogical approaches to teaching English language. This could also be a factor that influenced their ability to read and understand the language. Researcher’s findings support Wanpen’s et al. (2013) study, which indicated that taking courses in an English language curriculum helps facilitate reading comprehension of written English language. Participants also noted that since they were accustomed to written English language, they were able to use various strategies like decoding words, and re-reading words to help them through the reading comprehension process. Researcher’s findings support Dwaik and Shehadeh (2013) and Nylander’s (2014) studies, with respect to decoding vocabulary words (lexical inferencing) and participant English language proficiency.

Participants indicated they did not have background knowledge of written English language text translated into Portuguese language on QRH checklists. Participants indicated they often unilaterally translate vocabulary words into their native language (Portuguese), and that translation process occurs mostly under non-normal conditions. But, they do not translate every word on QRH checklists. It was noted, that translation processes occur if they have background knowledge of the English language vocabulary word/sentence in Portuguese language. As the airline indicated, it receives published/certified QRH checklists from the manufacturer that do not contain any changes to text. Portuguese flight crewmembers also indicated they are trained on text that appears on QRH checklists, which is provided to them by the manufacturer. Regarding participant’ response time when they utilized Portuguese language on QRH checklists, their response time was slow. This could be due to participants’ lack of background knowledge of translated text, and it could be that, they were aware of particular vocabulary words that had the same meaning in Portuguese language. Participants indicated they re-read text due to uncertainties with word meaning in the translated text, monitored their reading speed due to their desire to make correct inferences on each word/sentence, and decoded words such as abbreviations/acronyms and other vocabulary words in the text. On the other hand, there were participants that read and comprehended Portuguese language text with ease, as they were familiar with text translated into Portuguese language that had an equivalent meaning. It was noted that aforesaid strategies used to read and comprehend Portuguese language slowed their response time to electrical and hydraulic system faults. On the contrary, they were comfortable with the time they spent reading and comprehending text, so that they would not make incorrect inputs on the flight deck. They were concerned if they read the text too fast, they would miss a word or omit information, which could also lead to long response times. Hutchins et al. (2006) and Drury and Ma (2005) indicated that translation of written English language has the potential to impact ESL adults reading comprehension. It was also noted by Al-Sohbani and Muthanna (2013) that participants must have background knowledge of written English language, so that they may adequately understand translated language. They must also have adequate English language proficiency. As most participants indicated, they had background knowledge of abbreviations/acronyms on crew alerting systems and QRH checklists. There were some participants that indicated abbreviations/acronyms long form was difficult to understand in English language. This could have negatively impacted their ability to understand English language translated into Portuguese language on QRH checklists. In Al-Sohbani and Muthanna (2013) study, participants did not have adequate knowledge of written English acronyms and abbreviations, and when acronyms and abbreviations were translated into their native language, they were difficult to read and understand word meaning. In the researcher’s study, participants had adequate background knowledge and adequate English language proficiency when they use of written English language on crew alerting systems and QRH checklists. It is peculiar as to why their response time was longer on the written Portuguese language checklists than when they read and comprehended information on written English language crew alerting systems/QRH checklists. Throughout the researcher’s experiment, participants often utilized metacognitive strategies to read and interpret Portuguese language (i.e. re-read sentences). They cognitively translated (unilaterally) Portuguese language into different vocabulary words to attain word meaning, and they also reverted back to using written English language. When participants re-translated Portuguese language text to attain other forms of vocabulary words in Portuguese language, this was most likely due to their misunderstandings of sentence syntax. They also reverted back to use of cognitive mental model of written English language on QRH checklists. According to Kobayashi and Rinnert (1992), reverting back to English language can occur because an ESL adult lacks understanding of translated syntax meaning. This behavior by ESL individuals can result in inappropriate translation of technical information back into their native language. Evidence from Barani and Karimnia’s (2014) study suggested that many participants used metacognitive strategies such as re-read sentences and paraphrase words while they read English language text. It was indicated that they utilized these strategies for problem solving purposes, which were related to difficulties understanding word meaning. Part of Barani and Karimnia’s (2014) study was corroborated in the researcher’s study. The researcher found that participants re-read sentences to understand word meaning. Therefore, Portuguese language used on QRH checklists can be considered difficult to read and understand word meaning, if participants are accustomed to using written English language. Lexical inferencing was also utilized to guess word meaning due to participants’ inadequate background knowledge. This led to long response times, inadequate educated guesses to vocabulary word meanings, and inadequate responses on the flight deck to non-normal conditions (i.e. electrical and hydraulic faults). As participants’ English language proficiency was adequate (high and medium levels), it is peculiar as to why they did not understand the meaning some abbreviations and acronyms in the notes section of the QRH checklist. Flight safety was also negatively impacted when participants utilized Portuguese language to solve electrical and hydraulic faults. It was indicate that long response times impacted their ability to recover the aircraft from electrical and hydraulic faults. Fault recovery technique was negatively impacted and thus other un-related to the fault, routine tasks (normal conditions) were abandoned due to difficulties with reading and understanding the Portuguese translated checklists. Design and integration of written English language vocabulary word types are predicated on the fact that participants must have background knowledge on these types of words. When written English language words were translated into Portuguese language, it negatively impacted interpretation of information in Portuguese language. As ESL flight crewmembers indicated they unilaterally translate written English language into their native language, it was obvious to the researcher to translate English language into their native language, therefore making it easier for ESL flight crewmembers to read and comprehend text on the ECAM and QRH checklists, in the researcher’s experiment. Considering these factors, the researcher expected to find a significant positive correlation between participants NASA TLX workload scores and their response time when they read and comprehend technical information on the ECAM (written English language) Portuguese language QRH checklists. This outcome was likely due to participant’s lack of background knowledge with QRH checklists translated into their native language, and due to their English language proficiency and metacognitive strategies utilized to read and comprehend information on the written Portuguese language QRH checklists. The researcher expected to find a positive correlation between participant NASA TLX workload scores and their response time when they read and comprehend technical information on ECAM (written English language) written English language QRH checklists. However, there was not a significant positive correlation between the two variables. Therefore, the data is not generalizable to the population of ESL flight crewmembers. As previously discussed, this outcome was likely due to participant’s minimal difficulty they experienced while using written English language on the ECAM and QRH checklists. Their background knowledge, English language proficiency, and metacognitive strategies enabled them to perform well. Two-way ANOVA analysis revealed no significant main effect and interaction observed between ESL participant years of experience and English language proficiency and their reaction time, when the read and comprehend written English language on crew alerting systems and QRH checklists. This is an opposite finding from the researcher’s expectations. However, there are a number of factors that help explain these results. First of all, participants had a range of airline experience levels and experience related to background knowledge reading and comprehending written English language on crew alerting systems and QRH checklists. They were familiar with design and integration of written English language on crew alerting systems and QRH checklists. Participant familiarity with written English language design and integration on crew alerting systems and QRH checklists enabled them ability to understand text during the experimental trials. Second, there were participants that utilized metacognitive strategies to read and understand written English language. This may have helped them process information adequately during the experimental trials. Participant proficiency levels were adequate, and this could have also impacted their performance. As the researcher separated participant airline experience into two levels (20 years or greater versus less than 20 years), having less than 20 years of airline experience with high level of proficiency resulted in faster response times to crew alerts. On the other hand, there were some participants that had a long response time to crew alerts with medium level proficiency. Participants with 20 years of experience and greater with high level of English language proficiency responded slower to crew alerts than medium level participants. Participant number of airline years of experience does not appear to be a factor with a significant main effect on participant reaction time. Perhaps, background knowledge and training may be more efficient variables to research without specific numerical value focus (i.e. less than 20 years of airline experience, 20 years or greater of airline experience) in future research. As this experiment measured flight crewmember performance that were Portuguese natives, it would seem practical to test other flight crewmembers that have an array of linguistic backgrounds. Results could be different if testing participants with other linguistic backgrounds (e.g. Mandarin) during experimental trials, and may convey an interaction between aforesaid variables. Literature review indicated high/medium proficiency level participants use different strategies to read and comprehend written English language. There were participants that indicated they were highly proficient with reading and comprehending written English language, and aware of strategies to use while reading and comprehending written English language. They also indicated they were challenged with terminology on crew alerting system and QRH checklists. As Yildiz-Genc (2009) indicated, background knowledge and English language proficiency is a factor that influences ESL adults’ ability to read and comprehend written English language. In the researcher’s experimental study, participant proficiency levels were high and medium and they had adequate background knowledge in the text they read and comprehended during the trials. Therefore, this finding corroborates Yildiz-Genc (2009) finding that differences with participant English language proficiency are expected when they read and comprehend written English information. If the researcher had imposed a time limitation on the trials, the results may have been different. As Hashemi and Bagheri’s (2014) study indicated, no time limit resulted in better comprehension of texts, whereas a time limit had a negative impact on performance. The researcher’s finding corroborates Hashemi and Bagheri’s (2014) study. Second two-way ANOVA also indicated no significant main effect and interaction between participant English language proficiency and NASA TLX workload scores, when they read and comprehend information on crew alerting systems and QRH checklists. Crew alerting systems and QRH checklists that were analyzed contained text genre that was technical/scientific and text corpora contained high number of high frequency words and academic words, this likely had an positive effect on flight crewmember ability to read and understand text on crew alerting systems and QRH checklists. Coxhead (1998) and West (1953) indicated that high frequency words and academic words in text have a higher comprehensibility than other words (e.g. low frequency). Participants in the researcher’s study had background knowledge, years of experience, and training with technical words on crew alerting systems and QRH checklists. This likely reduced participant cognitive workload, enabled them to recognize, read and comprehend technical words, while perform tasks during non-normal conditions. Wanpen et al. (2013) study indicated that participant technical vocabulary knowledge helped participants with reading text. As Mehrpour and Riazi (2004) indicated, high proficiency, background knowledge in text is important when reading and comprehending different words in text corpora. As the researcher did not alter sentence length or simplify text (text was authentic), this could be the reason why participants performed well reading and comprehending written English language text on crew alerting systems and QRH checklists. On the other hand, there were participants that experience higher cognitive workload compared to other participants. This could be due to participants with high proficiency using metacognitive strategies.

6 Conclusion

Written English language on the ECAM system and associated QRH checklists did not have a substantial negative impact on ESL flight crewmembers’ performance. But, other languages should be investigated to determine if this is an expectation of other regions, and flight crewmembers with different linguistic backgrounds across the globe. In other words, is the issue of written English language still a factor in other regions of the globe? Since the researcher’s experiment focused on one region, other regions should be investigated as well. On the other hand, since translating English language into flight crewmembers’ native language was an issue that impacted their performance, other regions and languages of flight crewmembers should be included in future research studies.