Advertisement

Advances in Neurodevelopmental Disorders

, Volume 2, Issue 2, pp 199–205 | Cite as

A Cloud-Supported App for Providing Self-Directed, Localized Job Interest Assessment and Analysis for People with Intellectual Disability

  • Daniel K. Davies
  • Steven E. Stock
  • Cameron D. Davies
  • Michael L. Wehmeyer
ORIGINAL PAPER

Abstract

Obtaining meaningful, fulfilling employment is important to the enhanced quality of life of all people. Unfortunately, data show that people with disabilities, in general, achieve much poorer employment outcomes than do people without disabilities, and that people with intellectual disability have even less positive employment outcomes. There are many reasons for this, but at least one factor may be limitations to the ways of determining the job and vocational interests and work preferences of people with intellectual disability. This study examined the use of a cognitively accessible, iPad-based app that provided multiple opportunities for people with intellectual disability to view videos about potential work contexts and job tasks and to identify preferred work options. Participants in this study were able to use the app with minimal prompting and were able to identify job and work interests and preferences that matched those identified for them by vocational counselors.

Keywords

Employment Intellectual disability MyJobQuest 

Work has multiple positive benefits to the quality of life of all people, including providing income, opportunities for the formation of relationships, enhanced self-esteem, and a sense of purpose in life, so much so that people often build a sense of identity based upon their jobs. For people with intellectual disability, however, their identity, both self- and other-perceived, is often overshadowed by their disability. In addition to the benefits from work for all people, meaningful and inclusive work provides a significant opportunity for people with intellectual disability to feel that they are more like other people than unlike them.

The US Department of Labor Statistics reports that only 17.1% of Americans with disabilities were employed as of 2014, compared to 64.6% of Americans who do not have a disability (Bureau of Labor Statistics 2015). Among workers with a disability, 33% worked part time in 2014, compared to 18% of workers without a disability. This represents not only a significant cost to government and agency assistance services but also a tremendous personal cost to people with disabilities in the terms mentioned above—lost income, unfound relationships, and a sense of identity undermined by dependency. As poor as employment outcomes for people with disabilities in general are, they are even less positive for people with intellectual disability. A survey conducted by The Arc of the United States in 2010 found that 85% of people with intellectual disability were not employed at all, and only 10% were employed in a regular job in the community or were self-employed (The Arc of the United States 2011). A 2012 survey by Cornell University found that only 11.4% of non-institutionalized, working-age people with cognitive disabilities were engaged in full-time employment and that 34.4% of non-institutionalized working-age people aged 21–64 with cognitive disabilities were living at or below the poverty rate for the USA, the highest percent for any disability group (Cornell University 2012).

There is, obviously, a critical need to support people with intellectual disability to obtain meaningful employment. One key to the success of any employment effort lies in the quality of the job match. This involves assessing the vocational interests, skills, and needs of the prospective worker and matching these to an appropriate job to increase the likelihood of successful employment. The importance of preference indication and self-determination in job planning for people with intellectual disability has been acknowledged for a long time (Wehman 1992; Wehmeyer and Metzler 1995). These early indications of the importance of choice and self-determination to more positive employment-related outcomes for people with intellectual disability have been bolstered by more recent research. Wehmeyer et al. (2012) conducted a randomized controlled study of the effect of interventions to promote the self-determination of high school students with cognitive disabilities and determined that students who received interventions to promote skills related to self-determination showed significant growth in their self-determination, and at higher levels than students with disabilities who did not receive such interventions. In a follow-up study of the students from Wehmeyer et al. (2012), Shogren et al. (2015) found that self-determination status at the end of high school predicted significantly more positive employment and community access outcomes for youth with disabilities.

There are a limited number of strategies to determine the preferences of people with intellectual disability regarding work and employment. There are many career interest tools that have been developed to identify job preferences. However, a major problem with most career interest assessment tools is their reliance on reading and higher-level cognitive processing to complete, which limit their utility for people with intellectual disability. Non-text-based interest inventory tools may address the issue of literacy but can be limited in their effectiveness due to the static nature of their job representations; that is, they convey little information about the nature of the job to the job seeker. One may view a picture of a laundry and think that the tasks look appealing, without understanding how hot or loud such environments can be. Further, the high level of administrative support required by these tools limits opportunity for self-direction and unbiased self-expression for people with intellectual disability.

Several strategies that involve observing people with intellectual disability in actual work environments and contexts have been shown to be effective in the area of job matching. For example, Parsons et al. (2001) showed that situational assessments of three people with intellectual disability in real-world work contexts, with the recording of indices of happiness or unhappiness, helped determine preferred work activities and tasks. While such procedures may be the best way to determine preferences for some people with more extensive support needs, they are, obviously, exceedingly labor intensive and limited in the number and type of work places that can be visited.

Technology has the potential to address several limitations in the area of job preference assessment. Technologies can be developed to overcome limitations in literacy or cognitive functioning through universal design features, and the use of video and audio can more closely simulate typical work environments. Stock et al. (2003) conducted an evaluation of a Windows PC-based software program with 22 transition-age students and adults with intellectual disability receiving employment- and transition-related supports. Researchers compared job interest categories identified by school and employment agency support personnel for each participant from their knowledge of the student/adult. Afterward, participants used the PC software-based system to identify job interests on their own. Results showed that areas of job interests from the PC-based assessment process correlated significantly with professionals’ predictions of most desired jobs, suggesting that people with intellectual disability could reliability self-assess job interests if given adequate supports.

The MyJobQuest (MJQ) system described in this study is an interactive multimedia-rich tablet-based app that presents career choices to people via an interface that does not require reading skills or assistance from others to complete, thus providing a truly self-determined opportunity for personal expression of career choices. It utilizes features that facilitate self-directed interest assessment that were not available in the PC-based system evaluated by Stock et al. (2003).

The objectives of this study were to establish the technical merit, feasibility of use, and required functional features of the MyJobQuest system and to provide a preliminary empirical analysis of the usability and effectiveness of the MJQ system for enabling people with intellectual disability to communicate job preferences.

Method

Participants

Participants were 20 people with intellectual disability whose involvement in the study was voluntary. Participants were adults who were at least 18 years of age, with an age range of 20 to 60 years and an average age of 41.5 years. There were 12 females and 8 males. Full scale IQ scores for this group ranged from 44 to 71, with an average IQ score of 59.6. All sessions with the MJQ system occurred at the vocational support agency where the person regularly received supports/services. Human subject’s approval was obtained from a designated institutional review board and informed consent was obtained from all participants. Participants were also provided an honorarium for their involvement as well as information resulting from the preference self-assessment process.

Procedure

In general, the intent of the MJQ app development was to meet a series of criteria that would ensure active, self-directed job interest activities. The system included (a) use of system-generated multimedia prompts to guide the user through the app; (b) uncluttered interface designs and easily accessible screen elements to optimize touch screen input; (c) software designed with a linear approach (e.g., provides a straight path of operation as opposed to branching, and looping) for a simplified interface; (d) interface flow designs that incorporate the well-established teaching methodologies of consistency and repetition to promote learning and retention; (e) use of “error minimization” techniques, such as removing buttons from the screen when their appearance is unnecessary, or software code to reduce the likelihood of unintended double-clicking (e.g., so the user did not unintentionally start and stop a video simultaneously); (f) a set of interface customization options that allow administrators, instructors, or parents to adapt the software to the unique needs and abilities of each end-user; (g) built-in mechanisms to support tracking of progress; and (h) motivational considerations for enjoyable and compelling use of the software where appropriate.

To develop the job video content for the system, project researchers met with community job placement staff at the employment support agency to recruit community-based job sites who routinely hired trainees from the agency. As a result, complete sets of five unique video clips were obtained for each of 17 distinct community employment opportunities, with an average of 10.8 min per job to obtain the necessary task videos. Following further review with the employment support agency staff, 12 of the 17 community jobs were selected for use in the MJQ app. Each of these 12 job categories was represented in the system by five video clips of actual workers performing the most frequently required tasks in each category, for a final total of 60 localized job video clips in the system. The system was coded to randomize the pairings of video clips during an assessment, with the only constraint being it would not pair two videos from the same job category. For the purpose of this project, the app was deployed on iPad 2 devices.

Process

Following entry of an ID number to protect participant privacy, researchers prompted the person with intellectual disability to “listen to the computer and it will tell you what to do.” No additional training was provided. The app displayed a large “Start” button on the screen along with an audio message stating “Please press the Start button to begin your assessment.” Upon pressing Start, the user was then presented with a simple screen (depicted at the left in Fig. 1) (shown smaller than actual size) while an audio message played stating “Please select a picture for the job video you would like to watch first.”
Fig. 1

MJQ video viewer screen before (left) and after (right) watching each job video

After the user selected the desired picture and watched the first video displayed (full screen) on the iPad, the display returned to the video viewer screen, but this time with the already-viewed video grayed out to indicate it had already been viewed. At this time, the system provided audio- and text-based versions of a prompt for the user to “Now select the other picture to watch a different job video.” After the second video clip finished playing, both pictures re-appeared, and for the first time, two buttons with a “thumbs up” icon on them appeared below each picture, as shown on the right in Fig. 1. At the same time, an audio prompt was provided for the user to “Now select the button for the job you liked best.” The user then tapped the thumbs up button directly under the picture depicting the job they liked best from the two choices available. The app then repeated this process through all 30 forced choice job video comparisons, thus employing the approach of utilizing repetition and consistency to create independently accessible software interfaces for people with intellectual and other cognitive disabilities.

After the user made his or her final choice, a closing screen was displayed, and a confirmation message played stating “Good Work, you have finished your MyJobQuest assessment. Press the Done button to close this program.” Tapping the “Done” button initiated an automated upload of assessment results to the Reporting Repository, a cloud-based software system used for storing and reporting assessment result data. This data recording and reporting system allowed employment support staff to remotely view results in real time immediately upon completion of an assessment from any computer connected to the internet without requiring any data entry. To view assessment results, employment support staff navigated to the password-protected Report Repository web site, which listed all assessments completed to date by each participant and included a link to view the person’s results. Each completed assessment in the list also included button links to two reports. The first, shown in Fig. 2, was the Career Interest Summary Report, which listed each Job Category by order of preference and included the frequency in which the trainee chose job videos from each category (out of a possible 5 opportunities.) This report provided employment support staff and job seekers with an “at-a-glance” summary version of the results for each person remotely and in real time.
Fig. 2

MyJobQuest Career Interest Summary Report

A second report, called the Job Selection Report, was also available and showed the results of each forced choice trial by displaying the initial still images for each job task comparison. A red outline was used around the image to show which job task was selected in each of the 30 forced choice trials. This report was used by vocational counselors to help understand the summary results. This very graphical report was also designed specifically for participants with intellectual disability who had little or no literacy skills but were able to view and understand their own assessment results. Both reports were immediately printable to have a hard copy of the results if desired.

Determining Benchmarks for Preference Assessment

Evaluating the accuracy of assessment results from the MJQ system required the use of a benchmark that would indicate each participant’s “known” interests. It was determined that the best source for identifying career interests of each participant was their supported employment support professional. These employment support professionals (vocational counselors) were asked to use any resources available to them, including previous career assessment results on file, previous job/volunteer history, interviews, and their existing knowledge of the person, to predict job interest categories for each study participant and to record that information on a Job Interest Prediction form developed for use in the study. The employment professional then ranked what they expected to be the top three job categories as well as the bottom three categories expected out of 12 possible choices for each participant. A total of five professional vocational counselors completed the Job Interest Prediction forms and reported informally that the process of making the job interest predictions took “about an hour” for each person.

Participants were provided one prompt to start the session by encouraging them to “Press the start button to begin.” Subsequent instances of needing assistance to operate the system were recorded as user prompts for data collection purposes. Participants were assisted as needed per study protocol, but the audio prompts provided by the system and the simplified interface were the primary guidance available to participants to navigate the prototype app. In general, all participants were able to quickly learn the interface and operate the system, with only two participants needing help beyond the initial prompt. In both instances, the additional prompting was related to learning to tap the screen of the iPad correctly. Participants generally proceeded through the 30 random forced choice trials generated by the MJQ system without further human assistance.

Data Analyses

A chi-square analysis was conducted to evaluate the accuracy of the career interest results obtained from self-directed use of the MJQ prototype compared to the predictions made by the person’s vocational counselor. There were four possible accuracy scores for each participant representing how many of their preferred job categories matched the predictions made by the person’s employment support provider prior to the session. A person’s assessment score for each of the 12 job categories ranged from 5 (all five job task videos selected for a Job Category) to 0 (0 out of 5 job task videos selected for the Category). For the analysis, a score of 4 or higher was used as the threshold at which a job category was deemed “of significant interest” to the person. Thus, to be considered an area of interest, at least 80% (4 out of 5) of the job task videos for that job category had to be selected by the person. Accuracy scores were calculated for each person based on the criteria described in Table 1.
Table 1

Derivation of accuracy scores

Score

Explanation of score

3

Excellent: 3 out of 3 of the predicted interest areas received a score of 4 or 5 from the study participant’s MyJobQuest assessment results.

2

Very Good: 2 out of 3 of the predicted interest areas received a score of 4 or 5 from the study participant’s MyJobQuest assessment results.

1

Good: 1 out of 3 of the predicted interest areas received a score of 4 or 5 from the study participant’s MyJobQuest assessment results.

0

Poor: 0 out of 3 of the predicted interest areas received a score of 4 or 5 from the study participant’s MyJobQuest assessment results.

The chi-square table was populated with observed and expected values for each possible accuracy score. Expected values were calculated as follows: probability of achieving accuracy score by chance × number of cases. Disability professionals made predictions of the top three areas of interest and the bottom three areas of interest from the 12 job categories as previously described. Thus, two separate chi-square analyses were conducted, one for accuracy of predicting job categories that participants liked most, and the second for accuracy of predicting job categories that participants liked least.

Results

Table 2 shows the chi-square table for the first calculation, followed by the results of the chi-square procedure.
Table 2

Chi-square results: job categories of most interest

Score ➔

3

2

1

0

Total

Observed

2

10

8

0

20

Expected

1

1

5

13

20

Total

3

11

13

13

40

Degrees of freedom = 3; chi-square value = 21.39; p value is .000087; p < .01

As shown in Table 2, the MJQ job interest assessment results were statistically significant (p < .01) indicating that the prototype system was very effective at enabling people with intellectual disability to express their job interests accurately using job interest predictions made by support professionals as a baseline. Chi-square results for least favorite job categories were also significant (p < .01), providing evidence that MJQ was also very effective for expressing job “dislikes” and identifying job categories of least interest to individuals according to the predictions of support professionals. Table 3 summarizes these results.
Table 3

Chi-square results: job categories of least interest

Scores ➔

3

2

1

0

Total

Observed

3

14

2

1

20

Expected

1

1

5

13

20

Total

4

15

7

14

40

Degrees of freedom = 3; chi-square value = 23.8381; p value is .000027; p < .01

Usability data were also collected on the amount of assistance needed to complete each MJQ evaluation. No statistical analysis was conducted with these data due to the lack of availability of a comparison set of data beyond the time informally reported by the employment support specialists to review information and make their predictions (i.e., “about an hour” for each participant, as previously mentioned). Only two participants required prompts or assistance to complete the assessment, but, as noted previously, only for help with learning to use the iPad touch screen correctly. These two people accounted for all five corrective prompts required across the group of 20 participants.

Discussion

These results provide preliminary evidence that a cognitively accessible self-directed career interest assessment can effectively enable people with intellectual disability to accurately express their vocational interests. Results and reports of each assessment were immediately available to staff via the cloud-based Results Repository without requiring data entry or word processing, providing the benefit of savings of professional staff time and costs. Additionally, the relatively low cost of obtaining and implementing job video clips depicting real, locally available employment options provides a concrete, “what-you-see-is-what-you-get” opportunity for people with intellectual disability to be matched with locally available jobs that fit their interests. Thus, in summary, the results demonstrated that the MJQ system was an effective tool (p < .01) for self-expression of job interests by individuals with ID. Further, the independently usable, self-directed nature of the MJQ interface, combined with automated, cloud-based result tabulation and reporting, has the potential to reduce staff time burden resulting from (1) automated data collect through the cloud by the system and (2) by reducing the number of job site visits needed to identify vocational interests.

Limitations

There are several limitations to this study that need to be kept in mind when interpreting these results. First, obviously, this was a small convenience sample and there was no control group with which to compare results, and thus it is not feasible to generalize from these findings to a larger population. Second, determining the baseline for preferences from a proxy report, even when that reporter is a vocational professional who works with the person, introduces potential bias. People with intellectual disability may believe they like something because that is what they have been told by their employment support personnel. That said, the multiple iterations of forced choice videos make it very difficult to manipulate the outcomes. There remains a need for additional ways to evaluate the impact of such self-directed interest assessments, including measuring self-determination. The fact that participants required almost no external human assistance to successfully complete the assessment suggests that such efforts may contribute to greater sense of self-determination and self-direction. Further, Lee et al. (2011) found that the participation in self-directed transition planning activities supported by technology (e.g., cognitively accessible e-reader) resulted in greater enhancement in self-determination than did such participation without technology supports. But, any impact of the MyJobQuest app on self-determination is speculative at this point.

Summary

In summary, this study provided evidence that people with intellectual disability can reliably self-direct vocational interest assessment activities when provided cognitively accessible technology support to do so, and that they identify areas of vocational interest that are consistent with those identified by vocational professionals familiar with the person. Given the exceedingly poor employment outcomes for people with intellectual disability in general, such efforts seem a reasonable step in providing opportunities for people to engage in meaningful employment. Further, the easy customization of an app like MJQ allows for information on locally available and relevant jobs to be presented to people to reduce the need for higher-level generalization. The fact that people were able to use the app with virtually no training should be both a benefit to employment support professionals and a means to enhance self-direction and self-determination.

Notes

Author Contributions

DKD and SES: designed and executed the study, assisted with the data analyses, and assisted with writing the paper. CDD: assisted with the data collection and software development. MLW: consulted on research design and assisted with writing the paper.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Bureau of Labor Statistics (2015). Disability employment statistics. Accessed 18 Feb 2018 from http://www.dol.gov/odep/topics/DisabilityEmploymentStatistics.htm.
  2. Cornell University (2012). 2012 disability status report: United States. Accessed 17 Feb 2018 from http://disabilitystatistics.org/reports/2012/English/HTML/report2012.cfm.
  3. Lee, Y., Wehmeyer, M., Palmer, S., Williams-Diehm, K., Davies, D., & Stock, S. (2011). The effect of student-directed transition planning using a computer-based reading support program on the self-determination of students with disabilities. Journal of Special Education, 45, 104–117.CrossRefGoogle Scholar
  4. Parsons, M. B., Reid, D. H., & Green, C. W. (2001). Situational assessment of task preferences among adults with multiple severe disabilities in supported work. Journal of the Association for Personals with Severe Handicaps, 26, 50–55.CrossRefGoogle Scholar
  5. Shogren, K. A., Wehmeyer, M. L., Palmer, S. B., Rifenbark, G., & Little, T. (2015). Relationships between self-determination and postschool outcomes for youth with disabilities. Journal of Special Education, 48(4), 256–267.CrossRefGoogle Scholar
  6. Stock, S. E., Davies, D. K., Secor, R. R., & Wehmeyer, M. L. (2003). Self-directed career preference selection for individuals with intellectual disabilities: using computer technology to enhance self-determination. Journal of Vocational Rehabilitation, 19, 95–103.Google Scholar
  7. The Arc of the United States (2011). Still in the shadows with their future uncertain: a report on Family and Individual Needs for Disability Supports (FINDS), 2011. Washington DC: Author.Google Scholar
  8. Wehman, P. (1992). Life beyond the classroom: transition strategies for young people with disabilities. P.H. Brookes Pub. Co.Google Scholar
  9. Wehmeyer, M. L., & Metzler, C. A. (1995). How self-determined are people with intellectual disabilities? The National Consumer Survey. Mental Retardation, 33, 111–119.PubMedGoogle Scholar
  10. Wehmeyer, M. L., Palmer, S., Shogren, K., Williams-Diehm, K., & Soukup, J. (2012). Establishing a causal relationship between interventions to promote self-determination and enhanced student self-determination. Journal of Special Education, 46, 195–210.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Daniel K. Davies
    • 1
  • Steven E. Stock
    • 1
  • Cameron D. Davies
    • 1
  • Michael L. Wehmeyer
    • 2
  1. 1.AbleLink Smart Living TechnologiesColorado SpringsUSA
  2. 2.Beach Center on DisabilityUniversity of KansasLawrenceUSA

Personalised recommendations