Keywords

1 Introduction

The Georgia STEM Accessibility Alliance’s (GSAA) BreakThru electronic mentoring program responds to a National Science Foundation request for research on virtual worlds to support outcomes for students with disabilities (SwD). Reports issued by the National Science Foundation (NSF 1996, 2000, and 2004) emphasize the need for increased persistence among students with disabilities in STEM professions. Given the significant barriers to accessing higher education STEM programs (Burgstahler, 1994; NSF, 2000) experienced by SwDs, the need for solutions is clear. Electronic mentorship provides students with necessary support while overcoming many of the restrictions of the traditional, face-to-face mentoring model.

Inexperience with the software and interface metaphors of virtual worlds each present a number of challenges to new users. In some instances, the unfamiliarity of the platform makes it less appealing than more traditional communication methods. GSAA has attempted to meet the challenges of virtual world e-mentoring through a strategy focused on both accessibility and the use of gamification concepts to encourage use of the tools. This paper will explore the ways in which BreakThru has leveraged the affordances of virtual worlds to further enhance the e-mentoring model and discuss some of the findings regarding platform adoption that remain.

2 Virtual World Selection Process

Upon its inception in 2010, GSAA was confronted with an array of choices for virtual world platforms around which to develop BreakThru. GSAA had four major platform criteria: (1) It needed to enable very rapid prototyping and delivery of a functional product; (2) Computer hardware requirements needed to be minimal; (3) Costs of development and maintenance needed to be affordable; and, (4) It had to be as accessible as possible. Three virtual world platforms were selected as the most promising and compared according to these factors to select the best fit for BreakThru.

Second Life. Second Life is an online virtual world that is privately developed, owned and maintained by Linden Lab (http://www.lindenlab.com) which went live in 2003. While in many ways the most restricted virtual world platform in competition for hosting BreakThru, Second Life had a number of advantages which other platforms lacked. As of 2010, it was by far the most widely recognized and utilized virtual world platform (Meisenberger et al., 2008) among the options available. Second Life is a product, and the backing of an experienced developer with a financial investment in the software also meant that updates and new features could be relied upon with no additional resources needed from GSAA. In addition, Second Life also has a thriving economy supporting user created content both by individuals and specialized design groups offering development services. GSAA was able to leverage this third-party economy to provide its participants with a wide variety of virtual world resources and rewards for project activities.

GSAA conducted an assessment of three major platforms:, Forterra OLIVE, Second Life (http://www.secondlife.com), and OpenSim (http://opensimulator.org). GSAA selected Second Life as its primary virtual world platform based upon its adherence to the four criteria described previously (White and Todd, 2011; Todd, Todd, 2012). Second Life was the most mature platform with an extensive history of updates and active development by Linden Labs. The community of Second Life users and content authors enabled GSAA to prototype a functional virtual world for students enrolled in BreakThru mentoring within the first year of the project. Computer hardware requirements were minimal both as a result of the maturity of the Second Life software and extensive options to scale the visual fidelity of the virtual world according to end user hardware. Linden Labs also hosted the virtual world on their own servers, allowing GSAA to subscribe to Second Life as a service and minimize administrative overhead. Altogether, these advantages made Second Life both the fastest and most affordable platform option.

Accessibility of Second Life. Accessibility remained a major concern, and numerous accessibility barriers were reported with Second Life (Hickey-Moody and Wood, 2008). Second Life was mostly inaccessible to visually impaired users and the default client-side software, called a viewer, was incompatible with screen readers (Peters and Bell, Peters and Bell 2007; Verhoeven, 2007). The viewer worked better for hearing impaired users since a text chat option was available, but the advent of voice integration actually made the playing field more uneven (Peters and Bell, Peters and Bell 2007). However, the text chat would not necessarily align with spoken voice unless a transcriber was present within the virtual world. Users with mobility or dexterity limitations had the most success using the viewer, though more complex controls could potentially be difficult when using alternative input devices.

The accessibility issues highlighted as presented within Second Life also existed to a similar degree in other potential platforms. However, the much larger community of frequent Second Life users meant that there was also a larger focus on making Second Life accessible to those users. A number of open-source projects were available to GSAA that held potential to alleviate these barriers. Linden Labs allows for alternative viewers to connect to the virtual worlds hosted on their servers. One such alternative viewer Radegast (http://radegast.org/wp/), a Second Life viewer built for improved accessibility. Radegast eschews the traditionally 3-D graphics heavy approach to virtual worlds while maintain full functionality and interactivity. It also added features such as improved audio cues, automated text responses, and text-to-speech output. The existence of Radegast and other free accessibility tools such as Virtual Guidedog (http://www.virtualguidedog.com) made a strong argument that if no virtual world platform could be relied upon for native accessibility, then the presence of a strong user community held potential to alleviate at least some of the existing barriers (Kelly, 2008).

3 BreakThru Virtual World Development Process

GSAA partnered with the Vesuvius Group, LLC to rapidly prototype a Second Life virtual world suitable for use in electronic mentoring. Linden Labs sells parcels of 3-D virtual space on their server, called the Grid, as individual islands. Each of these parcels carries an associated annual maintenance cost and provides the owner and designated content managers with free reign over the aesthetics and functionality present in that space. In conjunction with Vesuvius, GSAA purchased two adjacent spaces on the Grid: one for secondary students and one for post-secondary students.

Training. Roughly 20 % of the space within the VLR is dedicated to a training obstacle course with detailed guideposts that explain how to use the Second Life client. Participants also undertake a 1-hour, mandatory training administered by GSAA staff that teaches the basic skills necessary to navigate, communicate, and interact with objects and other avatars in Second Life. The need for this training became apparent when early participants began encountering difficulties with the Second Life client. Initial plans for the obstacle course were for participants to undertake a self-guided training. However, many participants elected not to complete the course or simply failed to realize that it existed.

4 Methods of Communication Among BreakThru Dyads

Mentor and mentee participants connect with one another using a wide range of technologies. Participants are introduced to the range of communication methods supported by GSAA and then allowed to shape their mentoring relationship using whichever tools are most appropriate to their individual needs. GSAA monitors each dyad and collects data on the chosen method of communication.

Beginning in 2012, students were surveyed twice during each academic year of participation in BreakThru, once in the fall and once in the spring. Data collection took place five times: Fall 2012, Spring 2013, Fall 2013, Spring 2014, and Fall 2014. Students were asked to “Select all the ways you communicate with your mentor/mentee.” The aggregate results of this question across all five data collection periods is shown below Table 1.

Table 1. Communications Methods Utilized Across 5 Reporting Periods
Full size table

Participants generally reported a higher frequency of usage with communication platforms with which they were already acquainted. Email and phone, unsurprisingly, were reported as the most commonly used communication methods. Interestingly, SMS was reported as a more frequent method of communication than traditional telephone among secondary mentees. Regarding Second Life specifically, adoption was slightly higher among post-secondary mentors and mentees.

Anonymous usage statistics are collected within the VLR that record total logins and peak user concurrency each day. Aggregating this data month by month provides a clear picture of how usage ebbs and flows throughout the academic calendar. Discounting the summer months, students tend to be most active during September and October, and least active during March and November. March correlated with spring break for both secondary and post-secondary BreakThru participants, and that gap in instructional time may account for a portion of the consistently poor participation evident in March (Fig. 1).

Fig. 1.
figure 1

Fall 2012– Fall 2014 Total VLR Users per Month

Full size image
Fig. 2.
figure 2

Sep– Nov 2012 Peak Concurrency by Day

Full size image
Fig. 3.
figure 3

2013 Total Logins and Distinct Badge Users

Full size image
Fig. 4.
figure 4

2013 Monthly Peak Concurrency by Day

Full size image
Fig. 5.
figure 5

Total VLR Logins 2012–2014

Full size image

Annual usage patterns begin to emerge when multiple years of project data are compared. Usage is generally highest at or near the beginning of each semester. This also coincides with the time period during which new participants are enrolled in the program. Fall semester usage drops sharply in November and December each year, providing a narrow window of time during which users are logging in to the VLR. Spring Semester usage remains slightly more stable and time period during which students are logging in ranges from January until May with a peak in April. Both Fall and Spring semesters show peaks that are concurrent with the scheduling of guest speaker presentations or project-sponsored events taking place within the VLR. Project activities do not take place officially during the summer academic break, but a few users continue to log in periodically.

Overall use of the VLR peaked during September and October 2012 and tapered off dramatically as the semester continued. These months coincided with a series of mandatory training events which account for a significant amount of total monthly activity. August and December were each months during which participants were inactive owing to the start and end of fall semester.

September 2012 was the first month in which a significant number of participants had access to the island. However, peak concurrency never rose above 2 or 3 participants at a time (i.e., 1 dyad) except during the Fall Kick-off event hosted by GSAA in the VLR. The Kick-off was the first significant experience that many participants had with Second Life, and based on login records many students did log in subsequently until required to do so. Smaller peaks in October 2012 are associated with mandatory training events designed to acclimate participants to the interface of Second Life.

5 Interactivity and Gamification

Early data from 2012 strongly suggested that additional incentives would be necessary in order to entice participants into regular use of the VLR. Participation in e-mentoring activities remained high, but participants were relying on other communications platforms to connect with one another. During winter 2012, efforts were made to increase the range of activities available to BreakThru participants within the VLR. Early research suggested that gamification was an effective means of increasing engagement among partciipants (Chrons and Sundell, 2011). The inclusion of game-like activities also showed promise for faciliating experiential learning (Duncan, Miller & Jiang, Duncan et al. 2012; Inman et al., Inman et al. 2010). Gamification applies game design concepts to other contexts such as education, and GSAA began a review of potential areas where gamification could be utilized to increase the degree of interest in Second Life as a mentoring platform.

Gamification shows potential for enhancing engagement and motivation among students, and for improving the interactivity of online media (Weber, 2004; Pursel & Bailey, 2005; Hamari et al., 2014). Universities are experimenting with the application of badging as a method of recognizing student achievment (Johnson et al., 2013). Badges, or visible rewards for completion of specific activities and goals, have been shown to positively influence behavior in certain situations (Grant and Betts, 2013). Due to the individualized nature of e-mentoring dyads in the program, BreakThru cannot uniformly apply badges for completion of specific activities related to mentoring. Instead, activities were developed with the intention of increasing the frequency and duration of logins to the VLR. It was expected that participants would have differing levels of interest in certain activity types, and the intent was that badging activities would serve as an enticement to log in to the VLR more frequently and for greater duration.

Badge Activities. GSAA leadership in collaboration with Vesuvius Group, LLC developed a series of activities intended to provide greater immersion to students using the VLR during mentoring. These activities are not tied directly to mentoring. Instead, they provide a sense of accomplishment and encourage collaboration between participants. A mixture of individual and group activities were developed, and certain activities can only be accomplished cooperatively Table 2.

Table 2. Badge Rewards Available to BreakThru Participants in the VLR
Full size table

Activities were selected to encourage exploration of the entire island and provide a connection to STEM education. BreakThru participants who spent longer in the VLR and took time to explore areas of the island that were not introduced during mandatory training were the most likely to encounter badge completion activities. For example, the Botany badge requires participants to find and interact with 10 different species of plant life represented within the VLR. Finding and clicking on each provides the participant with accurate details about that plant’s biology. Other badges provide hidden training to participants about how to engage with one another in the VLR, such as the Traveler badge that requires participants to play with the various vehicles and mobility devices on the island (e.g., boats, bicycles, wheelchairs, etc.). Certain badge activities can only be completed cooperatively with another BreakThru participant (Image 1).

Image 1.
figure 6

Image of badge tracking UI element in the BreakThru VLR

Full size image

Badge completion is tracked through a custom interface element available in the VLR. This interface provides participants with instructions on how to complete each activity along with a progress indicator showing how close they are to finishing. Activities also have an associated graphical badge which is presented to students who complete them. Badge completion data is aggregated on a leaderboard so that participants can compare their progress to one another. Once a participant completes the activites associated with a given badge, they are rewarded with a graphical signifier on their UI as well as a visual token (hats, clothing, pets or other digital goods) that represents to other users that the badge activity has been completed. Additional rewards are available for participants who complete multiple badges Table 3.

Table 3. Monthly Badge Statistics by Distinct User and Total Interactions
Full size table

The badge implemtation in BreakThru shows significant spikes in usage that correlate with other events taking place within the VLR. While a high volume of usage was found among a small number of users during the initial launch event announcing the badge feature, the highest overall usage was recorded early in the Fall 2013 semester. Mentoring activities typically begin regularly taking place in September once new participants have finished enrollment both in classes and in the BreakThru program. The Fall 2013 Semester Kick-Off event both introduced new participants to the VLR and welcomed back returning participants from previous semesters. The Graduate Symposium in October 2013 enticed a significant number of participants to log in and attend a presentation by a special guest speaker. In each case, the badge system was not specifically addressed or referenced as part of the event, but the increased number of logged in users led to a much higher degree of interaction with the system.

Figure 3 shows the effect of badging on overall use of the VLR. Periods of elevated activity in the VLR correspond to an increased number of interactions with badge objects. Students returning from the holidays jumped back into the program quickly in January. However, overall logins for Jan-May 2013 drops 12.7 % as compared to Fall 2012 and owing to a significant drop in logins for March 2013. Aug-Dec 2013 rates dropped a further 3.5 % from spring, or 15.8 % compared to Aug-Dec 2012.

GSAA also tracks total Badge Interactions, or times a VLR participant clicked on a badge-related object, in an attempt to correlate overall badge activity more closely with Total Logins. Though not every student participated in badge activities, Fig. 3 depicts what might be expected for activities that a user completes over multiple logins. Distinct Badge Users records individual accounts that undertook an activity within the VLR related to at least one badging activity during each monthly reporting period. Since each user is counted once at most, the total possible number of Distinct Badge Users is much lower than the aggregate of anonymous Total Logins during the same time period. Anonymous login totals cannot be compared to distinct badge users. Instead, peak concurrency must be compared in a more general way with overall interactions with badge activities.

Figure 4 shows that during both spring and fall data collection periods, the levels of highest activity were recorded on days where project training or events were already scheduled to take place in the VLR. Fall 2012 data (see Fig. 2) prior to the introduction of gamification elements showed a similar pattern, suggesting that participants were not enticed to log in with any greater degree of frequency as a result of badging activities being made available.

Overall, the data describing both interactions with badging activities and peak user concurrency suggest that gamification as implemented within the BreakThru VLR did not increase the frequency of visits by participants. However, the badges themselves were sought after by a minority of participants who already had a reason to log in. Guest speakers and special events were of much greater significance in predicting a higher peak concurrency among VLR users. Once logged in, badge activities saw a significant uptick in overall usage. This may suggest that badges were seen more as fun distractions than opportunities for STEM engagement in spite of the STEM focus of each activity (Fig 5).

Peak usage during Fall 2012 predates the inclusion of gamification concepts. Yearly trends show a gradual decline in usage from 2012 to 2014 despite an expanded range of activities available within the VLR. A number of additional factors may conribute to the reduced participation in the VLR over time, including the specific mixture of student and mentor participants, level of participation, events scheduled by project staff.

6 Conclusion

Virtual worlds such as the BreakThru VLR provide an interactive, graphically rich experience to participants seeking that level of immersion. While the available technology does present several barriers to accessibility, appropriate training and use of open-source accommodations can alleviate those barriers and allow for a unique e-mentoring experience not available through more traditional methods. However, virtual worlds must be supported through communication via other methods.

Students will generally default to using methods of communication that are faster and more familiar. Participants overwhelmingly reported using the telephone and email. Secondary participants also reported significant use of short-messaging service (a.k.a. texting). These three options are each available from mobile devices with full functionality, whereas Second Life mobile clients sacrifice graphical fidelity and several features in order to adapt to mobile platforms. This strips away several of the advantages virtual worlds might have over other mobile communications methods.

The results of BreakThru suggest that virtual worlds are most useful to students as a supplementary platform with a high level of specialized content available. Students are most likely to utilize the VLR when provided with incentives such as guest speakers or special event. Within the mentoring model explored via BreakThru, gamification concepts are not a successful driver of virtual world platform adoption. Badging is interesting to some users, but does not function as a means of increasing usage of VLR features on its own.