For many children with autism spectrum disorder (ASD), access to ASD-specialized early intervention services is predicated on a formal diagnosis of ASD. Best practices in ASD diagnosis involve integration of data from caregiver report of developmental history and current symptoms, standardized assessments of cognitive, language, and adaptive functioning, and behavioral observations of ASD symptomology (Broder Fingert et al. 2019; Huerta and Lord 2012; Zwaigenbaum et al. 2015). While standardized diagnostic instruments, such as the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al. 2012) are available to support elicitation and observation of ASD symptoms, they were designed for use in face-to-face, distraction-free settings with standardized objects/toys. These assessments take place within an interpersonal context, throughout which the examiner sets up situations for eliciting and observing reciprocal social interaction, communication, play, and restricted and repetitive behaviors.

The coronavirus (COVID-19) global pandemic and social distancing measures imposed to curtail its spread have created unprecedented challenges to conducting ASD diagnostic evaluations, and many clinics have paused or reduced in-person services indefinitely. Given that COVID-19 has disproportionately affected minority groups (Tai et al. 2020), the pandemic has exacerbated the pre-existing inequities in health-care service access that exist in our field (Bishop-Fitzpatrick and Kind 2017; Iacono et al. 2016). Moreover, the current safety regulations for using personal protective equipment (PPE), especially the use of face masks, represent not only a departure from standardized procedures, but also a barrier to establishing the interpersonal context that is necessary for evaluating certain ASD symptoms. As such, COVID-19 has served as a massive disrupter to traditional service delivery, necessitating immediate large-scale adoption of innovative approaches to care delivery, including virtual ASD diagnostic services. Given that the gold standard approach for diagnosing ASD emphasizes a best-estimate clinical judgment approach, rather than reliance on a single or established group of measures (Broder Fingert et al. 2019; Huerta and Lord 2012; Zwaigenbaum et al. 2015), it is feasible to consider tools that support observation of ASD symptomatology using virtual assessment.

Beyond offering an alternative for assessment during the COVID-19 pandemic, virtual assessment has the potential to address the limited availability of and access to professionals who provide diagnostic and clinical services for children with ASD living in rural areas (Carbone et al. 2010; Drahota et al. 2020). Compared to those in urban or suburban areas, children living in rural and underserved areas are diagnosed at a later age (Lauritsen et al. 2014) and have decreased access to health care services (Mandell et al. 2005). Further, children from households with lower socioeconomic status and those from disadvantaged neighborhoods are also less likely to receive a timely ASD diagnosis (Durkin et al. 2010; Mazumdar et al. 2013). Creative solutions for conducting remote diagnosis will not only ensure continued access to care for children and families during COVID-19, but also have the potential to fundamentally shift the service delivery landscape by addressing the significant disparities in access to early diagnostic and intervention services in the United States as well as globally.

In recent years, efforts to increase access to ASD evaluation services for families living in rural or underserved communities (i.e., those facing the greatest challenges obtaining an in-person ASD diagnostic assessment; Drahota et al. 2020) have led to a gradual increase in the development of protocols for remote diagnosis. However, prior to the COVID-19 pandemic, the development of remote diagnostic tools had been slow-going, with limited empirical validation of these measures. Indeed, a recent systematic review found that only seven studies have examined the use of video- and web-based applications for supporting ASD assessment (Dahiya et al. 2020). In addition, most studies used asynchronous, or “store-and-forward,” methods of virtual assessment, that require the caregiver to record video of the child for later review by a trained clinician. This approach has several advantages, such as enabling sampling of child behavior across multiple days, eliminating the need to coordinate schedules with a clinician, the ease of uploading videos over time rather than relying on sufficient bandwidth for live video conferencing, and the potential for automated scoring via artificial intelligence. However, the utility of asynchronous virtual assessment during COVID-19 is somewhat limited. Asynchronous virtual assessment has high technology demands (e.g., the need to record, upload and store videos securely), with some methods requiring specialized software (e.g., Cognoa tool, Abbas et al. 2020) that is not widely available. Further, asynchronous methods are not as readily implementable due to the need for protocol and platform specific training and comparatively high start-up/maintenance costs (e.g., pay per use requirement of NODA, Smith et al. 2017). Importantly, asynchronous virtual assessment also does not allow for adjustment of the number or types of presses administered based on clinical judgment of responses and behaviors of the individual child. As a result, key behaviors (or lack thereof) may not be observed.

In addition to virtual diagnostic protocols, research has also begun to examine the utility of virtual screening protocols to assess children for potential ASD (e.g., Telehealth Evaluation of Development for Infants (TEDI; Talbott et al. 2020)), and several groups have adapted observational screening measures for telehealth during COVID-19 (e.g., Systematic Observation of Red Flags (SORF; Dow et al. 2020), Autism Detection in Early Childhood (ADEC; Nah et al. 2014), and Screening Tool for Autism in Toddlers (STAT; Corona et al. 2020b)). However, these tools are designed to be Level 2 screening measures and thus their virtual adaptation does not fully address the current need for methods of diagnostic assessment that are immediately implementable in community and research settings.

Diagnostic assessment using synchronous virtual methods is one way to directly address the immediate, practical needs of those working with young children with ASD. Synchronous virtual diagnostic assessment, in which a clinician and caregiver/child dyad interact face-to-face via videoconferencing software, utilizes basic technology that has become familiar for many families and healthcare providers due to work and school accommodations secondary to COVID-19. Additionally, no assessment-specific training is required for those who are trained to conduct diagnostic evaluations in person, as synchronous virtual assessment is viewed as a direct alternative to in-person service delivery. Fortunately, many of the developers of synchronous virtual assessments during COVID-19 have distributed tools to support dissemination of the instruments and their application. While additional research establishing the validity of these tools is imperative, there is an immediate need to identify practical options for remote clinical and research assessment. Yet, it has been difficult for many users to systematically evaluate which virtual assessment tool best fits their priorities and contexts because there is a lack of overarching information to aid in understanding and comparing the similarities and differences between these instruments.

This brief report provides an overview of the synchronous virtual diagnostic tools available to support experienced clinicians in conducting in-the-moment behavioral assessments for toddlers referred for ASD evaluation when the use of well-established in-person diagnostic tools and processes is not possible or feasible. We focus on tools designed for use with toddlers (12- to 36-month age range) to support a clinical diagnosis of ASD. This age range was prioritized given the well-documented benefits of early identification for long term child outcomes. All tools included in this report are conducted via observation of the child during interactions with their caregiver and involve synchronous communication with a trained clinician. In all cases, the clinician directs and/or coaches caregivers to conduct specific activities that press for social communication behaviors that correspond to DSM-5 diagnostic criteria. Asynchronous methods of virtual evaluation are excluded from this review given the aforementioned limitations to their immediate use for the majority of families, clinicians, and researchers.

It is important to note that at the time of writing, none of the tools meeting the above criteria, and subsequently discussed below, have undergone rigorous psychometric evaluation, although all include items or activities that are similar to those in diagnostic tools with established psychometric properties. Our aim, therefore, is to provide comparative information on the dimensions of target behavioral domains, specific activities and presses used, scoring approaches, and other key logistical considerations for assisting clinicians and researchers in selecting the most appropriate tool for their goals, settings, and populations, while the evidence base for these tools is being examined.

Overview of Measures

The five remote assessment tools included in this report were identified via solicitations from professional organization listservs (e.g., American Psychological Association Division 33 Intellectual & Developmental Disabilities/Autism Spectrum Disorder), communication from professional organizations, webinar announcements, personal communications, and a review of the literature. We recognize that this review may not be exhaustive of all synchronous virtual assessment tools that have been developed during COVID-19, but every effort was made to be as comprehensive as possible. Tools were selected for inclusion if they were appropriate for use with children across the 12- to 36-months age range (as indicated by the test developers, or as indicated by the target language level specified for the assessment) and involved direct observation of child behavior and synchronous communication between a caregiver and expert clinician. Importantly, all tools are designed to be used as one part of a comprehensive diagnostic assessment for ASD—none are intended to be the sole source of information for rendering a diagnosis. The developers of these tools were extremely generous in sharing their most recent materials with us to ensure the accuracy of the information at the time of writing. However, it is important to acknowledge that these assessments are still early in the validation process, and are thus subject to change. See Tables 1 and 2 for detailed information about each tool, which are presented in alphabetical order.

Table 1 Observed behaviors, DSM-5 diagnostic criteria and corresponding activity
Full size table
Table 2 Administration and scoring logistics
Full size table

ASD-DIAL: Autism Spectrum Disorder—Diagnostic Interview and Activities—Lifespan, Version 2

Description

The ASD-DIAL (Miller 2020) was developed at the Center for Autism Research at the Children’s Hospital of Philadelphia in response to COVID-19 and social distancing measures. This measure includes activities designed by ASD experts that have clinical utility for eliciting behaviors that map onto relevant ASD constructs. The Activities for Young Children module of the ASD-DIAL was designed for young children regardless of verbal ability.

Activities

There are 10 possible presses for young children; these presses assess social responsiveness, turn-taking, requesting, imitation, and play. The clinician selects the specific presses, number of presses, and order of presses to administer based on the child’s evaluation needs. Most administrations last between 30 and 60 min. The manual includes scripted instructions for the clinician to help guide caregivers through each activity, as well as notes and suggestions for administration. The ASD-DIAL uses materials that are likely already available in many homes, such as blocks, cars, and bubbles.

Scoring

Observations from the ASD-DIAL are used to complete a DSM-5 checklist regarding the absence or presence of symptoms; no independent item-level scoring system is provided. The level of clinician diagnostic certainty is also noted. There is a strong recommendation to use the ASD-DIAL in conjunction with other sources of information (e.g., caregiver interview, review of records, Childhood Autism Rating Scale, Second Edition (CARS-2; Schopler et al. 2010)) to inform a clinical diagnosis of ASD.

A-VABO: Adapted Virtual Autism Behavior Observation

Description

The A-VABO (Kryszak and Albright 2020) was developed at Nationwide Children’s Hospital and is designed to be used as one element of a multi-phase interdisciplinary virtual assessment protocol for ASD.

Activities

The A-VABO toddler form consists of 15 presses, which include responding to social bids, play and imitation, communication acts, and directly eliciting repetitive/stereotyped behavior. Each press has a permitted duration and number of trials as well as specific instructions for the materials needed, procedures to follow, and a script to guide caregivers through the activities. Some of the activities require specific toys and objects. However, during a pre-visit phase of the evaluation, the caregiver is asked to gather the materials from around the home (reading level of written instructions: grade 8.1), and if the caregiver is missing any of the materials, there is a list of possible item alternatives.

Scoring

The A-VABO uses item-level scoring in which the child’s response to each press is scored as either 0, 1, or 2; each score is operationally defined, with a score of 2 representing the most appropriate response. It is recommended that a consensus diagnostic decision be based on the integration of results from the A-VABO with other sources of information (e.g., speech assessment, clinical and medical interviews, the Developmental Profile-3 interview (Alpern et al. 2007), CARS-2 (Schopler et al. 2010)).

BOSA: Brief Observation of Symptoms of Autism (BOSA)

Description

The BOSA (Lord et al. 2020) was developed by Dr. Cathy Lord and her team at the UCLA CART Lab, in partnership with the UCLA Semel Institute. The BOSA was developed for administration either via telehealth or in person (with the clinician masked and maintaining social distance, but caregiver and child unmasked). This tool is an adaptation of the ADOS-2 and the Brief Observation of Social Communication Change (BOSCC; Grzadzinski et al. 2016) and is copyrighted by Western Psychological Services (WPS); permission from UCLA is required before the BOSA may be used. The BOSA-Minimally Verbal (BOSA-MV), described here, can be used for ambulatory children of any age. The BOSA can be administered and scored with children and caregivers who do not speak English, provided that the clinician is proficient in the target language and there is a valid translation of the ADOS-2 protocol in that language. While independent translation of the BOSA is not permitted, the BOSA team is currently working with WPS to create a Spanish translation.

Activities

The BOSA-MV takes 12 min and involves a standard sequence of activities: free-play and clean up with an initial toy set; blowing bubbles; free-play and clean up with a second toy set; blowing bubbles. The activities are not accompanied by explicit prompts or presses; rather, the clinician encourages the caregiver to engage their child with at least two toys from each toy set. In addition, caregivers receive an instruction sheet (reading level: grade 3.3) that provides the time limits, order, and descriptions of the activities, including suggestions for how to play with the toys included in each of the two toy sets. The toys required for the BOSA-MV are gathered from an ADOS-2 kit.

Scoring

Clinicians who use the BOSA should have experience administering the ADOS-2 in either clinical or research settings, as the scoring codes from ADOS-2 modules are used. The BOSA score is derived by rating as many codes as possible for select items across different ADOS-2 module scoring protocols; as such, the ADOS-2 training, kits, and scoring protocols may incur additional costs for clinicians. ADOS-2 item scores are converted to binary BOSA scores (present or absent) on the BOSA-MV Toddler Module DSM-5 Checklist. Clinicians can make scoring decisions based on other incidental observations and caregiver report. It is recommended that the BOSA be used in conjunction with additional measures such as the Autism Diagnostic Interview-Revised (Lord et al. 1994), CARS-2 (Schopler et al. 2010), adaptive functioning measures, and cognitive testing to inform diagnostic impressions. When the BOSA is used for research purposes, the authors encourage groups to establish reliability among the research team.

OOPS: HE: Observation of Play Screener: Home Edition

Description

The OOPS:HE (Nickel 2020) was developed by Dr. Robert Nickel at Oregon Health & Science University. The OOPS: HE was adapted from the original Observation of Play Screener (OOPS, Nickel 1997, revised 2001) which was created as a training tool for pediatricians on the characteristic behaviors of young children with ASD.

Activities

The OOPS: HE takes approximately 30 min and consists of 12 presses across 10 activities designed to observe the child’s social responses, turn-taking, requesting, imitation, and play. The clinician uses scripted instructions to guide the caregiver through each press. Some presses are flexible in duration, while others have a specific time limit and/or number of prompts. Caregivers receive an instruction sheet (reading level: grade 6.3) that outlines the recommended toys, suggested preparations (e.g., tips for camera placement), and overview of the OOPS: HE activities. Before the visit, the caregiver is asked to gather the recommended toys from around the home.

Scoring

The OOPS: HE uses item-level scoring in which operationally defined target behaviors are scored as present or absent. A total score is generated by summing the number of items mapping onto social communication and social emotional reciprocity that were absent and the number of items mapping onto restricted and repetitive behaviors that were present. The clinician also records general qualitative observations of the child’s behavior. The OOPS: HE is meant to be used in conjunction with a caregiver interview and uploaded videos of household routines (e.g., snack, getting dressed) to supplement observation during virtual evaluations.

TELE-ASD-PEDS

Description

The TELE-ASD-PEDS (Corona et al. 2020a) was developed by the TRIAD Team at Vanderbilt University Medical Center. This tool was developed prior to the COVID-19 pandemic to support caregiver-mediated remote assessment of ASD in children up to 36 months who do not yet have phrase speech. Items were selected by applying machine learning technology on a database of several hundred individuals with and without ASD. The items with the most predictive utility for young children were selected and transformed into interactive, caregiver-led tasks that comprise the TELE-ASD-PEDS. While there are no published data on the psychometric properties of this assessment, a clinical trial is currently underway to investigate its accuracy and limits. As part of this larger clinical trial, initial studies have found the TELE-ASD-PEDS to be acceptable, feasible, and comfortable for caregivers and providers, and preliminary estimates of diagnostic accuracy are promising (Corona et al. 2020b; Wagner et al. 2020).

Activities

The TELE-ASD-PEDS takes about 20 min to administer and consists of 10 presses across eight activities, which include those related to social responses, turn-taking, requesting, and play. The clinician uses a script to scaffold a caregiver through the tasks with their child. Caregivers receive an instruction sheet (reading level: grade 8.5) providing an overview of the assessment and instructions for gathering materials for the assessment from their home.

Scoring

The TELE-ASD-PEDS rating form is used to code the observation of specific behaviors; symptoms are scored either dichotomously (e.g., present or absent), or using a three-point Likert scale (e.g., symptom obviously consistent with ASD, present at subclinical levels, or not present). A total score is computed by summing across all items, and raters are asked to indicate their level of diagnostic certainty. The TELE-ASD-PEDS is intended to be used as one component of a comprehensive ASD virtual assessment.

Discussion

The COVID-19 pandemic, and associated social distancing mandates, have placed significant limitations on our ability to conduct standardized in-person ASD assessments, and have generated the need to re-evaluate our current strategies for diagnosing ASD. To date we have identified five measures designed for synchronous virtual interactions between a clinician and a caregiver of a toddler at risk for ASD, all of which involve the clinician guiding the caregiver, via telehealth, to engage in a series of activities in the home setting. It is critical to acknowledge that this is a rapidly evolving field, and it is very likely that the measures described will continue to be modified as empirical data become available. While no published psychometric data were available for any of the measures at the time of writing, we hope that the summaries offered in this report will support clinical decision-making related to the selection and use of a virtual observational diagnostic assessment for individual service settings.

The five measures share important commonalities in terms of the behavioral domains assessed, the activities and materials used, and the supports offered to guide clinicians and families through the remote administration process. Yet, there are also differences across measures that may influence adoption decisions. For instance, administration times can range from 12 to 60 min, the number of toys required can range from 5 to over 15, and the extent to which families are likely to have and/or can use everyday items in their homes varies. In addition, there are significant differences in cost (e.g., the BOSA-MV requires toys and protocols from an ADOS-2 kit). The assessment approaches also differ in terms of the extent to which target behaviors are elicited directly or observed passively. For example, the A-VABO includes a prescribed press for repetitive/stereotyped behaviors, the OOPS: HE encourages the caregiver to establish a context for eliciting repetitive/stereotyped behaviors, and the other three measures simply encourage the clinician to observe and note the occurrence of such behaviors as they arise naturally across activities. Additional differences in materials, presses, administration, and scoring conventions and interpretation should be considered when deciding when, how, and for what purpose to deploy any of these measures. Overall, our hope is that this review provides relevant information to enable clinicians and researchers to make informed decisions about the optimal assessment for their unique situations as long as COVID-19 precautions and/or other factors prevent the use of traditional face-to-face ASD assessments.

While virtual assessment of ASD will ideally serve to increase access to diagnostic services for traditionally underserved populations, there remains a subset of individuals for whom telehealth will not be feasible. Telehealth requires a reliable internet connection that can support video conferencing, and also requires that families have appropriate equipment for engaging in the virtual visit. Therefore, families living in geographic areas without broadband connection, or who lack the requisite hardware and software, will be unable to participate in a virtual assessment unless these technologies are provided. Further, four of the five assessments we reviewed require that the family use their own toys for the assessment, which cannot be assumed to be universally available in all homes, and thus may provide another obstacle. Consequently, families who live in rural areas or are economically disadvantaged (i.e., families who already have disproportionately limited access to health services), are poised to have their access to ASD evaluations even more restricted if telehealth is the sole option. Another consideration is that there is the potential for under-resourced families to receive less valid assessments if eventual psychometric data is based primarily on families who have the requisite toys and technologies. Thus, while remote assessment will broaden access to services for many, there is a subset of families who will need the health service sector to provide the needed materials (e.g., Wi-Fi hotspot, equipment, toys) and/or conduct in-person diagnostic evaluations. Every effort should be made to meet this need as safely as possible throughout the COVID-19 pandemic, and we hope that as virtual assessments become more advanced and refined, these factors will be considered.

Ultimately, the pandemic has propelled the ASD clinical and research communities from slow, incremental change with respect to virtual assessments to a resounding and urgent need for radical innovation. While validation studies on the available measures need to move forward, it is critical to acknowledge that we are merely in the early stages of the innovation “lifecycle” (Brown and Katz 2009; Tschimmel 2012) for the design, development, deployment, and sustained adoption of virtual assessments for ASD. Progression through the lifecycle should be guided and informed by tracking clinicians’ and families’ experiences in navigating the logistics of the virtual assessment and diagnostic decision-making process, and characterizing the profiles of families for whom the virtual assessments prove to be a "good fit" (e.g., acceptable, feasible, desirable). As a community, we must scope the needs, challenges, and pain points for families and clinicians and then use these data to modify and/or create tools and approaches that ensure more individualized, equitable, and sustainable individualized service delivery. This “re-set” in assessment development provides the unique opportunity to embed culturally sensitive items and algorithmic parity for diverse and underserved groups into the early validation process, rather than after measures have been widely adopted. In addition, the field should be poised to capitalize on rapid improvements in software (e.g., Zoom) and hardware (e.g., cameras) to address needs such as encryption, ease of use, and performance optimization for weak Wi-Fi.

Even when the widespread, frequent use of virtual assessments for ASD subsides as the effects of the pandemic are mitigated, the current moment can be considered a critical period of rapid innovation for virtual assessment tools. This is the case not only for methods of synchronous virtual diagnostic assessment such as those reviewed herein, but also for asynchronous virtual assessment (e.g., Cognoa tool (Abbas et al. 2020), NODA (Smith et al. 2017)) and virtual ASD screeners (e.g., SORF (Dow et al. 2020), TEDI (Talbott et al. 2020)). With collective action at the clinical, research, policy, and stakeholder levels, virtual assessments can outlast the COVID-19 pandemic and be integrated into diagnostic processes to support triage, enhance efficiency, ensure ecological validity, and address current health disparities by improving access to critical ASD diagnostic assessments for families in remote and underserved areas.