Introduction

Over the last decade, radiology education has undergone a dramatic transformation away from physical radiology classes and toward radiology education online1. Technological developments have led to the expansion of web-based education and conferencing [1]. The coronavirus disease of 2019 (COVID-19) pandemic has challenged the educational missions of academic radiology departments nationwide3. Physical distancing, the most effective strategy for preventing the spread of COVID-19, prevents learners from gathering in classrooms and laboratories4. This has caused radiology residency programs to rapidly transition to online learning and Video-conference platforms (VCPs) for teaching.

In the pre-COVID era, radiology residents primarily shadowed attending radiologists and radiology fellows and attended resident teaching sessions. These formats of radiology education posed significant challenges during the COVID-19 pandemic. E-learning describes the use of electronic resources (e.g., the Internet) for teaching and learning [2]. The precise definition of “blended learning” is debated, but in general, the term refers to learning through a combination of e-learning and traditional face-to-face teaching [3]. The term blended learning is also used to describe a learning environment with multiple delivery media designed to complement each other, which may be a more useful definition given that face-to-face teaching is not possible during the pandemic [4]. For the current educational challenges imposed by COVID-19, blended learning needs to occur almost exclusively in the digital space using a combination of pedagogical approaches to be effective [5,6,7,8,9]. Radiology residency programs show knowledge about the various web-based conferencing, radiology education seminars, and radiology education online content available today. We discuss particularly Radiology-Integrated Training Initiative (R-ITI) e-learning platform, UTAUT, Moodle, active image-based learning, Video-conference platforms (VCPs), OER, open education resources, course assets, accessibility, 5G Internet, game-based learning, radiology competition, virtual worlds (Table 1).

Table 1 Guiding principles for medical education during the COVID-19 pandemic*
Full size table

Online Lectures and Webinars (Live and Pre-recorded)

Online lectures are the most common mode of delivering radiology education online. They can either be live (synchronous) with the speaker delivering the lecture in real-time with simultaneous audience interaction or pre-recorded (asynchronous). The speaker can ask questions to the audience and vice-versa. This can be delivered via various Video-conference platforms (VCPs) like Zoom and Skype which will be discussed later. This is considered superior to pre-recorded lectures where there is no audience interaction. In lectures delivered in a virtual learning environment, student engagement and attention can be increased with student response systems (SRSs) [10]. An SRS is any method that makes the learning more student-centered by letting students share their ideas with their peers and instructors [10]. The most simple SRS system involves students raising their hands to contribute and can be done on most video conferencing platforms. The chat function available in most video conferencing platforms is another example of a simple SR. On the other hand, the advantage of pre-recorded lectures is that they can be visualized anytime by the audience at their convenience, and they can be reviewed. Recorded lectures are available on websites like udemy.com and youtube.com.

Video Conferencing Platforms

A clear advantage of this video conferencing technology is that it allows collaborative learning by allowing multiple trainees to view a singular study at the same time as the radiologist supervising them, with each having the ability to control the screen and scroll through images. Each participant can also annotate or point to findings while asking questions over audio or in text chat format. Although currently applicable to the COVID-19 pandemic in which social distancing is paramount, the ability to review cases remotely has multiple other benefits. Significant time savings can be attained by virtual read-out rather than by traveling on-site, especially at institutions having multiple campuses or having remote satellite locations. The ability to use multiple devices, like iPads and cell phones, allows for portability in the presence of the Internet. Another benefit is that trainees on-call can seek help from other residents, fellows, or attending radiologists who may not have immediate access to remote PACS. Having another person view a complex study may provide self-assurance for a trainee, and with the remote viewing tools, it is not a difficult task. As useful as video conferencing can be, there are some disadvantages. Whenever a new technology is introduced, there is a learning curve for the user of the software, which causes delays in its acceptance into the normal workflow. Other limitations include unreliable Internet leading to dropped connections, choppy video, or camera malfunctions, which may also derail the video conference. Another downside to the potential virtual workflow is that only a single computer screen can be shared per person at a given time, whereas many radiologists typically use 3 to 4 screens at once to display necessary information and images. Efforts must be made by the host computer or attending to keep the relevant images and series on the shared screen. Image quality is another factor that needs to be considered, as the remote participants are viewing a compressed stream of the host’s presentation, instead of a high-resolution PACS image. Additionally, if the host computer is sharing the screen of a diagnostic monitor, the participants’ monitors may not be of equal resolution, requiring the inconvenience of viewing smaller images or having to pan around the full-resolution screen image. The combination of these factors also makes virtual read-outs slower compared to an in-person read-out, and thus video conferencing should be used selectively. A security breach is also an important pitfall that should be considered. As the link is generated and hosted through the Internet, there have been reported cases of hackers attending private conferences (21), which results in a breach of protected health information. The use of a “waiting room” to screen participants and password protection can maintain confidentiality and aid in the prevention of critical data leaks [11]. Other software can be combined with VCPs like Pacsbin which can provide scrollable workflow of DICOM images and Google Classroom which provides a centralized online classroom for assignments, reports, and discussion with attendings [12]. Some of the most popular VCPs are Zoom, Skype, Google Meet, and Webex.

Radiology-Integrated Training Initiative (R-ITI)) E-Learning Modules

R-ITI is a comprehensive learning platform for examinations conducted by the Royal College of Radiologists. R-ITI’s e-learning resource is based on the knowledge and skills of practicing clinical radiologists. R-ITI is designed to support and enhance the learning of specialist registrars on the five-year radiology training scheme. R-ITI can also be used to reinforce, strengthen, or maintain core knowledge by practicing radiologists. The Royal College of Radiologists curriculum consists of 15 modules by subject area with individual sessions contained within each module [12].

Virtual Radiology Rotation

MAJ Chad C. Adams et al. describe a virtual radiology rotation curriculum being used to safely maintain medical student and intern education and engagement at the Radiology Department of Walter Reed Medical Center during the COVID-19 pandemic. The curriculum is designed as a 4-week block with each week representing one of the four main radiology subspecialties, namely neuroradiology, body radiology, thoracic radiology, and musculoskeletal radiology. A subspecialist radiologist from each section is identified as a primary mentor and tasked with designing a daily schedule for medical students and interns. The email listserv is used to disseminate timetable and updates, and it contains all current rotators as well as weekly mentors and course directors. Group membership is simple to update whenever there is a student addition or departure and whenever a member prefers to receive it with a different email address. Course materials are centralized in a cloud drive folder, which is shared with the group email address. New students receive an invitation to our shared folder, which includes the curriculum PDF, a brief “READ ME” orientation and lists of enrolled students by block. It also includes a folder for each weekly module, which houses assigned material that is not freely accessible online at home (e.g., downloaded articles, PowerPoint slides). On the other hand, assigned material that is freely accessible online (e.g., modules, videos, websites) is embedded as web hyperlinks within the curriculum PDF document Administrative access to the Drive is given only to the rotation directors. Rotators are not given administrative privileges to alter the contents of theve [13]. Video conferences are done, including online PowerPoints, and online discussion. At end of the rotation, students send their completed curriculum documents to the rotation supervisor, which satisfies the passing requirement for this pass/fail elective. These PDF documents use fillable checkboxes as student attestation on daily assignment completion and participation in each mentor’s end-of-week quiz. At the end of these documents, there is a free text block for positive or negative rotation feedback (to be received by the rotation supervisor, who is not a weekly mentor) and a digital signature block. The rotation supervisor also distributes a separate end-of rotation questionnaire with Likert scale answers [13,14,15,16,17]. In this way, the entire radiology rotation has been transferred online in this military hospital and can be emulated by others.

3D Printed Models and 3D Virtual Holographic Radio-Anatomy

3D printed models are used for teaching anatomy to radiology students. This allows very complex models to be printed very easily and quickly. Patient-specific models can be generated which are accurate to the CT/MRI images and thus help in advanced learning. In interventional radiology, 3D models play a very significant role as IR residents can practice catheter placement in models of vasculature like Circle of Willis under fluoroscopy, just like in real patients. Complex congenital heart diseases can be printed out which proves the reporting of cardiac MRIs and CTs. Complex anatomical variants can be printed out for their better understanding. Interactive virtual reality (VR) simulation and three-dimensional (3D) computer modeling are teaching techniques used throughout medical disciplines. These have reduced the cost of a trained anatomist and the need for cadavers. According to Stepan et al., VR head-mounted display (HMD) developed by Oculus VR, LLC (Irvine, CA), has been used to teach neuroanatomy. This technology provides an immersive and interactive experience: the user’s head movements are tracked in 360°, providing views of the anatomy from many different angles, and the user can move his or her position through the model using a handheld device. The platform also provides the ability to record 3D VR videos in which the HMD is used to watch a prerecorded “fly-through.” A 3D VR video was created for this study to show key anatomic relationships. Both the 3D VR video and the fully interactive, handheld controller virtual reality features were used. This study demonstrated that a VR model of neuroanatomy was as effective as traditional methods of teaching in terms of student exam performance. It also demonstrated that the VR model was superior in several subjective measurements: engagement, enjoyment, usefulness, and learner motivation [18].

Simulators

An ideal simulator should have a library of cases with image viewing and reporting capabilities, similar to real DICOM and PowerScribe software. They are not very popular as most residents draft reports of actual patients in their residency which are then corrected by the attending and final reports provided.

Virtual 3D Games

Alvarez et al. describe a 3D simulation game called Second life where learners go through radiological education on a virtual island named “Medical Master Island.” They are initially taught by instructors both asynchronously and synchronously, and then they are tested on this virtual island. Participants have experienced significant enhancement in their learning, as per their feedback, due to increased participation. This process of application of game design elements to non-gaming contexts is called Gamification. Its application to a learner’s environment increases retention, participation, and motivation [19].

Softwares

Many software are available that can be used to enhance the teaching of radiology trainees. Some of the free softwares are 3D Slicer, ImageJ, Uberstudent, Chamilo, Bitplane, and MRIcro.

3D Slicer is used for better visualization of images and reconstruction of 3D image, to segment anatomical structures in an image that can be used by trainees to better understand the images that are produced by the CT, MRI, and ultrasound machines. Virtual reality can be used to interact with these 3D images to better understand the abnormalities in an image and then describe them in the report.

ImageJ is a Java-based image processing software than can display, edit, analyze, process, and print various image formats. It can measure distances and angles; do geometric transformation such as scaling, rotation and flips, and also perform contrast manipulation. It can be used to demonstrate the pathologies in an image in multiple planes, and contrast can be added to better visualize difficult to visualize abnormalities for the trainees.

Uberstudent is a free Linux operating system for learning and teaching the skills of any subject. It is used in schools, colleges, and by researchers of all sectors to learn from and teach their peers.

MRIcro is a free medical image viewer with features like ROI drawing and analysis. These additional tools can add to the understanding of various abnormalities especially their densities.

Small Group Discussions

Although conferences with many members physically present in a room are no longer possible during the pandemic, it is possible to hold small group meetings among asymptomatic residents and faculty, in a bio-bubble. The period of these meetings is to be kept short, and the topic pertinent to a meeting cannot be virtually done. Adequate seating distance has to be kept among the members, and face masks, preferably N-95, have to be worn. Thus these meetings are to be held on important topics which cannot be done properly online like QA meetings or some multi-disciplinary meetings.

Online Multidisciplinary Meetings (MDMs)

Multidisciplinary meetings (MDMs) are another reason for people to gather in large groups. Like remote teaching, these meetings have also moved online. Remote access to MDMs not only shares the same benefits listed above for remote teaching but also has an additional benefit specific to radiology trainees. During the MDM, the radiologist will demonstrate the pathology on the screen and state the relevant findings in a manner that allows the medical and/or surgical teams to discuss treatment options. However, the radiologist will not typically demonstrate the thought process that allowed them to reach the conclusion they reached. As such, radiology trainees observing the meeting are only able to see the relevant final image and not the radiological interpretive process. With remote access to MDMs, the radiology trainee observing the meeting can open the scan on their computer and access the imaging in real time. This allows them to interpret the imaging as they would normally, while also benefiting from hearing the clinical context and the management plan [14].

Blended Learning

The precise definition of “blended learning” is debated but in general, the term refers to learning through a combination of synchronous or asynchronous e-learning and traditional face-to-face teaching [20]. With improvement in online resources and the ongoing pandemic, there is a continuous increase in blended learning where face-to-face teaching is kept to the minimum and only for those aspects of learning which cannot be taught well online. Blended learning can be added to flipped learning, where students are assigned the course materials before the online lecture so that they can read them before the online lecture.

Digital Radiology Teaching Files

Case-based radiology teaching files, including radiological images and information about the clinical presentation, disease diagnosis, or imaging technique, are an important component of radiology teaching [3,4,5,6]. From personal collections of “interesting cases” to libraries of indexed repositories, medical and radiological images are currently being used for radiology education. Kahn [7] reports his experience related to a Web-based virtual library of peer-reviewed radiological images for use in education and for making clinical decisions. Images were obtained from open-access content of five leading peer-reviewed radiology journals and one e-learning website [8]. Some prominent ones are www.urmc.rochester.edu/imaging/education/educational-resources/radiology-teaching-files and https://radiology.msu.edu/education/radiology-teaching-cases-system.

Discussion Forums

These are online decentralized places where discussions on any topic are done by all or most users, like a brainstorming session. The credibility of the resources is sometimes questionable, although users generally have an idea of their topic of discussion. Some examples are sites like Reddit.com and studentdoctor.net.

Learning Management Systems

A learning management system (LMS) is software for the delivery of educational courses, training programs, or learning and development programs [9]. The learning management system concept emerged directly from e-Learning. Learning management systems make up the largest segment of the learning system market. Sparacio et al. [6] describe a formative evaluation of the use of two different educator-centric LMSs, one in neuroradiology and one in pediatric radiology. NeuroRAD which is a neuroradiological digital library and learning community was implemented with Moodle, one of the most popular open-source educator-centric LMSs. PediatricEducation.org is a pediatric digital library and learning community, which was implemented with Manila, a commercial educator-centric LMS [6].

Conventional Websites

These are the oldest types of online education available. Radiological images are put up with detailed labeling. Explanatory texts are also provided. Some have integrative online testing like quizzes. Some give data like a traditional textbook. Generally, no dialogue is possible between the author and user. Some of the commonly used websites are auntminnie.com and acr.org. Some websites are editable by anyone like radioed and Wikipedia, and some are written by a single author like a blog.

E-Books

E-books are digital versions of physical textbooks. These can be easily purchased and distributed online and can be accessed on any digital reading device. Also, they reduce the physical contact and a trip to the library, thus reducing exposure during this COVID-19 pandemic.

Digital Journals and Blogs

More and more journals are becoming partially or completely online because of the ease of distribution and the reach of the Internet. These have the convenience of easy storage and cataloging than physical print versions. Blogs, written by experienced radiologists, are generally very informative and also keep updated about upcoming seminars and conferences [15].

Massive Open Online Courses (MOOC)

These are online courses that offer unlimited access through the web, favoring unlimited participation. They can be accessed through smartphones and other digital devices. Udemy, Coursera, and EdX are some of the platforms offering MOOCs. They are encompassed with course materials. Interactive courses with social media discussion forums have shown high course completion rates [16].

Social Media

Social media platforms like Twitter, WhatsApp, and Facebook are extremely widely used and can be employed for learning and discussion. Telegram has many channels and groups like “Radiology Tips and Notes,” which are very informative and give niche knowledge. The disadvantages are privacy, security, and addiction (Table 2). Photography-based social media apps like Pinterest and Instagram also have the potential for being used for radiology case presentations and quizzes as done by CTisus.com [17,18,19,20,21,22,23].

Table 2 Summary of various radiology educational tools
Full size table

Results

Online learning has come a long way from the early days of amateurish websites. It has several components like interactive websites, e-books, e-journals, blogs, VCPs, MOOC, 3D anatomy, virtual reality, and 3D gaming. There is no doubt that the COVID-19 pandemic has significantly accelerated the pace of advancement of online education tools. This article describes all the various modalities of online education, from the time-tested to the latest.

Limitations of This Review

Some of the online educational tools that are discussed in this review are very new, and enough experience with these tools has not yet been achieved. Also feedback from students regarding acceptance and usability, after usage of these tools has not been included in this review.

Conclusion

Online radiology education is not only supplementing but also replacing traditional methods of teaching.