Introduction

Currently available endoscopic techniques allow a wide spectrum of diagnostic and therapeutic procedures. Despite the tremendous technological advances, flexible endoscopy remains burdened by several drawbacks, such as the lack of proficiency, instability, triangulation of instruments in space, and force transmission to perform advanced procedures [1,2,3,4]. Moreover, techniques such as endoscopic submucosal dissection (ESD) are associated with long learning curves and an elevated rate of complications that limit their use to a restricted number of centers and operators [2, 4]. To overcome such limitations, several robots in endoscopy have been developed in the past decades, with the goal of improving the effectiveness, safety, and reliability of established procedures, as well as to expand the field of possible interventions [4]. In parallel with the evolution of endoscopy robots, artificial intelligence (AI) is also rapidly evolving. Although these two fields are evolving independently, there is a strict connection between them. Today’s endoscopic systems with AI mostly have a first or second degree of automation. This means that while the endoscopist is advancing the colonoscope the AI system recognizes polyps. For comparison, self-driving cars already have a fourth degree of automation onboard. This means that these cars can recognize the road and can drive in autonomy, commanded by AI systems. The connection between robotic endoscopy and AI will inevitably bring endoscopes to the fourth degree of automation. This means that the endoscope will advance, recognize, and resect the lesion alone. To better understand these two fields and all available current systems, a comprehensive review of the English-language literature on the use of Robotic Endoscopy and AI was performed using MEDLINE and Web of Science databases over the past 5 years up to May 2020. The following search strategy was used: [((“Robotics”) OR (“Robot”) AND (“Endoscopy”)) OR ((“Endoscopy robots”) OR ((“Robotic platform”) AND (“Endoscopy”))) OR ((“Artificial intelligence”) AND (“deep learning”))) NOT ((laparoscopic) OR (laparoscopy))]. Manual bibliographical searching was also performed to include any additional potentially relevant studies. BO, IB, and LGP screened the results and performed the study selection.

Types of Endoscopy Robotics in Gastrointestinal Endoscopy

Currently, there are three types of endoscopy robotics: robot-assisted flexible endoscopy platforms, active flexible endoscopy platforms, and robotic-driven flexible instrumentation/robotic flexible endoscopy with therapeutic functions (Tables 1, 2, and 3, respectively).

Table 1 Robot-assisted flexible endoscopy platforms
Full size table
Table 2 Active flexible endoscopy platforms
Full size table
Table 3 Robotic-driven flexible instrumentation/robotic flexible endoscopy With therapeutic functions
Full size table

Robot-assisted flexible Eendoscopy

The EndoDrive (ECE Medical Products, Erlangen, Germany) is the first marketed platform allowing a mechanical-assisted scope insertion and retraction during colonoscopy. This system is provided with a rotatable engine activated by a foot pedal that allows scope advancement leaving both hands free for instrument handling. Note that, manual rotation of the endoscopic shafts is required [5].

Colonoscopy with robotic steering and automated lumen centralization (RS-ALC, Enschede, Netherlands) (non-approved) is a colonoscope platform provided with electromechanically controlled wheels that allow automated lumen centralization. A preclinical study showed that this system was able to improve the intubation time and the polyp detection rate among novices but not expert endoscopists [6].

The master–slave endoscopic operation robot (EOR; Kyushu Institute of Technology, Kitakyushu, Japan) is a robotic system mounted on a standard colonoscope that allows advancing, steering, stabilizing, and rotating with the aid of two joysticks [7•, 8]. Haptic feedback (force sensation) has been installed in the latest version of the system, and a four-axis manipulation of the endoscope has become manageable with one hand [9]. More recently, the same authors have developed a remote-controlled master–slave robot capable of remotely operating up to three different endoscopic instruments (knife forceps, grasping, and injection needles), named the endoscopic therapeutic robot system (ETRS) [7•].

Another master–slave robot designed in 2017 by Woo et al. was a bicomponent and consisted of a tilting device and an insertion/rotation device with a grip similar to the conventional colonoscope. This robot also had a haptic feedback function. The robot enables colonoscope’s insertion up to 1.5 m, rotation of 360 degrees, and tilting up to a ± 180° angle [10].

In 2017, Li et al. developed another master–slave robotic gastroscope consisting of a gastroscope intervention mechanism and a pneumatic-driven clamping function, enabling uniform pneumatic pressure around the scope when compared to EndoDrive and the EOR devices [11].

In 2019, a revolute joint-based auxiliary transluminal endoscopic robot (REXTER) (not approved) was developed as a detachable robotic arm that can be installed on the tip of a conventional endoscope. The system can grasp and lift the mucosal layer allowing it to perform ESD with better visibility of the submucosal layer. The device was tested in an in vitro study revealing that the robot-assisted method was associated with a significantly lower rate of perforation when compared to the conventional method (1/10 vs 6/10) for unskilled operators without significant difference in terms of operation time. No difference was noted in terms of operation time and rate of perforation for skilled operators [12].

A robotic add-on steering module with a joystick interface was developed by Ruiter et al. to enable a single operator to control both the scope and the instruments [13]. A remote controller is held in the left hand and controls the endoscope tip angulation, air inflation, water injection, and suction, whereas the right hand actuates endoscope shaft introduction, rotation, and withdrawal. In a feasibility study on 22 subjects, cecal intubation was successful in 68% of cases in a median time of 21 (16–36) min [14]. When comparing the results obtained among experts and novices, no difference in the visualization performance was reported, whereas to reach their intubation time, experts required 6 ± 6 sessions, while novices required 12 ± 6 [15].

Active Flexible Endoscopy

Aer-O-Scope (GI View Ltd., Ramat Gan, Israel) is a single-use flexible colonoscope that utilizes a propulsion system for colonic intubation and scanning (Fig. 1). The propulsion is activated by the inflation of two balloons and by CO2 insufflation between them, generating a pneumatic force that ensures the advance of the scope without the need for pushing force [16, 17]. To reduce the risk of intestinal damages, specific electronic sensors ensure that the operating pressure does not exceed 60 mbar [18]. The system is provided with a 360° Omni-directional visualization and with a multi-lumen tube including channels for irrigation, insufflation, suction, and therapeutic access. A human study on 58 subjects showed that the Aer-O-Scope achieved cecum intubation in 98.2% of cases, with a polyp detection rate of 87.5% when compared to conventional colonoscopy [19]. The system received the CE mark and FDA approval in 2016 [16].

Fig. 1
figure 1

The Aer-O-scope. (With permission from GI-View, http://www.giview.com/)

Full size image

The NeoGuide Endoscopy System (NES) (NeoGuide Systems, Inc., Los Gatos, CA, USA) is a specially designed colonoscope consisting of 16 articulated electromechanically controlled segments aimed to precisely follow the colonic anatomy and to prevent loop formation. As the NES is inserted, the system automatically generates real-time 3D images of the colon, enabling the visualization of the scope position, loops, and pathological findings [20]. In the human pilot study, NES was used in 11 patients, resulting in cecum intubation in 10 cases [21]. The FDA approved the NeoGuide system in 2006 [22].

The Endotics (ERA Endoscopy SRL, Peccioli, Italy) (not FDA approved, received CE mark) is a self-propelling pneumatically-driven flexible colonoscope (Fig. 2a and b). It has a steerable 7.5 mm tip provided with a light source and a camera, a flexible body, and a thin tail. The locomotion is achieved by an inch-worm mechanism: the front segment anchors to the colon while the central segment is contracted to bring the rear segment forward where it is attached, allowing sequential anchoring and progression. The device has one channel for insufflation and one for suction and is provided with a 3-mm operative channel for therapeutic procedures. The Endotics system is remotely controlled by a hand-held console that allows endoscopic steering of 180° in every direction [23]. The main advantage of Endotics has been the low discomfort reported by the patients [23, 24], even though the cecum intubation rate has been reported as very low (27.5% compared to 82.5% with conventional colonoscopy) in a pilot study on 40 patients [23]. In a clinical trial on 71 patients, Tumino et al. reported that cecal intubation was achieved in 81.6% of cases (vs 94.3% with conventional colonoscopy, P = 0.03) within a longer time (45.1 ± 18.5 min vs. 23.7 ± 7.2 min, P < 0.0001) and with a lower polyp detection rate [24]. More recently, the same authors published a retrospective study showing that the Endotics system was able to achieve a complete colonoscopy in 93.1% of cases previously failed with conventional endoscopes (95% performance) [24]. In a single-center prospective study, the Endotics system was evaluated by an experienced endoscopist starting his training with the device on two consecutive blocks of patients (group A 27 patients, group B 28 patients). Cecal intubation rate was 92.7 vs 100% in groups A and B, respectively (P = 0.05) within 55 vs 22 min (P = 0.0007). Moreover, overall polyp and adenoma detection rates were 26.7 and 40%, and 92.7% of patients judged the procedure as no or mild distress, expressing the willingness to repeat it [25].

Fig. 2
figure 2

a The Endotics endoscope. b Polypectomy with the Endotics colonoscope. (With permission from Endotics, http://www.endotics.com/)

Full size image

The Invendoscope (Invendo Medical GmbH, Kissing, Germany) is a single-use motor-controlled colonoscope maneuvered by a hand-held joystick (Fig. 3). The 170 cm scope is provided with a 3.1-mm working channel and a 180° bending tip, and allows insufflation, rinsing, and suction, similarly to conventional endoscopes. During the introduction, an inverted sleeve progressively unrolls to protect the inserted section of the endoscope. By doing so, the forces exerted by the instrument on the colonic wall are minimized for both diagnostic and therapeutic procedures. The Invendoscope was evaluated in a first prospective pilot study on 34 patients showing a cecum intubation rate of 82% without complications [26]; more recently, a newly designed Invendoscope, the InvendoSC20 was tested on 61 patients and cecum intubation was reached in 98.4% of patients with a median time of 15 min [27]. Similar results were achieved by Straulino et al. (cecal intubation rate of 95% within 14.23 min) despite a relatively high rate of endoscope loop formation (70%) [28]. The Invendoscope SC210 has received the CE mark and FDA approval in 2017 [29].

Fig. 3
figure 3

Invendoscope. (With permission from Invendo Medical, part of the Ambu Group, https://www.ambuusa.com/)

Full size image

The Meshworm is a novel worm-inspired colonoscope composed of three segments driven by tendons connected with motors. Each segment can be compressed or extended depending on the motor’s rotation, and both the front and the rear segments are actuated by additional motors and are able to bend up to 90° along two axes. To date, a 50 cm long prototype has been developed, but the size of this system can be increased easily. Human studies will be required in the future to assess the efficacy and safety of this system in a real-life setting [30].

The ColonoSight (Stryker GI Ltd., Haifa, Israel) is a self-advancing, electronically controlled, reusable Colonoscope (EndoSight) covered by a single-use plastic sleeve (ColonoSleeve) that progressively unfolds through air insufflation. The scope insertion is powered by an electro-pneumatic unit that inflates the outer sheath without air introduction inside the colonic lumen. After reaching the caecum, the air is delivered distally to the tip enabling mucosal visualization. The ColonoSight device was shown to be associated with a cecal intubation rate of 90% in a mean time of 11.2 (+ − 6.5) min in a multicentre trial on 178 subjects [31]. FDA approved this system in 2004.

Robotic-Driven Flexible Instrumentation/Robotic Flexible Endoscopy with Therapeutic Functions

The ISIS-S SCOPE (Karl Storz/IRCAD, Tuttlingen, Germany) (not approved) is a robotized version of the Anubiscope (Karl Storz Endoskope, Tuttlingen, Germany) [32]. The 18-mm endoscope has a 35 cm passive shaft and a 22 cm bending segment with frontal jaws that allow instrument triangulation (Fig. 4). It has two 4.2 mm channels and 10 degrees of freedom and is electromechanically controlled through externally actuated traction wires [33]. Note that the common workspace is focused at 9 cm from the camera with a maximum aperture of 2.5 cm from the camera, which may be considered too wide for the gastrointestinal lumen [3].

Fig. 4
figure 4

The STARS system during Endoscopic Submucosal Dissection

Full size image

The Endoluminal Assistant for Surgical Endoscopy (EASE; KARL STORZ/IRCAD, Strasbourg, France, Tuttlingen, Germany) (not approved) is the evolution of the ISIS-S a master–slave system (Fig. 4) [34, 35]. The system is composed of a mobile master console and a detachable endoscope with a maximum diameter of 16 mm which is provided with three working channels (two 4.3-mm and one 3.2-mm). Several instruments including multiple graspers, a hook knife, an insulated tip knife, an endoscopic needle holder, and multiple conventional endoscopic instruments can be used with this device. Ten degrees of freedom are allowed with this system, and while sitting at the console, the operator can have two screens, one frontal with the endoscopic view and one graphical with the user interface that is mapping the position of both the scope and the instruments. In a comparative study on in–vivo porcine models, robot-assisted ESDs performed by a laparoscopic surgeon were compared with conventional ESD by an experienced endoscopist. Robot-assisted procedures were associated with a lower risk of perforation (5% vs 33.3%; P = 0.041) and of both dissection and procedure duration [29.42 ± 14.29 vs 43.43 ± 12.55 min (P = 0.01) and 33.36 ± 14.20 vs 47.38 ± 13.40 min (P = 0.01), respectively] [36].

The K-FLEX (EasyEndo Surgical, Daejeon, Korea) (not approved) is a robotic endoscopic platform applicable to advanced endoluminal surgery. The device is composed of a master console, a driving robot arm, a bendable overtube with two surgical instruments, for an overall diameter of 17 mm. The surgical arm has a joint configuration capable of precise movements [37].

The Flex Robotic System (Medrobotics, Raynham, Mass, USA) (not approved) is composed of a robotic endoscope and an operational console with a touchscreen, HD visual display, and a joystick. The endoscope was originally designed for lesions situated in the oropharynx, hypopharynx, and larynx [38, 39, 40•, 41], but has been recently modified to perform procedures in the lower GI-tract up to 25 cm from the anal margin [42]. The scope is provided with two 4-mm working channels, and several accessories such as scissors, needle drivers, graspers, and dissectors can be used. In a recently published randomized, controlled, pilot study comparing conventional ESD versus robotic-assisted ESD, the FLEX system has been associated with a higher percentage of complete en bloc resections (P < .0001) with a lower procedural time (34.1 vs 88.6 min, P = 0.001) and a lower rate of perforations (30% vs 60%, P = 0.18) [43•]. This system received FDA approval for the treatment of lesions located in the anus, rectum, and distal colon in 2017 [44].

The EndoMaster EASE® System (EndoMaster Pte. Ltd., Singapore) (not FDA approved) is a robotic arm mounted externally onto a double-channel endoscope (Fig. 5). All together this system can deliver up to nine degrees of freedom [45]. While one endoscopist holds the endoscope, one operator controls the master controller. This robot has been developed to perform both intraluminal and transluminal endoscopic procedures, and its performance has been evaluated in several animal studies [46,47,48,49,50,51,52]. In humans, a multicenter prospective study showed that ESD was effectively carried out in five patients with early-stage gastric neoplasia within 16 min (ranging 3–50 min) without complications [45]. The main limitations of the device are the limited maneuverability linked to the dimensions and the impossibility to exchange instruments [3]. This device, however, allows the operator to recreate manual human wrist movements and several accessories can be deployed at the tip of the endoscope [53].

Fig. 5
figure 5

The EndoMaster EASE® System with endoscopic view during intervention. (With permission from EndoMaster Pte. Ltd.)

Full size image

Endomina (Endo Tools Therapeutics, SA-ETT, Gosselies, Belgium) (not FDA approved) received CE Mark for the treatment of obesity. Endomina is a single-use over-the-scope triangulation device that allows creating a double plicature of the greater curvature through the use of interrupted sutures, thus creating the Endoscopic Sleeve Gastroplasty. The first generation of the instrument was electromechanically actuated by traction cables and is controlled by two joysticks [54]. The second generation is completely mechanical and its safety and feasibility have been evaluated in two studies by Huberty et al., reporting an excess weight loss of 31% at 6 months and 29% at 12 months without the occurrence of any severe adverse events (Fig. 6a–c,) [55, 56].

Fig. 6
figure 6

a The Endomina platform. b Endoscopic view during suturing c Final outcome after endoscopic sleeve gastroplasty for obesity

Full size image

Artificial Intelligence in Gastrointestinal Endoscopy

AI was first described in 1950 by Alan Turing in the book Computers and Intelligence. He described a simple test, known as the “Turing test,” to determine whether computers were capable of human intelligence [57]. However, several limitations in early models prevented wide-spread application to medicine. In the early 2000s, many of these limitations were overcome by the advent of deep learning. Now AI systems are capable of analyzing complex algorithms and self-learning and AI can be applied to clinical practice. The advent of data-driven deep learning (DL) overcame the inefficient and incomplete feature extraction that results from manual extraction by using traditional machine learning and this has revolutionized the research and development of AI [58, 59].

After having been trained by a large number of labeled images, the DL algorithm can classify enormous quantities of images. A variety of DL algorithms have thus been created.

Deep Learning and Convolutional Neural Networks

DL is complex structured learning and part of a big family of machine learning. This topic is very complex and for the purpose of this article, only basics will be approached. Convolutional neural networks (CNN) are part of deep learning, capable of medical image recognition or “analyzing visual imagery.” CNNs are designed to think like humans by using large image datasets that allow learning patterns in correlating images. Furthermore, CCNs are interconnected similarly to biological neurons. Once CNN has recognized a pattern, it can classify it just like humans do. These models are trained with big datasets of images. From here CNNs can recognize or not images that are similar to what they have learned or not. With time, this capability is increasingly becoming better and CNNs can even make predictions. For instance, CNN can be used to train algorithms to recognize a polyp in endoscopy. By using a large number of endoscopic images with different types of polyps and adenomas, a CNN can lead to recognize and classify colorectal polyps. This principle has gained an enormous fraction in GI endoscopy. Today, AI in GI endoscopy can be classified into two groups: computer-aided detection (CADe) system and computer-aided diagnosis (CADx) system. A CADe system is designed to detect and track GI lesions, while a CADx system focuses mainly on identifying and characterizing them. CADx has been used in the upper GI tract for the diagnosis of esophageal cancer, Helicobacter pylori (H. pylori) infection, and early gastric cancer.

Horie et al. described a CNN-based model capable of distinguishing between superficial esophageal cancer and advanced esophageal cancer with an accuracy of 98% [60]. AI has also been applied to improve imaging in Barrett’s esophagus. de Groof et al. recently developed a CAD system that was 90% sensitive and 88% specific (89% accurate) in classifying images as neoplastic or nondysplastic Barrett’s esophagus. The CAD system had higher accuracy than 53 non-expert endoscopists (88% vs 73%) [61]. Similarly, a real-time CAD system was developed and trained using 1480 malignant narrow-band images and 5191 precancerous narrow-band images and was able to differentiate early esophageal squamous cell carcinoma from precancerous lesions with 98% sensitivity and 95% specificity (area under the curve, .989) [62]. Several studies on H. pylori infection also showed that the CADx system had a higher accuracy than endoscopists in the diagnosis of Helicobacter Pylori infection [63]. Kanesaka et al. studied a CADx system to identify and delineate early gastric cancer in real-time magnifying NBI, with an accuracy of 96.3% [64]. Zhu et al. proposed a CNN CAD system for predicting the depth of invasion of gastric cancer (M/SM1 vs deeper than SM1). Its accuracy (89.16%) and specificity (95.56%) were significantly better than those of experienced endoscopists [65•]. CNN-based models have also been applied to detecting small-bowel capsule endoscopy anomalies [66,67,68,69]. The CAD system was used by Hassan et al. to draw inferences between bleeding and nonbleeding in capsule endoscopy with sensitivity and specificity up to 99% [66]. Similar to this, Xiao et al. [67] developed a CAD system that reaches >99% of performance score for GI bleeding detection in capsule endoscopy. Cazacu et al. reported using artificial neural networks (ANN) in conjunction with EUS to assist in differentiating chronic pancreatitis from pancreatic cancer, with a sensitivity of 95% and specificity of 94% [70]. Computer-assisted diagnosis can be applied to colonoscopy to improve the detection of and differentiation between benign versus malignant colon polyps [71]. A recent randomized controlled trial of 1058 patients demonstrated a significant increase in adenoma detection rates with the use of CAD compared with standard colonoscopy (29% vs 20%, P < .001), with an increased detection of diminutive adenomas (185 vs 102, P < .001) and hyperplastic polyps (114 vs 52, P < .001). There was no statistical difference in the detection of larger adenomas [72••]. A CNN was developed to determine invasiveness of colorectal mass lesions suspected to be cancer where a diagnostic accuracy of 81.2% was achieved [73]. Although much of the technology reported has been proof of concept, two systems are approved for use. ENDOANGEL (Wuhan EndoAngel Medical Technology Company, Wuhan, China), a CNN-based system developed in 2019, can provide an objective assessment of bowel preparation every 30 s during the withdrawal phase of a colonoscopy, achieving a 91.89% accuracy [74]. A recent randomized controlled study used the ENDOANGEL device to monitor withdrawal speed and colonoscopy withdrawal time showing significant improvement in adenoma detection rates (ENDOANGEL-assisted colonoscopy versus unassisted colonoscopy: 17% vs 8%) [75]. The second system, GI Genius (Medtronic, Minneapolis, Minn, USA), is an AI-enhanced endoscopy aid device developed to identify colorectal polyps by providing a visual marker on a live video feed during endoscopic examination. It is approved for use in Europe and undergoing clinical evaluation in the United States. In a validation study, GI Genius had an overall sensitivity per lesion of 99.7% and detected polyps faster than endoscopists in 82% of cases [76]. In a recent randomized controlled trial, Repici et al. demonstrated a 14% increase in adenoma detection rates using this CAD system [77••]. Byrne et al. developed a CADx system for differentiating between adenomatous and hyperplastic diminutive colorectal polyps using NBI endoscopic videos. When tested on 106 real videos containing small polyps, overall accuracy, sensitivity, and specificity of this model in identifying adenomas and proliferative polyps were 94%, 98%, and 83%, respectively [78]. Mori et al. conducted a real-time CADx for endocytoscopy (×520 ultrahigh magnification images with staining and NBI, respectively), which was verified by a single-group prospective study [79]. CADe was shown to achieve the threshold of preservation and incorporation of valuable endoscopic innovations for a “diagnose-and-leave” strategy, namely a > 90% negative predictive value (NPV) for diminutive adenomas [80]. Moreover, some studies have attempted to apply AI to inflammatory bowel diseases. Ozawa et al. constructed a CADx system using a CNN to identify normal mucosa (Mayo 0) and the mucosal healing state (Mayo 0–1) with the area under the receiver operating characteristic curves (AUROCs) of 0.86 and 0.98 [81]. Maeda et al. also developed a CADx system to predict persistent histological inflammation in patients with ulcerative colitis and achieved a high accuracy (91%) and specificity (97%) [82]. Computer vision AI algorithms can also be used to “observe” the technical aspects of the procedure and then document the activity. They can be used for automated cecum detection, automated devices recognition, and automated polyp size measurement [83].

Conclusion

Robotic endoscopy is a large field of innovations that are gaining popularity owing to the minimally invasive diagnostic and increasingly more advanced therapeutic applications that can be provided. The ideal robotic endoscope should be affordable, versatile, effective in achieving precise movements with a comfortable position for the endoscopist, and, most of all, should be “smart” with incorporated AI. Many robotic endoscopy systems have been developed, but the majority of them have not reached clinical application. We believe that in the future, healthcare professionals will take advantage of self-driven robotic endoscopes that will perform complex endoscopic procedures such as AI-led ESD.