Authors:
Carlos Robles-Medranda1, Juan Alcivar-Vasquez1, Michel Kahaleh5, Isaac Raijman3,4, Rastislav Kunda2, Amy Tyberg5, Avik Sarkar5, Haroon Shahid5, Juan C. Mendez6, Jorge Rodriguez1, Merfea, Ruxandra C. 1, Barreto Perez, Jonathan1, Martha Arevalo-Mora1, Miguel Puga-Tejada1, Daniel Calle-Loffredo1, Haydee Alvarado1, Hannah P. Lukashok1.
Affiliations:
- Instituto Ecuatoriano de Enfermedades Digestivas, Guayaquil, Ecuador
- Department of Advanced Interventional Endoscopy, Universitair Ziekenhuis Brussel (UZB)/Vrije Universiteit Brussel (VUB)
- Houston Methodist Hospital
- Baylor Saint Luke’s Medical Center
- Robert Wood Johnson Medical School Rutgers University
- Mdconsgroup, Artificial Intelligence Department
Corresponding Author:
Carlos Robles-Medranda, MD
Head of the Endoscopy Division
Instituto Ecuatoriano de Enfermedades Digestivas
Av. Abel Romero Castillo y Av. Juan Tanca Marengo SN
Torre Vitalis II, Office 405-406
Guayaquil 090505, Ecuador
Phone: +593-42109180
Email address: carlosoakm@yahoo.es
Disclosure:
Carlos Robles-Medranda is a key opinion leader and consultant for Pentax Medical, Boston Scientific, Steris, Medtronic, Motus, Micro-tech, G-Tech Medical Supply, CREO Medical, and Mdconsgroup. Ratislav Kunda is a consultant of Olympus, Boston Scientific, Omega Medical Imaging, M.I.Tech, Tigen Pharma, Ambu. The other authors declare no conflicts of interest.
Abstract
Background and aims: Digital single-operator cholangioscopy (DSOC) findings achieve high diagnostic accuracy for neoplastic bile duct lesions; however, endoscopists’ intra and interobserver agreements vary widely. We have recently proposed an AI model to classify bile duct lesions during real-time DSOC and currently pursue clinical validation of our AI model, compared with high DSOC experienced endoscopists.
Methods: A multi-center diagnostic trial. Four DSOC experts endoscopists (blinded to clinical records), observed and classified a set of videos among neoplastic or non-neoplastic bile duct lesions. All videos were blinded for DSOC experts and for the AI software (Mdconsgroup, Guayaquil, Ecuador). The neoplastic bile duct criteria are in accordance with the Robles-Medranda et al and the Mendoza classifications. The experts assessed neoplastic bile duct by presence or absence of disaggregated criteria. Likewise, the statistical software computed disaggregated answers. The final diagnosis of malignancy was based on histological results, and 1-year clinical follow-up outcomes. NCT05147389.
Results: A total of 170 videos from 170 patients from 4 different centers were analyzed with the AI model. There was an equal distribution among neoplastic and non-neoplastic DSOC diagnosis (Table 1). DSOC AI software achieved statistically significant accuracy values (p <0.001) for neoplastic diagnosis with a ≥ 90% sensitivity, ≥ 68% specificity, ≥ 65% positive and ≥ 83% negative predictive values when compared with endoscopist expert.
Conclusions: The proposed AI model accurately recognized between neoplastic and non-neoplastic bile duct lesions with good accuracy, being statistically significant over experts in DSOC. This model may shorten learning curves time in less experienced endoscopists, while attaining accurate biliary lesion recognition skills.
Table 1. Baseline data characteristics.
| Total (N=170) | Neoplasia (N=85) | Non-neoplasia (N=85) | |
| Age (years), median (IQR) | 62.5 (57.0 – 68.8) | 64.0 (59.0 – 71.0) | 59.0 (52.0 – 65.0) |
| Young adults (18-39) | 2 (1.2) | – | 2 (2.4) |
| Adults (40-64) | 104 (61.2) | 45 (52.9) | 59 (69.4) |
| Elderly (≥65) | 64 (37.6) | 40 (47.1) | 24 (28.2) |
| Gender (female), n (%) | 79 (46.5) | 45 (52.9) | 34 (40.0) |
| DSOC indication, n (%) | |||
| Suspicion of tumor | 58 (34.1) | 49 (57.6) | 9 (10.6) |
| Indeterminate stenosis | 46 (27.1) | 15 (17.6) | 31 (36.5) |
| Indeterminate dilation | 31 (18.2) | 21 (24.7) | 10 (11.8) |
| Filling defect | 35 (20.6) | – | 35 (41.2) |
| Jaundice, n (%) | 127 (74.7) | 77 (90.6) | 50 (58.8) |
| Pruritus, n (%) | 59 (34.7) | 34 (40.0) | 25 (29.4) |
| Abdominal pain, n (%) | 76 (44.7) | 56 (65.9) | 20 (23.5) |
| Weight loss, n (%) | 77 (45.3) | 73 (85.9) | 4 (4.7) |
| Total bilirubin, median (IQR) | 3.89 (2.50 – 9.00) | 9.00 (4.50 – 22.6) | 3.00 (0.900 – 3.50) |
| Stricture location, n (%) | |||
| Common bile duct | 48 (28.2) | 13 (15.3) | 35 (41.2) |
| Hilium | 48 (28.2) | 39 (45.9) | 9 (10.6) |
| Common hepatic duct | 70 (41.2) | 33 (38.8) | 37 (43.5) |
| Intrahepatic | 4 (2.4) | – | 4 (4.7) |
| Cystic duct | – | – | – |
| Previous ERCP, n (%) | 54 (31.8) | 19 (22.4) | 35 (41.2) |
| Previous stent placement, n (%) | 44 (25.9) | 15 (17.6) | 29 (34.1) |
| DSOC diagnosis, (%) | |||
| Non-neoplasia | 85 (50.0) | – | 85 (100.0) |
| Neoplasia | 85 (50.0) | 85 (100.0) | – |
| Biospy diagnosis, (%) | |||
| Adenocarcinoma | 11 (6.5) | 11 (12.9) | – |
| Atypical | 6 (3.5) | 6 (7.1) | – |
| Cholangiocarcinoma | 67 (39.4) | 67 (78.8) | – |
| Inflammatory | 69 (40.6) | – | 69 (81.2) |
| IPMN of the bile duct | 1 (0.6) | 1 (1.2) | – |
| Normal biliary tissue | 2 (1.2) | – | 2 (2.4) |
| Primary sclerosing cholangitis | 14 (8.2) | – | 14 (16.5) |
Table 2. AI overall accuracy for diagnosing neoplasia comparing with single-endoscopist expertise based on CRM classification system and Mendoza consensus.
| Sensitivity | Specificity | PPV | NPV | Agreement | ROC curves | |
| Expert 1 (n=94) | ||||||
| AI | 46/47; 97.87% (88.71 – 99.95) | 28/47; 59.57% (44.27 – 73.63) | 46/65; 70.77% (58.17 – 81.4) | 28/29; 96.55% (82.24 – 99.91) | 74/94; 78.72% (69.07 – 86.49) | 0.848 |
| CRM criteria | 43/47; 91.49% (79.62 – 97.63) | 36/47; 76.6% (61.97 – 87.7) | 43/54; 79.63% (66.47 – 89.37) | 36/40; 90% (76.34 – 97.21) | 79/94; 84.04% (75.05 – 90.78) | 0.836 (P=.816) |
| Mendoza criteria | 47/47; 100% (92.45 – 100) | 4/47; 8.51% (2.37 – 20.38) | 47/90; 52.22% (41.43 – 62.87) | 4/4; 100% (39.76 – 100) | 51/94; 54.26% (43.66 – 64.58) | 0.761 (P=.077) |
| Expert 2 (n=135) | ||||||
| AI | 59/67; 88.06% (77.82 – 94.7) | 46/68; 67.65% (55.21 – 78.49) | 59/ 81; 72.84% (61.81 – 82.13) | 46/54; 85.19% (72.88 – 93.38) | 105/135; 77.78% (69.82 – 84.48) | 0.790 |
| CRM criteria | 60/67; 89.55% (79.65 – 95.7) | 38/68; 55.88% (43.32 – 67.92) | 60/ 90; 66.67% (55.95 – 76.26) | 38/45; 84.44% (70.54 – 93.51) | 98/135; 72.59% (64.25 – 79.91) | 0.755 (P=.497) |
| Mendoza criteria | 67/67; 100% (94.64 – 100) | 29/68; 42.65% (30.72 – 55.23) | 67/106; 63.21% (53.29 – 72.37) | 29/29; 100% (88.06 – 100) | 96/135; 71.11% (62.69 – 78.58) | 0.816 (P=.538) |
| Expert 3 (n=136) | ||||||
| AI | 60/68; 88.24% (78.13 – 94.78) | 46/68; 67.65% (55.21 – 78.49) | 60/ 82; 73.17% (62.24 – 82.36) | 46/54; 85.19% (72.88 – 93.38) | 106/136; 77.94% (70.03 – 84.59) | 0.791 |
| CRM criteria | 57/68; 83.82% (72.9 – 91.64) | 44/68; 64.71% (52.17 – 75.92) | 57/ 81; 70.37% (59.19 – 80.01) | 44/55; 80% (67.03 – 89.57) | 101/136; 74.26% (66.07 – 81.37) | 0.803 (P=.777) |
| Mendoza criteria | 68/68; 100% (94.72 – 100) | 24/68; 35.29% (24.08 – 47.83) | 68/112; 60.71% (51.04 – 69.81) | 24/24; 100% (85.75 – 100) | 92/136; 67.65% (59.1 – 75.41) | 0.751 (P=.433) |
| Expert 4 (n=136) | ||||||
| AI | 67/68; 98.53% (92.08 – 99.96) | 42/68; 61.76% (49.18 – 73.29) | 67/ 93; 72.04% (61.78 – 80.86) | 42/43; 97.67% (87.71 – 99.94) | 109/136; 80.15% (72.45 – 86.49) | 0.848 |
| CRM criteria | 63/68; 92.65% (83.67 – 97.57) | 33/68; 48.53% (36.22 – 60.97) | 63/ 98; 64.29% (53.97 – 73.71) | 33/38; 86.84% (71.91 – 95.59) | 96/136; 70.59% (62.17 – 78.09) | 0.753 (P<.01) |
| Mendoza criteria | 68/68; 100% (94.72 – 100) | 2/68; 2.94% (0.36 – 10.22) | 68/134; 50.75% (41.98 – 59.48) | 2/ 2; 100% (15.81 – 100) | 70/136; 51.47% (42.75 – 60.12) | 0.755 (P<.05) |
References
- Kahaleh M, Raijman I, Gaidhane M, Tyberg A, Sethi A, Slivka A, et al. Digital Cholangioscopic Interpretation: When North Meets the South. Dig Dis Sci [Internet]. 2021 [cited 2021 Nov 16]; Available from: https://pubmed.ncbi.nlm.nih.gov/33783691/
- Sethi A, Tyberg A, Slivka A, Adler DG, Desai AP, Sejpal D V., et al. Digital Single-operator Cholangioscopy (DSOC) Improves Interobserver Agreement (IOA) and Accuracy for Evaluation of Indeterminate Biliary Strictures: The Monaco Classification. J Clin Gastroenterol [Internet]. 2020 [cited 2021 Nov 16]; Available from: https://pubmed.ncbi.nlm.nih.gov/32040050/
- Kahaleh M, Gaidhane M, Shahid HM, Tyberg A, Sarkar A, Ardengh JC, et al. Digital single-operator cholangioscopy interobserver study using a new classification: the Mendoza Classification (with video). Gastrointest Endosc [Internet]. 2021 Aug [cited 2021 Nov 16]; Available from: https://pubmed.ncbi.nlm.nih.gov/34478737/
- Robles-Medranda C, Valero M, Soria-Alcivar M, Puga-Tejada M, Oleas R, Ospina-Arboleda J, et al. Reliability and accuracy of a novel classification system using peroral cholangioscopy for the diagnosis of bile duct lesions. Endoscopy [Internet]. 2018 [cited 2021 Nov 16];50(11):1059–70. Available from: https://pubmed.ncbi.nlm.nih.gov/29954008/
- Robles-Medranda C, Oleas R, Sánchez-Carriel M, Olmos JI, Alcívar-Vásquez J, Puga-Tejada M, et al. Vascularity can distinguish neoplastic from non-neoplastic bile duct lesions during digital single-operator cholangioscopy. Gastrointest Endosc [Internet]. 2021 Apr 1 [cited 2021 Nov 16];93(4):935–41. Available from: https://pubmed.ncbi.nlm.nih.gov/32707155/
- Saraiva MM, Ribeiro T, Ferreira JPS, Boas FV, Afonso J, Santos AL, et al. Artificial intelligence for automatic diagnosis of biliary stricture malignancy status in single-operator cholangioscopy: a pilot study. Gastrointest Endosc [Internet]. 2021 Sep [cited 2021 Nov 16]; Available from: https://pubmed.ncbi.nlm.nih.gov/34508767/