Table 2 Comparison of diagnostic performance between CLA-HDM and six radiologists, and between radiologists with and without AI assistance
Radiologists | Internal testing cohort (n = 171) | External testing cohort 1 (n = 105) | External testing cohort 2 (n = 92) | ||||
|---|---|---|---|---|---|---|---|
Without AI (%) | With AI (%) | Without AI (%) | With AI (%) | Without AI (%) | With AI (%) | ||
1 | Accuracy | 84.2 (81.3, 87.4) | 86.8 (83.9, 89.8)↑# | 82.9 (79.1, 86.7) | 83.8 (80.0, 87.6)↑ | 83.2 (78.8, 87.0) | 82.6 (78.3, 86.4) |
Sensitivity | 68.4 (62.6, 74.9) | 73.7 (67.8, 79.5)↑ | 65.7 (58.1, 73.3) | 67.6 (60.0, 75.2)↑ | 66.3 (57.6, 73.9) | 65.2 (56.5, 72.8) | |
Specificity | 89.5 (87.5, 91.6) | 91.2 (89.3, 93.2)↑ | 88.6 (86.0, 91.1) | 89.2 (86.7, 91.8)↑ | 88.8 (85.9, 91.3) | 88.4 (85.5, 90.9) | |
2 | Accuracy | 82.2 (79.5, 85.1) ** | 85.7 (83.0, 88.3)↑## | 80.5 (76.7, 84.8) | 84.3 (80.5, 88.1)↑# | 79.9 (76.1, 84.2) | 82.6 (78.8, 87.0)↑ |
Sensitivity | 64.3 (59.1, 70.2) * | 71.4 (66.1, 76.6)↑## | 60.9 (53.3, 69.5) | 68.6 (61.0, 76.2)↑ | 59.8 (52.2, 68.5) | 65.2 (57.6, 73.9)↑ | |
Specificity | 88.1 (86.4, 90.1) | 90.5 (88.7, 92.2)↑ | 86.9 (84.4, 89.8) | 89.5 (87.0, 92.1)↑ | 86.6 (84.1, 89.5) | 88.4 (85.9, 91.3)↑ | |
3 | Accuracy | 81.0 (78.1, 83.9) ** | 84.8 (82.2, 87.7)↑## | 80.5 (76.2, 84.8) | 80.5 (76.7, 84.3)↑ | 79.4 (75.0, 83.7) | 81.0 (76.6, 85.3)↑ |
Sensitivity | 62.0 (56.1, 67.8) * | 69.6 (64.3, 75.4)↑# | 60.9 (52.4, 69.5) | 61.0 (53.3, 68.6)↑ | 58.7 (50.0, 67.4) | 62.0 (53.3, 70.7)↑ | |
Specificity | 87.3 (85.4, 89.3) * | 89.9 (88.1, 91.8)↑ | 87.0 (84.1, 89.8) | 87.0 (84.4, 89.5)↑ | 86.2 (83.3, 89.1) | 87.3 (84.4, 90.2)↑ | |
4 | Accuracy | 81.0 (78.1, 84.2) ** | 86.3 (83.6, 89.2)↑### | 75.2 (71.4, 79.5) ** | 81.0 (77.1, 84.8)↑## | 77.7 (73.9, 82.2) | 82.1 (78.3, 86.4)↑# |
Sensitivity | 62.0 (56.1, 68.4) * | 72.5 (67.3, 78.4)↑### | 50.5 (42.9, 59.1)* | 61.9 (54.3, 69.5)↑# | 55.4 (47.8, 64.1) * | 64.1 (56.5, 72.8)↑ | |
Specificity | 87.3 (85.4, 89.5) * | 90.8 (89.1, 92.8)↑ | 83.5 (81.0, 86.4) | 87.3 (84.8, 89.8)↑ | 85.1 (82.6, 88.0) | 88.0 (85.5, 90.9)↑ | |
5 | Accuracy | 78.7 (75.4, 81.9) *** | 79.8 (76.9, 83.0)↑ | 77.1 (72.9, 81.0) * | 82.4 (78.6, 86.7)↑# | 76.6 (72.3, 81.0) | 79.4 (75.5, 84.2)↑ |
Sensitivity | 57.3 (50.9, 63.7) ** | 59.7 (53.8, 66.1)↑ | 54.3 (45.7, 61.9) | 64.8 (57.1, 73.3)↑# | 53.3 (44.6, 62.0) * | 58.7 (51.1, 68.5)↑ | |
Specificity | 85.7 (83.6, 87.9) ** | 86.6 (84.6, 88.7)↑ | 84.8 (81.9, 87.3) | 88.3 (85.7, 91.1)↑ | 84.4 (81.5, 87.3) * | 86.2 (83.7, 89.5)↑ | |
6 | Accuracy | 76.9 (73.7, 80.1) *** | 82.2 (78.9, 85.1)↑## | 77.1 (73.3, 81.0) * | 81.9 (78.1, 86.2)↑# | 74.5 (70.1, 78.8) * | 78.3 (73.9, 82.6)↑ |
Sensitivity | 53.8 (47.4, 60.2) *** | 64.3 (57.9, 70.2)↑## | 54.3 (46.7, 61.9) | 63.8 (56.2, 72.4)↑ | 48.9 (40.2, 57.6) ** | 56.5 (47.8, 65.2)↑ | |
Specificity | 84.6 (82.5, 86.7) ** | 88.1 (85.9, 90.1)↑ | 84.8 (82.2, 87.3) | 87.9 (85.4, 90.8)↑ | 83.0 (80.1, 85.9) ** | 85.5 (82.6, 88.4)↑ | |