Introduction: The Challenge of Imaging Heterogeneity
Optical coherence tomography (OCT) has revolutionized the field of ophthalmology, providing high-resolution, cross-sectional visualizations of the retina that are essential for the diagnosis and management of macular diseases. However, the rapid proliferation of OCT technology has led to a fragmented landscape of imaging hardware. Different manufacturers, such as Heidelberg Engineering (Spectralis) and Zeiss (Cirrus), utilize distinct optical setups and processing algorithms, resulting in variations in image contrast, noise levels, and spatial resolution. For artificial intelligence (AI) and deep learning (DL) models, this heterogeneity presents a significant hurdle known as domain shift. A model trained on images from one vendor often performs poorly when applied to images from another, limiting the scalability and clinical utility of AI in diverse healthcare environments.
In a landmark study published in JAMA Ophthalmology, Tang and colleagues address this challenge by developing and validating a domain-shift AI technology designed for vendor-agnostic multidisease detection from 3D OCT scans. By leveraging advanced unsupervised test-time domain adaptation, the researchers aimed to create a robust diagnostic tool capable of maintaining high performance across different hardware platforms and clinical settings.
Highlights
High Diagnostic Accuracy Across Vendors
The model demonstrated robust performance (AUROC up to 0.999) across different OCT vendors despite being trained on a single manufacturer’s data.
Safety-First Triage
Negative predictive values (NPV) exceeded 97.5% across all testing datasets, ensuring a low rate of missed pathology in clinical triage scenarios.
Managing the Unknown
The introduction of an uncertain category allowed the model to identify and flag rare or previously unseen macular conditions with high specificity.
Domain Adaptation Success
The use of Test Entropy as an unsupervised adaptation method effectively bridged the gap between different imaging domains without requiring extensive retraining.
Study Design and Methodology
This multicenter retrospective cohort study utilized a massive dataset comprising 18,992 OCT scans from 6,005 patients. The data were sourced from a wide array of institutions, including tertiary eye hospitals, private centers, and open databases across Hong Kong and Vietnam. The study period spanned from January 2008 through September 2022, with model development and analysis occurring between 2022 and 2024.
The core of the technology is a Residual Neural Network (ResNet) 3D model. Unlike many previous models that focus on 2D slices (B-scans), this architecture analyzes the entire 3D volume, capturing the full spatial context of macular pathology. To address the vendor-agnostic requirement, the researchers employed an unsupervised test-time domain adaptation method called Test Entropy. This technique allows the model to adjust its internal parameters in real-time based on the statistical properties of the incoming test data, effectively normalizing the differences between images from Vendor 1 (Spectralis) and Vendor 2 (Cirrus).
The model was trained exclusively on 3D scans from Vendor 1. It was then subjected to rigorous external testing against nine datasets, which included 3D scans from Vendor 1, 3D scans from Vendor 2, and even 2D scans to test its versatility. A critical innovation was the triage module, which categorized scans into urgent, semi-urgent, and routine based on the detected pathology, and an uncertain category for out-of-distribution (OOD) cases.
Results: Performance and Clinical Reliability
The primary outcomes measured were the area under the receiver operating characteristic curve (AUROC), positive predictive value (PPV), and negative predictive value (NPV). The results were consistently high across the board.
For 3D scans from Vendor 1, the AUROC ranged from 0.779 to 0.999. More impressively, when the model was applied to 3D scans from Vendor 2—data it had never seen during training—the AUROC remained strong, ranging from 0.754 to 0.991. When applied to 2D scans, the model still achieved an AUROC between 0.801 and 0.950, highlighting its adaptability.
From a clinical safety perspective, the NPV is perhaps the most critical metric. All micro-average NPVs exceeded 97.5%. This means that if the AI labels a scan as normal or routine, there is a very high probability that it has not missed a significant disease. In terms of triage, the clinically important miss rate for urgent cases (such as active neovascular age-related macular degeneration) was 6.16% for Vendor 1 and 6.70% for Vendor 2. For semi-urgent cases, the miss rates were 4.41% and 8.67%, respectively.
Addressing the Uncertain: The OOD Challenge
One of the most significant risks in clinical AI is the occurrence of conditions that the model was not trained to recognize. If a model is forced to classify a rare condition into a pre-defined category, it may produce a confidently wrong diagnosis. Tang et al. addressed this by incorporating an uncertain category. This module demonstrated a specificity greater than 95.0% and an accuracy greater than 92.7% across external datasets. While the sensitivity for this category varied, its presence serves as a crucial safety valve, flagging complex or rare cases for human review rather than providing an incorrect automated diagnosis.
Expert Commentary: Bridging the Implementation Gap
The success of this domain-shift technology represents a significant step forward in ophthalmic AI. Most current AI solutions are localized or proprietary, functioning well within a specific clinic using a specific machine but failing in the real-world where hardware is diverse. By proving that a model can be trained on one vendor and successfully deployed on another using unsupervised adaptation, the researchers have provided a blueprint for more scalable AI implementation.
However, several considerations remain. While the NPV is excellent, the PPV (ranging from 46.0% to 72.0%) suggests that some false positives will still occur, requiring clinician oversight to prevent unnecessary referrals. Furthermore, as a retrospective study, the true impact on clinical workflow and patient outcomes must be validated in prospective, real-world trials. The varying sensitivity of the uncertain category also indicates that while the model is good at not being wrong, it may not yet be perfect at identifying every rare condition.
Mechanistically, the use of Test Entropy is a sophisticated approach to the domain-shift problem. It shifts the burden of adaptation from the training phase to the inference phase, making the model more dynamic and responsive to the specific characteristics of the image it is currently analyzing. This reflects a broader trend in AI research toward more flexible, context-aware systems.
Conclusion: A Path Toward Universal Diagnostics
The study by Tang et al. highlights the transformative potential of vendor-agnostic DL models in modern ophthalmology. By overcoming the limitations of hardware-specific training, this technology paves the way for broad deployment in diverse eye care settings, from primary care screenings to tertiary hospital triaging. Such systems can significantly streamline the detection of macular diseases, ensuring that patients with urgent conditions are prioritized and that the overall burden on eye care professionals is reduced.
Future research should focus on integrating this technology into electronic health records and exploring its application in longitudinal monitoring of disease progression. As AI continues to mature, the focus will likely shift from purely diagnostic accuracy to the seamless, cross-platform integration demonstrated in this multicenter cohort study.
References
1. Tang ZQ, Zhang YH, Ran AR, et al. Domain-Shift AI Technology for Vendor-Agnostic Multiple Macular Disease Detection From 3D OCT Scans. JAMA Ophthalmol. 2026 Feb 26. doi: 10.1001/jamaophthalmol.2026.0029.
2. Schmidt-Erfurth U, Waldstein SM. A technology update on optical coherence tomography. Invest Ophthalmol Vis Sci. 2014;55(12):8459-8476.
3. Ting DSW, Peng L, Varadarajan AV, et al. Deep learning in ophthalmology: The path to the clinic. Nat Med. 2019;25(2):251-256.