Prevalence and Characteristics of Null Studies in Ophthalmology: A Comprehensive Analysis from 2014 to 2025

Highlight

– Approximately 20.5% of ophthalmology studies published between 2014 and 2025 reported null primary outcomes.
– The proportion of null studies decreased significantly from 25.6% in 2014 to 15.1% in 2025.
– Positive studies were more frequently funded, prospective, and related to drugs, devices, or surgery compared to null studies.
– Null studies tended to be published in journals with lower impact factors and rankings, underscoring potential dissemination biases.

Study Background

Evidence-based ophthalmology relies heavily on published research to guide clinical decisions and health policy. Null studies—those reporting no statistically significant primary outcomes—play a critical role in providing unbiased knowledge, counteracting publication bias, and informing meta-analyses. However, selective publication favoring positive results can distort the scientific literature and clinical practice. Understanding the prevalence of null findings and their dissemination characteristics in ophthalmology literature is essential to assess potential biases and gaps.

Study Design

This retrospective analysis evaluated original research articles published from January 1, 2014, through December 31, 2025, across 90 PubMed-indexed journals classified under the ophthalmology category per the 2025 Journal Citation Reports. The study applied automated rule-based algorithms to extract study characteristics, complemented by a large language model (gpt-4.1)-based pipeline to classify main study outcomes as null or positive based on abstracts.

The classification model underwent performance validation on a sample of 500 manually labeled abstracts, achieving a Cohen’s kappa of 0.87, surpassing the prespecified threshold of 0.75, indicating high reliability.

Key Findings

Out of 93,963 eligible studies, 19,231 (20.5%) reported null primary outcomes. Notably, the proportion of null studies declined significantly over time, dropping from 25.6% in 2014 to 15.1% in 2025 (p < 0.001). This trend suggests evolving publication patterns or research focus.

Comparative analysis between null and positive studies highlighted several distinctions:

• Funding: Positive studies showed a higher likelihood of receiving financial support—68.4% versus 58.0% for null studies (risk difference 9.6 to 11.2, p < 0.001), indicating potential influences of funding on study outcomes or publication.
• Study Design: Positive studies were more often prospective (17.5% vs. 16.0%, p < 0.001), however, randomized controlled trials were equally represented in both groups (6.5%).
• Research Focus: Positive studies more frequently addressed therapeutic interventions including drug-related (13.6% vs. 11.8%), device-related (9.3% vs. 6.9%), and surgical topics (13.4% vs. 9.3%) (all p < 0.001).
• Publication Metrics: Null studies were more commonly published in journals with lower average impact factors (2.59 ± 1.62) compared to positive studies (2.73 ± 1.69; mean difference 0.11 to 0.17, p < 0.001), and had worse average journal rankings (40.11 vs. 37.80, p < 0.001), suggesting dissemination biases against null findings.

Expert Commentary

This comprehensive analysis underscores the ongoing challenge of publication bias within ophthalmology research. The decreasing proportion of null studies over time may reflect pressures to publish positive or novel results, which could skew clinical guidelines if null findings remain underreported. The similar rates of randomized trial design among null and positive studies indicate that methodological rigor alone does not guarantee publication likelihood.

Funding disparities raise concern about the possible influence of sponsorship on research outcomes or reporting. The predominance of positive studies with therapeutic focus aligns with clinical relevance but might overshadow equally important negative evidence. These findings align with broader biomedical literature documenting selective reporting and emphasize the need for policies promoting transparency and balanced dissemination.

Limitations: The reliance on abstract-based classification, despite high validation accuracy, may misclassify some studies. Also, impact factors and journal ranks are imperfect proxies for dissemination quality. Further, this work does not assess unpublished data or gray literature.

Conclusion

Null studies constitute a significant yet declining portion of published ophthalmology research. Their reduced prevalence and tendency to appear in lower-impact journals potentially limit comprehensive evidence synthesis. Increased emphasis on the publication and appraisal of null results is vital to reduce bias, improve clinical guidelines, and foster scientific integrity in ophthalmology.

Funding and ClinicalTrials.gov

The study did not detail specific funding sources. The inclusion of trial registration data or funding specifics would further enhance transparency in future meta-research.

References

Hecht I, Sella R, Kanclerz P, Sorin V, Tuuminen R. Null studies in ophthalmology journals: a large-scale analysis of PubMed indexed research. American journal of ophthalmology. 2026 Sep 1;PMID: 42680128.

Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, Hing C, Kwok CS, Pang C, Harvey I. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010 Feb;14(8):iii, ix-xi, 1-193.

Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124.