Why check a reference list
A retracted reference
does not announce itself.
Retraction is a normal correction mechanism. The difficulty is that the correction rarely reaches the people citing the article. Five findings from the meta-research literature, each with its study design and population.
Five findings
Every figure below states the study design and the sample it came from. Denominators and caveats are listed at the end of the page.
- 30.6%
Retractions are concentrated in the health sciences.
Health-sciences publications account for 16,210 of 52,990 records in the Retraction Watch database.1 In nursing, 408 articles were retracted between 2000 and 2024, 84.8% of them since 2020; 42.6% reported research involving human participants, totalling 21,369 patients. Across 16,041 medical retractions from 1975 to 2024, oncology (19.9%), cardiovascular medicine (15.6%) and pharmacology (14.5%) contributed most.2
Bibliometric analyses of the Retraction Watch database · n = 52,990 and 61,645 records
- 8.4%
Retracted trials alter the conclusions of evidence syntheses.
Among 163 meta-analyses containing at least one retracted study, pooled effect estimates changed by 10% or more in 35%, and by half or more in 14%.3 In a larger cohort of 1,330 retracted randomised trials across 3,902 replicable meta-analyses, removing them reversed the direction of the pooled effect in 8.4% and changed statistical significance in 16.0%. The affected syntheses had informed 157 clinical practice guideline documents.4
Systematic review with meta-analysis · retrospective cohort · n = 173 and 3,902 meta-analyses
- 5.4%
Post-retraction citations rarely disclose the retraction.
Of 13,252 citation contexts occurring after retraction in biomedicine, 722 stated that the article had been retracted.5 Two specialty replications report 4.2% and 1.8%. In one specialty cohort, articles citing a retracted paper were themselves more likely to be retracted (OR 11.09, 95% CI 7.06–17.43)6 — an association, not an established causal path.
Citation-context analysis · 7,813 retracted papers indexed in PubMed, 1960–2020
- 7%
Retraction status is indexed inconsistently across databases.
When 150 retracted articles were assessed against five documentation criteria in six databases, PubMed and Ovid MEDLINE satisfied approximately 87%, ProQuest PsycINFO 23%, and EBSCO CINAHL — a core nursing database — 7%.7 Only 3% of retracted records are indexed consistently across four major databases.8 Manual verification therefore depends on which database the reader happens to hold.
Cross-sectional database audit · 150 retracted articles across six platforms
- 30mo
Screening has a shelf life.
The median interval from publication to retraction is approximately thirty months,9 and the retraction rate has roughly trebled over a decade to about 0.2% of published papers.10 A reference verified at protocol stage may therefore carry a notice before the manuscript reaches peer review.
Time-to-event bibliometric analyses · n = 2,047 articles; rate per published paper
Points of use
Three moments where the check belongs
Failure modes, and what the report does about them
Each row pairs a way a reference check can fail with the design decision taken in response. These are failure modes, not claims about any particular tool; the benchmark comparison is published separately.
The reference carries no DOI
Vancouver style, standard in medicine and nursing, omits identifiers. A check that requires one either skips the reference or reports it as clean.
Resolution from bibliographic fields
Title, journal, year and first author are matched, and the report states which fields agreed. Where the evidence is too thin, the reference is reported as unresolved rather than as carrying no notice.
Only the most recent notice is reported
A concern later escalated to retraction and a single correction that still stands collapse into one label.
All notices, in chronological order
Each notice appears with its date and the reasons given. One reference in our regression set carries three: an expression of concern, a correction, then a retraction.
A restored article is reported as retracted
Reversal is uncommon, so it is frequently absent from the state vocabulary — producing a false positive against a sound article.
A separate state, and the only positive finding
156 articles in the database were retracted and subsequently restored. Reinstatement is never rendered in the retraction colour, in any output.
One score for two different questions
A composite confidence value mixes which article was identified with what the record holds about it, and neither can be recovered from it.
Match and status reported separately
A status is only as reliable as the identification beneath it. Both are stated for every entry, and no composite score is produced.
A source fails and the result still looks complete
A fixed count of databases is printed whether or not each responded, so an incomplete check is indistinguishable from a negative one.
The responding sources are named
Every report states which databases answered, which did not, and the date of the data. An unavailable source is never converted into no notice found.
The result cannot be cited in a methods section
Without a date, a source list and a run identifier, a screening step cannot be described in enough detail to be repeated.
A dated, attributable record
Each report carries a run identifier, the database version date, the sources consulted, and a methods sentence written for direct inclusion in a manuscript.
References
- Contreras-Muñoz MP, Zahn-Muñoz C, Solís-Albanese E, Martínez-Rojas E. Analysis of retractions in nursing from publications between 2000 and 2024: a bibliometric analysis using Retraction Watch. Nurs Rep. 2025;15(10):349. doi:10.3390/nursrep15100349
- Fifty years of retracted medical publications from 1975 to 2024: a comprehensive analysis of trends, reasons, and countries using the Retraction Watch database. J Korean Med Sci. 2025;40(46):e300. doi:10.3346/jkms.2025.40.e300
- Graña Possamai C, Cabanac G, Perrodeau E, Ghosn L, Ravaud P, Boutron I. Inclusion of retracted studies in systematic reviews and meta-analyses of interventions. JAMA Intern Med. 2025;185(6):702–9. doi:10.1001/jamainternmed.2025.0256
- Xu C, Fan S, Tian Y, et al. Investigating the impact of trial retractions on the healthcare evidence ecosystem (VITALITY Study I): retrospective cohort study. BMJ. 2025;389:e082068. doi:10.1136/bmj-2024-082068
- Hsiao T-K, Schneider J. Continued use of retracted papers: temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine. Quant Sci Stud. 2021;2(4):1144–69. doi:10.1162/qss_a_00155
- Song F, Wu B, Wei G, et al. A systematic analysis of temporal trends, characteristics, and citations of retracted stem cell publications. BMC Med. 2025;23:131. doi:10.1186/s12916-025-03965-8
- Suelzer EM, Deal J, Hanus K, Ruggeri BE, Witkowski E. Challenges in identifying the retracted status of an article. JAMA Netw Open. 2021;4(6):e2115648. doi:10.1001/jamanetworkopen.2021.15648
- Ortega JL, Delgado-Quirós L. The indexation of retracted literature in seven principal scholarly databases. Scientometrics. 2024;129(7):3769–85. doi:10.1007/s11192-024-05034-y
- Steen RG, Casadevall A, Fang FC. Why has the number of scientific retractions increased? PLoS One. 2013;8(7):e68397. doi:10.1371/journal.pone.0068397
- Van Noorden R. More than 10,000 research papers were retracted in 2023 — a new record. Nature. 2023;624(7992):479–81. doi:10.1038/d41586-023-03974-8
Notes on populations, denominators and known caveats
Health-sciences share and nursing figures — Retraction Watch downloaded 15 October 2024. Nursing represents approximately 2.5% of health-science retractions; ethical misconduct was the stated reason in 87.3%. The fifty-year medical series draws 16,041 medical retractions from a 61,645-record download of 1 March 2025.
Evidence-synthesis impact — The 35% figure comes from 50 systematic reviews containing 173 meta-analyses published in 25 high-impact general and internal medicine journals; 74% of those reviews appeared before the included study was retracted. The VITALITY cohort covers 1,330 retracted randomised trials; removal also shifted effect magnitude by more than half in 15.7% of replicable meta-analyses.
Citation disclosure — The unit of analysis is the citation context: a sentence in a later article containing a citation, so one citing paper may contribute several. A citation that does not state the retraction establishes neither what its authors knew nor that the citation was inappropriate. The two replications are single-specialty and smaller (stem-cell research, otolaryngology).
Odds ratio of 11.09 — Derived within a cohort of 517 retracted stem-cell publications to July 2024. The association is confined to one specialty and does not establish that citing retracted work causes retraction; both may follow from shared underlying practices.
Database documentation criteria — The five criteria concern how clearly a platform marks retracted status, not whether the record is present. Results are specific to the platforms and interfaces tested at the time of the audit.
Time to retraction and rate — The foundational analysis reports a mean of 32.9 months across 2,047 articles, declining over time; recent specialty studies report medians near 30 months. The interval varies substantially by reason, with plagiarism detected considerably faster than data fabrication. The rate is quoted in preference to the annual count: the widely repeated figure of more than 10,000 retractions in 2023 was dominated by a single publisher’s mass retraction of approximately 8,000 articles, and 2024 was lower, so it characterises an event rather than a trend.
Screening a reference list takes about a second per reference.
Free, no sign-up, and nothing you paste is stored.