Same Gene List, Different GO Enrichment: How Much the GO Release Changes the Result (2016 to 2025, Measured)

Everything below comes from gene lists with known answers and a public RNA-seq data set, run against archived GO releases. The scripts, results and figure are public, and a second machine reproduced the result tables byte for byte. (한국어판)

TL;DR

• We kept the gene lists, the identifiers, each list's gene universe and the test fixed, and changed only the Gene Ontology release: January 2016, January 2020, January 2023 and October 2025.
• Planted answers survived. Lists with 15 cholesterol-biosynthesis proteins hidden among 300 found "cholesterol biosynthetic process" in every release.
• Random lists stayed clean. 20 lists of 300 random proteins gave no significant terms, except 2 terms in one list in one release (1 of 80 list-release pairs), which is what the FDR cutoff allows.
• A real list drifted a lot. Genes induced by dexamethasone in the airway data set gave 65 significant terms with the 2016 release and 84 with 2025, and only 26 are shared (Jaccard 0.21).
• The expected biology depended on the release. Dexamethasone is a glucocorticoid, but "response to glucocorticoid" was not significant with the 2016 release (FDR 0.21). It was from 2020 on (FDR 0.005-0.027).
• Practical rule: report the GO release date, not just the tool version, and rerun old enrichments before comparing them with new ones.

Why this question

Every enrichment tool reads a snapshot of the Gene Ontology: the ontology itself (terms and how they nest) and the annotations (which gene carries which term). Curators add and remove both all the time, and when a tool's snapshot was taken depends on the tool and its version. Wadi et al. (2016) showed that many popular tools were running on annotations years out of date, and that this changed enrichment results.

What we wanted to know is how big the change is for a list that does not change at all, and whether it matters for the parts of a result people actually read: does the planted answer survive, do random lists stay clean, and does the expected biology of a real experiment show up?

Setup

Item	Details
GO releases	go-basic.obo and the human GAF (goa_human.gaf.gz) of 2016-01-01, 2020-01-01, 2023-01-01 and 2025-10-10, from the GO release archive
Random lists	20 lists of 300 proteins drawn from the 5,661 human proteins DIA-NN detected in a public benchmark run. Expected: no terms
Planted lists	5 lists: 15 of the 16 detected Reactome cholesterol-biosynthesis proteins plus 285 other detected proteins. Expected: "cholesterol biosynthetic process" (GO:0006695)
Real lists	Bioconductor airway (Himes et al. 2014): four airway smooth-muscle cell lines, dexamethasone vs untreated, DESeq2 with the paired design ~ cell + condition. 511 genes up and 489 down (padj < 0.05, |log2FC| ≥ 1) among 16,596 tested
Test	One-sided hypergeometric test (over-representation analysis), terms with 10-500 annotated background genes, all three GO aspects, Benjamini-Hochberg, significant at FDR < 0.05
Background	The list's universe (the detected proteome, or the 13,195 tested genes with a UniProt accession), restricted to members annotated in that release
Identifiers	Fixed: UniProt accessions, genes mapped through one HGNC table
Evidence	Two settings: all evidence codes, and all but IEA (electronic annotations)

The lists were fixed before any release was read. Annotations are propagated up the ontology over is_a and part_of, as most tools do. To separate the two kinds of change, we also ran a hybrid: the 2016 annotations read through the 2025 ontology.

Results

All evidence codes:

Release	False terms, 20 random lists	Planted lists that found the term	Airway up: terms	Airway down: terms	"Response to glucocorticoid", FDR (rank)
2016-01-01	0	5/5	65	311	0.207 (285)
2020-01-01	2 (1 list)	5/5	80	321	0.005 (16)
2023-01-01	0	5/5	86	341	0.027 (66)
2025-10-10	0	5/5	84	262	0.025 (49)

Overlap of the significant terms with the 2025-10-10 release (Jaccard: shared terms divided by all terms in either set; planted is the mean of 5 lists):

Release	Planted	Airway up	Airway down	Airway up, no IEA	Airway down, no IEA
2016-01-01	0.55	0.21	0.36	0.06	0.28
2020-01-01	0.44	0.41	0.46	0.29	0.42
2023-01-01	0.59	0.53	0.55	0.44	0.59

What the numbers say

1. The planted answer survived every release. All 5 planted lists found "cholesterol biosynthetic process" in all four releases and under both evidence settings, ranked 1st to 5th. The terms around it changed, though: the planted lists' significant terms have a mean Jaccard overlap of 0.55 between 2016 and 2025.

2. Random lists stayed clean. Across 20 random lists and four releases, one list in the 2020 release gave 2 DNA-replication terms (FDR 0.024-0.027). The other 79 list-release pairs gave none. Changing the release did not push random lists above the false-positive level the FDR cutoff allows.

3. A real list drifted a lot. The dexamethasone-up list has 65 significant terms with the 2016 release and 84 with 2025; 26 are in both. Put the other way, 26 of the 65 terms of 2016 (40%) are still significant in 2025. The down list does a little better (Jaccard 0.36). These overlaps are for one cutoff (FDR < 0.05), so part of the change is terms crossing it, and the number of tested terms changes between releases, which changes the multiple-testing correction too.

4. The expected biology appeared only in newer releases. "Response to glucocorticoid" (GO:0051384) is the textbook answer for dexamethasone-induced genes. With the 2016 release it ranks 285th at FDR 0.21. With 2020 it is 16th (FDR 0.005), and it stays significant in 2023 and 2025. In the hybrid run (2016 annotations, 2025 ontology) it is still at FDR 0.19, so what missed it was the 2016 annotations, not the 2016 ontology.

5. In this list, the expected term depended on electronic annotations. Without IEA annotations, "response to glucocorticoid" was never significant (FDR 0.86-1.0). In the 2025 release, 99 of the 139 human proteins annotated to it, or to a term below it, have it only through IEA. Most of those rows cite GO_REF:0000107, which transfers experimental annotations from orthologs in other species. Dropping IEA is often presented as the stricter choice; for this list it removed the expected term. Without IEA the term is also small (11-20 annotated background genes, 1-2 in the list), so read that result with care.

6. Annotations moved more than the ontology, at least for what reaches the results. Reading the 2016 annotations through the 2025 ontology kept a Jaccard overlap of 0.56-0.83 with the 2016 significant terms; changing the annotations at a fixed ontology kept 0.25-0.58. Per protein the two changes are about equal (mean Jaccard 0.63 and 0.61), and of the 5,569 detected proteins annotated in both 2016 and 2025, only 16 have exactly the same propagated term set.

7. Even the planted genes moved. Of the 16 detected Reactome cholesterol-biosynthesis proteins, 13 to 16 carry the GO term depending on the release (all evidence codes; 11 to 16 without IEA). ACAT2 has it only in 2020, GGPS1 loses it after 2020, and SQLE has only the broader "sterol biosynthetic process" in 2023 (panel d). The planted term was still found because enough of the other planted proteins in each list kept it.

What to do

• Report the GO release date, not just the tool version. The same test on the same list gave a different answer with each release. In Bioconductor, for example, GO_dbInfo() and org.Hs.eg_dbInfo() record the ontology date as GOSOURCEDATE, and org.Hs.eg.db takes its gene annotations from NCBI's gene2go file, dated by GOEGSOURCEDATE. On the R 4.3 install we used for this site they say 2023-07-27 and 2023-Sep11, more than two years before the newest release here. Web tools such as g:Profiler show their data version with the results.
• Rerun before you compare. If you compare your enrichment with a published one, or with your own analysis from two years ago, run both lists with the same release first. Otherwise part of the difference is the release.
• Do not drop IEA by reflex. For this list it removed the expected term. If you drop it, say so and expect fewer, smaller terms.
• Keep identifiers fixed. Map genes once, by stable IDs (Ensembl, UniProt) rather than symbols, so symbol renames do not add to the drift.
• Treat single terms near the cutoff as unstable. A term at FDR 0.03 in one release can be at 0.2 in another. Terms that hold across releases, like the planted cholesterol term here, are the ones to build a story on.

Limits

• One real experiment (dexamethasone in airway), at one cutoff, with over-representation analysis only. GSEA, other organisms and other gene-set databases (KEGG, Reactome, MSigDB) were not tested.
• Our own test, not specific tools. We read the GO Consortium's human GAF directly. Tools that take annotations from elsewhere (for example org.Hs.eg.db, built from NCBI gene2go) may move differently between releases; we did not run them.
• Four releases. The 2026 releases were listed in the archive but not readable when we ran this, so October 2025 is the newest.
• The hybrid run splits the change in one order only (ontology first), and terms that became obsolete were dropped rather than followed to their replacements, which counts some curator moves as annotation changes.
• Genes were mapped with a 2025 HGNC table, so a UniProt accession merged after 2016 can lose some of its 2016 annotations.

Reproduce it

Data, scripts, results and the figure: sbmlab-benchmarks/go-release-drift. bash download.sh fetches the four GO releases, the HGNC table and the airway data (about 180 MB, checked against md5sums.txt), and python3 enrich.py runs every list against every release in under a minute with the Python standard library. The tests in test/test_go_ora.py check the OBO and GAF readers, propagation, the hypergeometric tail against exact sums, and Benjamini-Hochberg.

References

1. Wadi L, Meyer M, Weiser J, Stein LD, Reimand J. Impact of outdated gene annotations on pathway enrichment analysis. Nat Methods 2016;13(9):705-706.
2. Himes BE, et al. RNA-Seq transcriptome profiling identifies CRISPLD2 as a glucocorticoid responsive gene that modulates cytokine function in airway smooth muscle cells. PLoS One 2014;9(6):e99625. (airway data)
3. Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol 2014;15:550.
4. The Gene Ontology Consortium. The Gene Ontology knowledgebase in 2023. Genetics 2023;224(1):iyad031.

Related

• GO and pathway enrichment analysis: a practical guide
• STRING enrichment and the background trap: 300 random proteins, about 100 "significant" GO terms
• DESeq2 vs edgeR vs limma on the airway data