The sample is the 100 newest submissions listed in four arXiv categories: cs.CR (cryptography and security), cs.AI, cs.LG (machine learning) and cs.CY (computers and society). We fetched it on 2 November 2026. All 100 papers were submitted on 30 October 2026. One paper failed during analysis, which leaves 99. Each paper went through Citation Gap Analysis, the same code customers run. We then kept only the suggestions we could tie to a real published work, and dropped any work the paper already cites.
The count says the tool found something to check in most new papers. It does not say those papers are wrong to leave the work out. Whether a missing citation matters is for the authors and their reviewers to decide.
What counts as a gap
A suggestion counts only when all three of these hold:
- CrossRef returns a record for the title the model suggested.
- That record is the work the model named. At least 80% of the content words must be shared, or the titles must be identical once punctuation is removed. CrossRef's top hit for a title is sometimes a different paper with similar words, and this test removes those.
- The work does not appear anywhere in the paper's text. That means no matching DOI, no matching title, no matching main title (the part before a colon), and no reference entry or eight-line stretch of text holding 80% of the title's content words. If the model named several works in one suggestion and the paper cites any of them, the suggestion does not count.
The text check covers every page of the PDF, not just the reference list, so a work mentioned in the body also counts as cited. Each of these choices makes the count smaller.
From suggestions to counted gaps
| Step | Suggestions | Papers with at least one |
|---|---|---|
| Suggested by the model | 787 | 99 |
| With a title CrossRef could search | 764 | 99 |
| CrossRef record found (the tool's own 60% bar) | 308 | 95 (96%) |
| Record is the work the model named | 285 | 92 (93%) |
| Already cited in the paper (removed) | 37 | |
| Counted: exists and not cited | 248 | 86 (87%) |
Most suggestions never reach the count. 456 of the 787 got a CrossRef hit too weak to accept, and 23 had no title to search. Those may still be real works under a different title, but we cannot confirm them, so they are not counted. The 86 papers with a counted gap have a median of 3 and a maximum of 8.
How accurate the count is
We picked 18 counted gaps from this run at random (seed 4242) and checked each against the paper's text by hand. We searched for the work's title, distinctive phrases from it, and its authors' surnames in the reference list.
- 18 of 18 were correct gaps. In each case the named work exists and the paper does not cite it.
- 16 of 18 CrossRef records were the exact work named. In the other two, the named work is real but CrossRef's record is a neighbour. One named Adebayo et al., "Sanity Checks for Saliency Maps" (2018), and the record is a 2022 follow-up called "More Sanity Checks for Saliency Maps". The other named the IEEE 1588 clock synchronization standard, and the record is a NIST workshop report about it. For these two, the DOI in the CSV points to the neighbour.
Before this run, a separate check of 15 counted gaps from an earlier run on an overlapping sample found all 15 correct. We used that earlier check to set the rules above, so it is not an independent test.
Limitations
- The suggestions come from a language model's recall. The count covers works the model thought of, so it is not a full list of what each paper could cite.
- Relevance is not judged. A counted gap is a real work the model linked to the paper's topic, and whether the paper should cite it is a separate question.
- "Not cited" means no trace in the PDF text as pdfplumber extracts it. A work cited under a very different title, or only in a figure or a scanned page, would be missed. That error raises the count, and none of the 18 checked cases showed it.
- CrossRef mostly indexes DOI-registered works, so arXiv-only preprints, many standards and some conference papers are rarely matched. That error lowers the count.
- This is one run on one day: four categories and their cross-lists, 100 papers, all submitted within about four hours. Of the 99 analyzed, 67 have cs.LG, cs.AI, cs.CY or cs.CR as their primary category, and 32 are cross-listed from elsewhere. New submissions have not been through peer review, so these numbers may not hold for published papers.
- The benchmark retries rate-limited CrossRef requests. The live tool does not, so some of its suggestions show "could not verify" instead.
- A first version of this count from the same day reported 94%. It counted works the paper already cited and CrossRef neighbours. The rules above replace it.
Data
Download the CSV (100 rows, one per paper, 40 KB). License: CC BY 4.0.
| Column | Meaning |
|---|---|
| arxiv_id | arXiv identifier, for example 2610.42147 |
| title | Paper title as listed on arXiv |
| published | arXiv submission time, UTC |
| primary_category | First category arXiv lists for the paper |
| status | ok, or error if the paper could not be analyzed |
| n_gaps_flagged | Suggestions the model made |
| n_confirmed_exists | Suggestions with a CrossRef record at the tool's own bar |
| n_counted_gaps | Suggestions that pass all three rules above |
| counted_gap_titles | CrossRef titles of the counted gaps |
| counted_gap_dois | CrossRef DOIs of the counted gaps, in the same order |
The code that produced it is public: backend/benchmark in the site's repository. A run costs about $17 in model calls.
Suggested citation
Ampel, B. (2026). Citation gaps in new arXiv papers [Data set]. Purplelink LLC. https://purplelink.llc/research/citation-gaps-in-new-arxiv-papers/
To run the same check on your own manuscript before you submit, use Citation Gap Analysis. The post on why citation gaps happen explains what the tool is for.