How to catch AI-hallucinated citations in your bibliography

Disclosure up front: one of the tools compared below is mine (the BibTeX Validator). I have tried to write this so it is useful even if you never touch it. Manual checking against a DOI is free and remains the gold standard; the automated tools, mine included, exist to make that checking fast enough that you actually do it.

Why AI invents citations that look real

A large language model does not look up a database of papers when you ask it for a reference. It predicts a plausible sequence of words. A citation is a very predictable sequence: an author surname, a year in parentheses, a title in the house style of the field, a journal name, a volume, a page range, maybe a DOI. The model has seen millions of these, so it can assemble one that is grammatically and stylistically flawless. Whether the paper behind it exists is a separate question the model was never actually answering.

This is why hallucinated references are dangerous in a way that a typo is not. A misspelled author name looks wrong. A hallucinated citation looks exactly right. Since 2023, journal correction notices, retraction reports, and methods papers across medicine, law, and the social sciences have repeatedly documented fabricated references slipping into submitted and even published work. The common thread is not carelessness; it is that the fake looks indistinguishable from the real until someone checks the source.

What a hallucinated citation looks like

Hallucinations are not random gibberish. They cluster into a few recognisable patterns. Learning them makes manual spot-checks much faster.

• Real authors, invented title. The model pairs genuine, well-known researchers in a field with a paper title they never wrote. The names check out on Google Scholar, so a quick author search gives false comfort.
• Real paper, wrong metadata. A real study attributed to the wrong year, journal, volume, or page range. Everything is almost right, which is exactly what makes it slip through.
• Plausible but dead DOI. A DOI in the correct registrant format (for example a 10.1145/… that looks like ACM) that resolves to nothing, or resolves to a completely different article.
• Frankenstein reference. First author from one paper, title from another, journal from a third. Each fragment is real; the combination is not.
• The confident near-match. A paper that almost exists: the model has blended two similar titles into one that reads perfectly and matches neither.

The single most reliable tell is the source itself. If you cannot open the paper and see that its title, authors, year, and venue match your citation, treat the citation as unverified.

The methods for catching them, compared

There is no single right tool. The honest comparison is about trade-offs between reliability, speed, and how many references you have to check. Here is how the common approaches stack up.

Manual checking

Open https://doi.org/ followed by the DOI, or search the exact title in Google Scholar or the publisher's site. Confirm the title, author list, year, and venue all match one real paper. This is the only method that catches every pattern above, because you are looking at the source. Its weakness is human: at forty references, thoroughness decays, and the one you skim past is the one that was fake. Manual checking is the right tool for the final verification of a handful of suspicious entries, not for grinding through an entire list.

Reference managers

Zotero and Mendeley are excellent at organising a library and generating consistent output, and if you added each reference by importing it from a real database, it is real by construction. But they were not built to answer the question "does this entry, which I pasted in from a chatbot, correspond to a real paper?" A reference manager will happily store a hallucination in perfect format. Use them for organisation, not for detection.

Crossref and OpenAlex

The scholarly metadata registries are the ground truth. A DOI either resolves to a record in Crossref or it does not; a title either matches a real work in OpenAlex or Semantic Scholar or it does not. If you are comfortable with an API, you can query these directly. Most researchers are not, which is the gap automated validators fill.

Automated validators

An automated bibliography checker parses your .bib file and resolves each entry against databases such as Crossref, Semantic Scholar, or OpenAlex, then reports which references are missing DOIs, fail to resolve, or have metadata that does not match the record it found. Several exist, and a genuinely useful comparison acknowledges that: some run as web tools, some as command-line utilities, some as browser extensions that check against Google Scholar. They differ in which databases they query, whether they check the full metadata or only the DOI, and how they present near-matches.

My own BibTeX Validator sits in this category: paste or upload a .bib file and it checks each entry against scholarly databases and flags the ones that look wrong, in one pass, with nothing stored. The honest limitation, true of every tool in this class, is that an automated pass is a filter, not a verdict. It narrows dozens of references down to the few worth a human look. It cannot certify that a reference is genuine; only that it found, or failed to find, a matching record. Treat any validator, mine included, as the first pass, and confirm the flagged entries by opening the source yourself.

A workflow that actually gets done

The best method is the one you will still be doing at reference forty. This one is fast enough to survive a real deadline:

1. Never paste a citation straight from a chatbot into your manuscript. If an AI tool suggests a source, treat it as a lead, not a reference. Find the real paper, then cite that.
2. Run the whole .bib file through an automated validator to flag missing DOIs, non-resolving entries, and metadata mismatches in one pass.
3. Manually verify every flagged entry by opening its DOI or searching the exact title. Confirm title, authors, year, and venue against one real source.
4. Spot-check a random sample of the entries that passed. Validators miss the Frankenstein and near-match patterns most, so open five or six that looked fine.
5. Fix or remove anything you cannot verify. A missing citation is a weakness; a fabricated one is an integrity problem. If you cannot find the source, do not cite it.

Tools that pair well

These free, in-browser utilities support the workflow above. Nothing is stored; files are processed in memory.

• BibTeX Validator — the automated first pass: flags missing DOIs, non-resolving entries, and metadata mismatches across a whole .bib file.
• Citation Generator — build a clean, correct reference from a DOI or identifier so you are not hand-typing metadata.
• Reference Converter — move references between BibTeX, RIS, and EndNote without reformatting by hand.

For the wider picture, see the BibTeX and citations hub, the guide to fixing common BibTeX errors, and turning a DOI into a BibTeX entry.

The short version

AI hallucinates citations because it predicts plausible text rather than retrieving real records, so the fakes look flawless. The only certain check is the source: does a real paper exist whose title, authors, year, and venue match your citation? Automated validators make that check fast enough to run on every reference; manual verification of the flagged ones makes it certain. Do both, and do not cite what you cannot open.