# Verify AI-Generated Docs Against the Code Before You Merge Them

An assistant can draft a README, an API page, or a migration note that reads like your project and still describes a flag you do not have, a default you changed, or an error you never throw.

The draft is useful. The merge is the dangerous step. Treat generated documentation like a pull request from a fast new teammate who has not run the code: you verify it against the repo, then you merge it.

This is the developer companion in a short cluster on documentation drift. [A Weekly Drift Check for Product Docs](https://lavv.us/blog/toolbitai/weekly-drift-check-for-product-docs) covers a weekly check of product docs against the live product. A Telegraph note covers writing a one-page brief before you ask for a README draft.

## Generate from the right inputs

Give the assistant the files that actually define the behaviour: the module, the OpenAPI spec, the flags, the tests. A marketing page or an old README will teach it the stale version.

Ask for a draft of a specific section, not “document the whole service.” The smaller the scope, the cheaper the check.

## A verification pass that fits in a review

Work through the draft with the repo open.

- **Names.** Every command, flag, environment variable, route and field must appear in the code or the spec, with the same spelling.
- **Defaults and limits.** If the draft says a timeout is 30 seconds, find the constant. If it says a plan allows 10,000 rows, find the check.
- **Examples.** Run them, or at least compile them. Invented snippets are common and very costly.
- **Errors and edge cases.** If the page lists error codes, grep for them. Missing ones are a problem; extra ones are also a problem.
- **Scope.** Delete advice that belongs in another repo, or that describes a roadmap as if it had shipped.

If you cannot point to a file for a sentence, cut the sentence or mark it as unknown. Do not leave a confident guess in the main branch.

## What tools can and cannot do

Some documentation tools are built to notice when a page and a repo have drifted, or to open a draft from a diff. That can shrink the search. It does not replace the pass above. A useful starting point, if you want a shortlist, is Toolbit’s [comparison of AI documentation sync tools](https://www.toolbit.ai/blog/ai-documentation-tools-that-keep-technical-do), which is organised around a drift test. Run any candidate on a page you already know is wrong, and see whether it flags the same mismatch you found.

The review still needs a named author, the same way a code PR does. If nobody can explain the page without re-reading the model’s prose, it is not ready.

## Checklist

- The draft was generated from current code or a current spec, not only from an old README.
- Every command, example and default has been checked against the repo.
- Invented or out-of-scope lines have been removed.
- A person who owns the code is listed on the change.

## Takeaway

Merge AI-generated docs only after you have checked them against the code. Fluency is not evidence that a flag, an example, or an error code is real.