Keeping AI off the verdict path

Two opinions are not a gate

If a model wrote the change, checking it with another model gives you a second opinion. Opinions are useful; codument runs an adversarial review pass itself. But an opinion is not a verdict. It shifts with the model version, the prompt, the sampling temperature, and the day. A gate that says "stale" on Tuesday and "fresh" on Wednesday for the same bytes is not measuring the repo. It is measuring the model.

So codument is built on one rule: no model call anywhere on the verdict path. Anything that decides pass or fail is a pure function of repo state. Models are welcome one layer up, where judgment lives. They never get to hold the stamp.

What a deterministic verdict looks like

Here is the gate catching a one-character change. In a small auth fixture, isSessionExpired was changed from > to >=: a real contract change, because sessions now expire at the exact boundary millisecond instead of one past it. No test broke. A model reviewer might shrug. The gate does not get the option:

  1 changed file(s): 1 source, 0 docs

  Symbol drift — resolve each: update the doc, or ack a contract-neutral move
    • isSessionExpired (changed) in auth
        contract changed → update docs/features/auth.md at intent altitude
        internal only   → codument ack src/auth/session.ts::isSessionExpired(). --reason "..."

  Dependents that may need re-review
    • wallet (depends on auth)

Nothing in that output was generated. Each exported symbol's token stream is fingerprinted across two git refs, with comments and whitespace skipped and declaration-local names rewritten to positions before hashing, so a reformat or a local rename moves nothing. The dependents line comes from the registry's declared edges. We ran this command three times in a row and compared the outputs byte for byte: identical, every run. That is the whole point. Your laptop, your colleague's laptop, and CI cannot disagree about what changed.

Determinism also means the verdict can be data instead of prose:

{
  "version": 2,
  "gate": "ok",
  "isGitRepo": true,
  "changedFileCount": 1,
  …

A deterministic JSON verdict can be diffed, cached, piped into CI annotations, and audited months later. A paragraph of model prose can do none of those things.

Where judgment lives instead

Determinism has a blind spot, and the design says so instead of hiding it: a fingerprint can prove a documented symbol moved, but it cannot know whether the move changed anything a doc promises. That call requires understanding, which means it belongs to the agent and the human, not the hash.

So the gate does not guess. It presents exactly two resolutions: update the doc, or acknowledge the move as contract-neutral. The acknowledgment is not a snooze button. It is a signed adjudication, bound to the exact fingerprint transition it judged:

Acknowledgments (1)
  56bf952bf8856618  src/auth/session.ts::rotateToken(). (3a992885→388fb915)
    Jakub Suplicki <42224168+jakubsuplicki@users.noreply.github.com>: moved intact, contract unchanged

The next time that symbol moves, the fingerprints no longer match and the ack expires on its own. There is no standing exemption to accumulate, no "ignore this file" list quietly rotting in a config. The deterministic layer verifies the ack's form and binding; it never pretends to verify its truth. That honesty about the division of labor is what keeps the two layers from corrupting each other.

What this rules out by design

Holding the no-model line means saying no to features that sound helpful:

• No LLM-scored "doc quality" on the gate path. A quality score that moves when a vendor ships a new model version would make the gate's history meaningless.
• No semantic-similarity staleness detection. Embedding distance between a doc and its source is an opinion with extra steps, and it cannot be re-derived bit-for-bit in two years.
• No auto-generated doc patches to make findings disappear. A model that silently rewrites prose to satisfy a checker is the mirror-edit anti-pattern with better manners.

Each rejection follows from the same test: could two honest machines, given the same repo, reach different conclusions? If yes, it stays off the verdict path.

The honest limit

Determinism buys you trust in what changed, never in what matters. The gate can prove that a documented symbol moved and that its doc did not, byte-for-byte, forever. It cannot tell load-bearing from trivial, and it does not try. The judgment stays with the agent and with you.

That split is the design: a floor you can rerun, and judgment on top of it. Every mechanism the gate ships lands on one side of that line or the other, on purpose.