[{"data":1,"prerenderedAt":434},["ShallowReactive",2],{"blog-list":3},[4,152,266],{"id":5,"title":6,"body":7,"date":137,"description":138,"draft":139,"extension":140,"log":141,"meta":142,"minutes":143,"navigation":144,"path":145,"seo":146,"stem":147,"tags":148,"__hash__":151},"blog\u002Fblog\u002Fkeeping-ai-off-the-verdict-path.md","Keeping AI off the verdict path",{"type":8,"value":9,"toc":129},"minimark",[10,15,19,22,26,42,56,59,62,71,74,78,81,84,93,96,100,103,116,119,123,126],[11,12,14],"h2",{"id":13},"two-opinions-are-not-a-gate","Two opinions are not a gate",[16,17,18],"p",{},"If a model wrote the change, checking it with another model gives you a second opinion. Opinions are useful; codument runs an adversarial review pass itself. But an opinion is not a verdict. It shifts with the model version, the prompt, the sampling temperature, and the day. A gate that says \"stale\" on Tuesday and \"fresh\" on Wednesday for the same bytes is not measuring the repo. It is measuring the model.",[16,20,21],{},"So codument is built on one rule: no model call anywhere on the verdict path. Anything that decides pass or fail is a pure function of repo state. Models are welcome one layer up, where judgment lives. They never get to hold the stamp.",[11,23,25],{"id":24},"what-a-deterministic-verdict-looks-like","What a deterministic verdict looks like",[16,27,28,29,33,34,37,38,41],{},"Here is the gate catching a one-character change. In a small auth fixture, ",[30,31,32],"code",{},"isSessionExpired"," was changed from ",[30,35,36],{},">"," to ",[30,39,40],{},">=",": a real contract change, because sessions now expire at the exact boundary millisecond instead of one past it. No test broke. A model reviewer might shrug. The gate does not get the option:",[43,44,46],"term-fig",{"cmd":45},"codument review --strict",[47,48,54],"pre",{"className":49,"code":51,"language":52,"meta":53},[50],"language-text","  1 changed file(s): 1 source, 0 docs\n\n  Symbol drift — resolve each: update the doc, or ack a contract-neutral move\n    • isSessionExpired (changed) in auth\n        contract changed → update docs\u002Ffeatures\u002Fauth.md at intent altitude\n        internal only   → codument ack src\u002Fauth\u002Fsession.ts::isSessionExpired(). --reason \"...\"\n\n  Dependents that may need re-review\n    • wallet (depends on auth)\n","text","",[30,55,51],{"__ignoreMap":53},[16,57,58],{},"Nothing in that output was generated. Each exported symbol's token stream is fingerprinted across two git refs, with comments and whitespace skipped and declaration-local names rewritten to positions before hashing, so a reformat or a local rename moves nothing. The dependents line comes from the registry's declared edges. We ran this command three times in a row and compared the outputs byte for byte: identical, every run. That is the whole point. Your laptop, your colleague's laptop, and CI cannot disagree about what changed.",[16,60,61],{},"Determinism also means the verdict can be data instead of prose:",[43,63,65],{"cmd":64},"codument review --json",[47,66,69],{"className":67,"code":68,"language":52,"meta":53},[50],"{\n  \"version\": 2,\n  \"gate\": \"ok\",\n  \"isGitRepo\": true,\n  \"changedFileCount\": 1,\n  …\n",[30,70,68],{"__ignoreMap":53},[16,72,73],{},"A deterministic JSON verdict can be diffed, cached, piped into CI annotations, and audited months later. A paragraph of model prose can do none of those things.",[11,75,77],{"id":76},"where-judgment-lives-instead","Where judgment lives instead",[16,79,80],{},"Determinism has a blind spot, and the design says so instead of hiding it: a fingerprint can prove a documented symbol moved, but it cannot know whether the move changed anything a doc promises. That call requires understanding, which means it belongs to the agent and the human, not the hash.",[16,82,83],{},"So the gate does not guess. It presents exactly two resolutions: update the doc, or acknowledge the move as contract-neutral. The acknowledgment is not a snooze button. It is a signed adjudication, bound to the exact fingerprint transition it judged:",[43,85,87],{"cmd":86},"codument ack --list",[47,88,91],{"className":89,"code":90,"language":52,"meta":53},[50],"Acknowledgments (1)\n  56bf952bf8856618  src\u002Fauth\u002Fsession.ts::rotateToken(). (3a992885→388fb915)\n    Jakub Suplicki \u003C42224168+jakubsuplicki@users.noreply.github.com>: moved intact, contract unchanged\n",[30,92,90],{"__ignoreMap":53},[16,94,95],{},"The next time that symbol moves, the fingerprints no longer match and the ack expires on its own. There is no standing exemption to accumulate, no \"ignore this file\" list quietly rotting in a config. The deterministic layer verifies the ack's form and binding; it never pretends to verify its truth. That honesty about the division of labor is what keeps the two layers from corrupting each other.",[11,97,99],{"id":98},"what-this-rules-out-by-design","What this rules out by design",[16,101,102],{},"Holding the no-model line means saying no to features that sound helpful:",[104,105,106,110,113],"ul",{},[107,108,109],"li",{},"No LLM-scored \"doc quality\" on the gate path. A quality score that moves when a vendor ships a new model version would make the gate's history meaningless.",[107,111,112],{},"No semantic-similarity staleness detection. Embedding distance between a doc and its source is an opinion with extra steps, and it cannot be re-derived bit-for-bit in two years.",[107,114,115],{},"No auto-generated doc patches to make findings disappear. A model that silently rewrites prose to satisfy a checker is the mirror-edit anti-pattern with better manners.",[16,117,118],{},"Each rejection follows from the same test: could two honest machines, given the same repo, reach different conclusions? If yes, it stays off the verdict path.",[11,120,122],{"id":121},"the-honest-limit","The honest limit",[16,124,125],{},"Determinism buys you trust in what changed, never in what matters. The gate can prove that a documented symbol moved and that its doc did not, byte-for-byte, forever. It cannot tell load-bearing from trivial, and it does not try. The judgment stays with the agent and with you.",[16,127,128],{},"That split is the design: a floor you can rerun, and judgment on top of it. Every mechanism the gate ships lands on one side of that line or the other, on purpose.",{"title":53,"searchDepth":130,"depth":130,"links":131},2,[132,133,134,135,136],{"id":13,"depth":130,"text":14},{"id":24,"depth":130,"text":25},{"id":76,"depth":130,"text":77},{"id":98,"depth":130,"text":99},{"id":121,"depth":130,"text":122},"2026-07-12","Why a safety layer for agent output has to be deterministic all the way down, and what that rules out by design.",false,"md","LOG 03",{},6,true,"\u002Fblog\u002Fkeeping-ai-off-the-verdict-path",{"title":6,"description":138},"blog\u002Fkeeping-ai-off-the-verdict-path",[149,150],"internals","design notes","e36ITFJS52617U4R8l2Abkb6_hHIV7ZGF7P6mKXKxe8",{"id":153,"title":154,"body":155,"date":137,"description":256,"draft":139,"extension":140,"log":257,"meta":258,"minutes":143,"navigation":144,"path":259,"seo":260,"stem":261,"tags":262,"__hash__":265},"blog\u002Fblog\u002Fthe-diff-is-not-the-deliverable.md","The diff is not the deliverable",{"type":8,"value":156,"toc":248},[157,161,164,167,171,174,177,181,184,187,190,199,203,206,214,217,221,224,227,235,238,242,245],[11,158,160],{"id":159},"code-got-cheap-verdicts-did-not","Code got cheap. Verdicts did not.",[16,162,163],{},"An agent can produce a plausible four-file diff in less time than it takes to read this page. That did not make software cheap; it moved the cost. The scarce artifact in an agent-speed workflow is no longer the change. It is a defensible statement about the change: what it touches, what it risks, and whether the objections against it survived being tested.",[16,165,166],{},"That statement is the deliverable. The diff is just its input.",[11,168,170],{"id":169},"self-review-is-the-author-grading-their-own-homework","Self-review is the author grading their own homework",[16,172,173],{},"The obvious fix is to ask the agent to review its own work, and it fails for a boring reason: completion bias. An author who re-reads their own reasoning re-anchors to it. The same mental model that produced the bug now inspects the bug, nods, and moves on. This is not a discipline problem you can prompt away. It is a property of context.",[16,175,176],{},"So independence has to come from context, not effort. codument's review adversary gets the diff and the documented contract it crosses, and never the author's chain of thought. On hosts with subagents that is a fresh context window; where there is none, the same agent runs an adversarial pass against the identical bundle. Weaker, and the design says so out loud, but the rest of the gate does not depend on it.",[11,178,180],{"id":179},"give-the-adversary-a-contract-not-a-vibe","Give the adversary a contract, not a vibe",[16,182,183],{},"\"Assume the change is wrong\" is a stance, not a method. Wrong against what? Generic review prompts flail and nitpick because they have no ground truth to check against.",[16,185,186],{},"codument hands the reviewer one: the diff, the documented invariants it crosses, the tests that pin those invariants, and the ownership and blast-radius facts the registry already knows. The bundle introduces no new opinions. Every line of it is derived from committed docs and the deterministic change analysis. The adversary's job narrows from \"find something to say\" to \"show which of these documented promises this diff breaks.\"",[16,188,189],{},"Until a recorded review covers the current diff, the gate says exactly that:",[43,191,193],{"cmd":192},"codument review --require-review",[47,194,197],{"className":195,"code":196,"language":52,"meta":53},[50],"  ✗ --require-review: no current adversarial review covers this diff.\n    Run a fresh adversarial review of this diff and record it under .codument\u002Freviews\u002F, then re-run.\n",[30,198,196],{"__ignoreMap":53},[11,200,202],{"id":201},"a-finding-blocks-only-if-it-can-lose","A finding blocks only if it can lose",[16,204,205],{},"Reviewer prose is never trusted, including the adversary's. A finding is a candidate until it is reduced to a named test, and it blocks only while that test is genuinely red on a re-run the gate performs itself. This session is real: the agent shipped a plausible-looking fix that capped each rotation but still extended the session on every refresh. The adversary's finding named a test, and the gate re-ran it:",[43,207,208],{"cmd":192},[47,209,212],{"className":210,"code":211,"language":52,"meta":53},[50],"  ✗ --require-review: 1 confirmed finding(s) unresolved.\n    • src\u002Fauth\u002Fsession.ts:rotateToken — the per-rotation cap still extends the session on every refresh; continuous rotation keeps a session alive forever (test: test\u002Fsession-ttl.test.ts)\n",[30,213,211],{"__ignoreMap":53},[16,215,216],{},"No red test, no block. A finding that cannot be reduced to a test, a design smell, a naming complaint, an untestable worry, stays recorded and advisory. It reaches the human; it never silently vetoes a merge. That single rule bounds the failure mode that kills review tools in practice, which is not missed bugs. It is alarm fatigue.",[11,218,220],{"id":219},"fixing-the-code-reopens-the-gate","Fixing the code reopens the gate",[16,222,223],{},"The recorded review binds to a fingerprint of the reviewed sources and the tests its findings name. Change any of them and the review is void. Fix the code: the fingerprint moves, and the adversary must look again. Quietly edit the failing test instead of the code: the fingerprint moves too, so a finding cannot be green-washed out of existence.",[16,225,226],{},"After the real fix landed and a fresh review was recorded against the fixed diff:",[43,228,229],{"cmd":192},[47,230,233],{"className":231,"code":232,"language":52,"meta":53},[50],"  ✓ Adversarial review covers this diff\n",[30,234,232],{"__ignoreMap":53},[16,236,237],{},"That line is the deliverable. Not \"an agent looked at it\", but: an independent pass attacked this exact diff against its documented contract, its one confirmed objection was reproduced, fixed, and re-verified, and the artifact saying so is bound to these bytes.",[11,239,241],{"id":240},"what-stays-with-you","What stays with you",[16,243,244],{},"The honest limit, because honest limits are the house style: the gate certifies process, not safety. It can prove a review covered this diff and that no confirmed objection is outstanding. It cannot prove the change is good, and it does not try. Judgment stays with you.",[16,246,247],{},"What changes is what you are judging. Not a wall of plausible code and a reviewer's mood, but a verdict that survived being re-run.",{"title":53,"searchDepth":130,"depth":130,"links":249},[250,251,252,253,254,255],{"id":159,"depth":130,"text":160},{"id":169,"depth":130,"text":170},{"id":179,"depth":130,"text":180},{"id":201,"depth":130,"text":202},{"id":219,"depth":130,"text":220},{"id":240,"depth":130,"text":241},"When an agent writes the code, review stops being a courtesy and starts being the product. What a reviewer actually owes the merge.","LOG 01",{},"\u002Fblog\u002Fthe-diff-is-not-the-deliverable",{"title":154,"description":256},"blog\u002Fthe-diff-is-not-the-deliverable",[263,264],"practice","review","NVXuUc_aXv_IVfcg6AKCh9MeTqQxmRQAY0qr7uojaFU",{"id":267,"title":268,"body":269,"date":423,"description":424,"draft":139,"extension":140,"log":425,"meta":426,"minutes":427,"navigation":144,"path":428,"seo":429,"stem":430,"tags":431,"__hash__":433},"blog\u002Fblog\u002Fyour-agent-changed-14-files-which-docs-went-stale.md","Your agent changed 14 files. Which docs went stale?",{"type":8,"value":270,"toc":414},[271,275,278,281,285,292,295,304,308,311,314,323,326,330,333,342,345,349,352,355,358,367,370,374,377,386,389,392,396,399,408,411],[11,272,274],{"id":273},"docs-are-load-bearing-now","Docs are load-bearing now",[16,276,277],{},"Coding agents read documentation before they write code. That changed what a stale doc costs. It used to mislead the next engineer, who would notice the smell and double-check the source. Now it steers the next agent session, which will confidently build on whatever the doc claims. A wrong sentence in a feature doc becomes wrong code within the hour.",[16,279,280],{},"In agent-speed development, documentation stopped being a courtesy. It is an input.",[11,282,284],{"id":283},"staleness-is-a-measurement-problem","Staleness is a measurement problem",[16,286,287,288,291],{},"Most teams treat doc drift like weather: everyone complains, nobody measures. But drift has a precise definition once you write down one fact per file: which doc owns it. codument keeps that fact in a registry, ",[30,289,290],{},"docs\u002F.registry.json",", mapping every source file to the feature doc responsible for it.",[16,293,294],{},"With ownership recorded, staleness becomes checkable. A source file changed and its owning doc did not: that is not a vibe, it is a finding.",[43,296,298],{"cmd":297},"npx codument review",[47,299,302],{"className":300,"code":301,"language":52,"meta":53},[50],"$ npx codument review\n\n  2 changed file(s): 1 source, 0 docs\n\n  Stale docs (source changed, mapped doc did not)\n    ⚠ auth: docs\u002Ffeatures\u002Fauth.md (changed: src\u002Fauth\u002Fsession.ts)\n",[30,303,301],{"__ignoreMap":53},[11,305,307],{"id":306},"only-the-owner-wakes","Only the owner wakes",[16,309,310],{},"File-level checks cause alarm fatigue: touch a big file and every doc that mentions it screams. codument resolves staleness per symbol instead. Each exported symbol's token stream is fingerprinted across two git refs, and names bound inside a declaration are rewritten to positions before hashing. A local rename that preserves meaning moves nothing.",[16,312,313],{},"When a documented symbol genuinely moves and its owning doc does not, exactly one feature wakes, and the finding names the symbol:",[43,315,317],{"cmd":316},"npx codument review --strict",[47,318,321],{"className":319,"code":320,"language":52,"meta":53},[50],"  Symbol drift — resolve each: update the doc, or ack a contract-neutral move\n    • rotateToken (changed) in auth\n        contract changed → update docs\u002Ffeatures\u002Fauth.md at intent altitude\n        internal only   → codument ack src\u002Fauth\u002Fsession.ts::rotateToken(). --reason \"...\"\n",[30,322,320],{"__ignoreMap":53},[16,324,325],{},"The verdict is a pure function of two git refs: same repo state, same output, no model call anywhere on the path.",[11,327,329],{"id":328},"when-no-doc-is-owed-acknowledge-dont-paper","When no doc is owed: acknowledge, don't paper",[16,331,332],{},"Some moves change no contract. The worst response is editing prose to appease a checker, because junk mirror edits rot docs as surely as staleness does. codument's answer is an acknowledgment:",[43,334,336],{"cmd":335},"codument ack src\u002Fauth\u002Fsession.ts::rotateToken(). --reason \"moved intact, contract unchanged\"",[47,337,340],{"className":338,"code":339,"language":52,"meta":53},[50],"$ codument ack src\u002Fauth\u002Fsession.ts::rotateToken(). --reason \"moved intact, contract unchanged\"\n\n✓ acknowledged src\u002Fauth\u002Fsession.ts::rotateToken(). (3a992885→388fb915, self)\n",[30,341,339],{"__ignoreMap":53},[16,343,344],{},"An ack is bound to the exact fingerprint transition it adjudicated. The next time that symbol moves, the ack expires on its own. There is no ride-forever exemption to accumulate.",[11,346,348],{"id":347},"the-gate-only-trusts-what-it-can-re-run","The gate only trusts what it can re-run",[16,350,351],{},"A drift check keeps docs honest; it says nothing about whether the change itself holds up. For that, codument runs two adversaries across the workflow, and neither one gets to be creative.",[16,353,354],{},"The plan adversary contests the plan before code exists. Its objections must be grounded: each one cites a committed doc, a test, or the stated scope. It never blocks; the human adjudicates every objection in chat.",[16,356,357],{},"The review adversary attacks a fingerprint-bound bundle of the diff plus the documented invariants it crosses. Its findings only block when a named test is genuinely red on a live re-run. No red test, no block. And because the recorded review binds to both the reviewed sources and the tests its findings name, fixing the code or touching a test invalidates the review, and the gate reopens:",[43,359,361],{"cmd":360},"npx codument review --require-review",[47,362,365],{"className":363,"code":364,"language":52,"meta":53},[50],"  ✗ --require-review: 1 confirmed finding(s) unresolved.\n    • src\u002Fauth\u002Fsession.ts:rotateToken — the per-rotation cap still extends the\n      session on every refresh; continuous rotation keeps a session alive forever\n      (test: test\u002Fsession-ttl.test.ts)\n",[30,366,364],{"__ignoreMap":53},[16,368,369],{},"A reviewer's prose is never trusted. A verdict is computed, not generated.",[11,371,373],{"id":372},"the-payoff-is-retrieval","The payoff is retrieval",[16,375,376],{},"Here is the part that makes the discipline pay rent every day, not just on the day something drifts. The same registry that catches staleness can be projected the other way: given a feature, hand back the smallest grounded working set for it. The owning doc, its invariants with their test pointers, the primary sources, the declared risk.",[43,378,380],{"cmd":379},"codument context --feature auth",[47,381,384],{"className":382,"code":383,"language":52,"meta":53},[50],"$ codument context --feature auth\n\n  auth — docs\u002Ffeatures\u002Fauth.md  ~88 tok\n    Invariants & boundaries\n      - An expired session must never authorize\n      - Sessions expire one hour after issue.\n    primary sources: src\u002Fauth\u002Fauthorize.ts, src\u002Fauth\u002Fsession.ts\n    risk: security\n",[30,385,383],{"__ignoreMap":53},[16,387,388],{},"An agent that starts from this pack reads a fraction of what a whole-repo crawl costs, and everything it reads has been kept true by the gate above. In codument's packaged benchmark the context pack measured 1,746 estimated tokens against 3,068 for whole-file context, an eight-file working set instead of sixteen. It is a fixture benchmark, not your repo, but the direction is the point: docs in, trusted context out.",[16,390,391],{},"That loop is the whole argument. A doc you can trust is retrieval infrastructure. A doc you cannot trust gets routed around, and then it is just weight.",[11,393,395],{"id":394},"try-it-on-the-history-you-already-have","Try it on the history you already have",[16,397,398],{},"You do not need to adopt anything to see your own number. Two read-only commands replay your committed history against a proposed ownership map and report every feature whose code moved while its doc got no attention:",[43,400,402],{"cmd":401},"npx codument scan && npx codument audit v1.0.0..HEAD",[47,403,406],{"className":404,"code":405,"language":52,"meta":53},[50],"$ npx codument scan\n$ npx codument audit v1.0.0..HEAD\n",[30,407,405],{"__ignoreMap":53},[16,409,410],{},"Read the damage report, then decide. Nothing is gated, nothing needs committing; delete the scaffolds and you have adopted nothing.",[16,412,413],{},"One honest limit, because honest limits are the house style: codument reports facts and gaps. It does not certify that a change is safe, and it does not try. The judgment stays with you; the instrument just makes sure you are judging the change that actually happened, against docs that are actually true.",{"title":53,"searchDepth":130,"depth":130,"links":415},[416,417,418,419,420,421,422],{"id":273,"depth":130,"text":274},{"id":283,"depth":130,"text":284},{"id":306,"depth":130,"text":307},{"id":328,"depth":130,"text":329},{"id":347,"depth":130,"text":348},{"id":372,"depth":130,"text":373},{"id":394,"depth":130,"text":395},"2026-07-11","The failure mode of AI-assisted development is quiet. Nothing crashes, the tests pass, the diff merges. And somewhere in your docs, a paragraph that used to be true just stopped being true.","LOG 02",{},7,"\u002Fblog\u002Fyour-agent-changed-14-files-which-docs-went-stale",{"title":268,"description":424},"blog\u002Fyour-agent-changed-14-files-which-docs-went-stale",[263,432],"change control","6zCXc1JmjKeyQ6yAJFt_i5q94YH2qQk20FIMkC5nqnc",1783802572397]