Why a knowledge file rots, and the one habit that keeps it alive
I keep a distilled file of working rules for the AI assistant I build with — the accumulated “here is how I actually work,” boiled down from a self-report I write at the end of most sessions. It started as a handful of notes and it is now the thing a fresh session reads before it touches anything. Splitting it so it stops costing me on every load was last month’s job. This month’s realization is subtler and more important: a file like that does not stay true on its own. It rots.
The rot is quiet. Every session generates a new note — a rule I sharpened, a mistake I do not want repeated, a decision that changed how something works. Those notes pile up in a folder. The distilled rule file, meanwhile, sits unchanged, slowly falling behind the notes. Nothing breaks. The file still loads, still reads fine, still looks authoritative. It is just increasingly describing how I worked six weeks ago instead of how I work now. A knowledge base that is never re-consolidated is a photograph slowly going out of date while everyone treats it as a mirror.
The fix is not cleverness, it is cadence. Periodically — when enough new notes have stacked up, or enough time has passed — the notes get merged back into the distilled file, and the file becomes current again. That is the whole discipline. Consolidate on a schedule, not “when I get around to it,” because “when I get around to it” is precisely never, and the gap between the notes and the file just widens until the file is lying.
Two things I learned building the machinery that does the merge, both of which generalize past my particular setup:
The distilled file is the source; everything else is derived. There is exactly one authored artifact. The cheaper, load-every-session version and the searchable evidence index are both generated from it at read time — never edited by hand. The instant you let someone hand-edit a generated copy, you have two sources of truth and no way to know which is right. Derived things must be derived, always, or they drift from their parent the moment someone “just fixes one thing” in the wrong place.
And: the file you edited is not the file being served. This one got me. I changed the source, checked it, called it done — and the running system was still handing out the old version, because editing a file and reloading the thing that serves it are two separate acts and only one of them happened. The habit that saves you is boring and non-negotiable: after you change something, go read the thing that is actually running, not the thing you just saved. “It looks right in my editor” is a different claim from “the live thing is right,” and only the second one matters.
None of this is exciting. It is maintenance — the unglamorous work of keeping a thing you already built from quietly becoming false. But a knowledge system you do not maintain is worse than no system, because it wears the authority of being written down while being wrong. The schedule is the whole trick. Put the re-consolidation on the calendar, treat the source as the only thing you edit, and check the live copy every single time. Do that and the file stays a mirror instead of a photograph.