LLM-as-a-Courtroom

Summary

Shared knowledge—decisions, plans, documentation, processes—is a living, breathing resource that evolves as companies do. Falconer is building a shared memory layer for cross-functional teams and their agents to maintain their shared knowledge. A core part of our solution means watching for code changes and proposing documentation updates automatically. “Documentation rot” is a term used a lot by our users. The code changes, documents go stale, and the knowledge that was once accurate becomes a liability. Many tools now allow users to find information with AI. But findability alone doesn’t equate to accuracy. An easily findable doc that’s out of date doesn’t solve the problem. The hardest part of the problem isn’t finding right quanta of knowledge, rather it is the ability to trust it when you do find it. Automating documentation updates Falconer automatically updates documents based on changes to the code. But when a PR merges, how do you decide which documents to update? This isn’t a simple pattern-matching problem. Yes, you can filter out obvious non-candidates—test files, lockfiles, CI configs. But after that, you’re depending on judgment. The PR code is complex and contextual, as are the documents. Different teams have different priorities. A change that’s critical for customer support documentation is probably irrelevant for engineering specs. And the audience reading the document matters as much as the change itself. If a human were reading all code changes and updating docs manually—reading each PR, understanding the diffs, searching for affected documents, deliberating on whether each one needs updating, and making the actual updates—it would take days for a relatively small number of PRs. Falconer’s agent does this in seconds. However, building an intelligent agent is just one piece of the puzzle. We also needed to build the infrastructure to process tens of thousands of PRs daily for our enterprise customers. And more importantly, we needed to build a jud...