How Much Context Do Your Docs Consume? Experiments in Finding Out

Your AI agent has a fixed budget of working memory and your documentation is spending it without you knowing. How Much Context Do Your Docs Consume? Experiments in Finding Out Working in technology in 2026 means spending a large part of your days (and some evenings) pulling the AI slot machine lever. Repository documentation is now runtime context for AI systems. Every token consumed by always-loaded files reduces space available for reasoning, tools, code and execution. When context exceeds a platforms available context or per-read limits, the platforms may truncate, compact, partially load or omit parts of the markdown file. This is done silently, in the background with no error or no warning. The model proceeds as if it needed the complete file. This behaviour varies by tool, model, session state and how the file is accessed. This is often referred to as the silent truncation problem. So what? Rules, constraints and guardrails that you have so lovingly defined may simply go unread by the agent. We know that if an agent is missing context it may simply hallucinate and default to the underlying training data. That's a single-session problem. The cost problem is worse. The multiplication problem causes compounding at scale. A large CLAUDE.md file is not a one off cost. It is loaded in every session, by every developer and across every repository. So a bloated CLAUDE.md (18,000 tokens) consumed by 10 developers across 5 daily sessions results in ~900,000 tokens/day. Every token your documentation wastes is a token that can't be used for solving problems in your code. Documentation is not free. So I built a thing The docs context checker scans your repository's markdown files and reports how much of each platform's usable context they consume. The question it answers: "What percentage of a platform's usable context do these repository files consume?" The key word is usable. It doesn't measure against a model's theoretical million-token window. It measures against what the platform actually gives you after system prompts, tool definitions, and output space are accounted for. The Design Approach I'm calling this section 'design approach'. The word design is doing a lot of heavy lifting. This is where I ended up from puling the AI slot machine lever until my arm nearly fell off. The approach is using the same tokenisation as GPT-4o/GPT-5 (o200k_base) which is good enough for now. I wanted this to be something that I can run locally without access to APIs and using my precious budget. I wanted something that is (1) useful for routine checks (2) and try and learn something about markdown and context windows. Here is what I built: Content-aware. Classifies each file into segment types (prose, code, tables, frontmatter, headings, lists). Load frequency classification. Distinguishes between files loaded at session startup (CLAUDE.md, AGENTS.md) and files retrieved on demand. Always-loaded files are a fixed tax on every conversation. Their cost isn't a file size problem - it's a multiplication problem. Operational budget model. This is the critical bit…this does not measure against the raw model limit. This measures against the platform's operational budget - the space left after system prompts, tool definitions, and output are accounted for. Each platform is configured with its own budget: Platform configuration What the output looks like For each file the checker produces a per-file report with token count, severity, segment breakdown, context consumption per platform, and estimated read cost. Here's a trimmed example from a real scan of 117 documentation files. One file, 13,381 tokens, consuming 11.2% of Claude Code's operational budget. : Example of output for a file And the summary view of the full scan which shows a healthy repository. Using a severity band - this shows there are 3 files which are signifcant in size but the remaining are marked as comfortable. Example of summary output What this doesn't tell you The checker measures context cost, not value. A comprehensive guide that lets an agent answer correctly is worth more than a brief summary that forces it to guess. The tool tells you where the weight is. What you do about it is a judgement call. Two actions are safe right now: Trim what loads on every session. Files like CLAUDE.md and AGENTS.md are loaded before the agent starts working. If they're bloated with background rationale or historical context that doesn't affect agent behaviour, every wasted token is a tax on every session, every developer, every day. Check what's too long to read in one pass. Some platforms have per-file read limits. When a file exceeds them, the agent gets a truncated version and carries on as if it read the whole thing. If your constraints are at the bottom of an oversized file, the agent may never see them. Put critical content early, or split the file. Limitations and what's next? The checker is part of a broader project called Agent Ready Docs. The pipeline checks whether documentation exists, is structured, and is parseable. That's necessary but not sufficient. The missing piece is we don't yet know whether passing those checks makes a difference to how an agent performs. Early experiments suggest the answer isn't obvious. I deliberately broke the formatting of a markdown table across six variations and asked Claude to extract values. It got them right every time - except when cells were empty. The thing most people would optimise for didn't matter - nice tables with headers. It turns out the thing most people may overlook did (https://usecommune.com/n/joycestack/a/i-broke-every-table-formatting-rule-and-claude-didn-t-care-e) That's one small experiment. My project is not an accurate reflection of how AI reads projects. AI selectively inspects, extracts and summarises work. Before optimising for AI agents, I would like to experimentally validate how different tools and models retrieve, read, chunk and reason over project content.I need evidence to figure out which checks matter, which are superficial and what is missing. Another flaw in the project is that it treats all documentation the same. I think it needs a documentation classification step to be really useful. Do they need different quality checks? I don't know yet. Closing It seems to me that one ought to understand what it is they are optimising for: accuracy, speed or token cost. They all pull indifferent directions. Trimming a file may save tokens but may remove context that an agent needs. Accuracy may matter more for anything that touches production 🤯🤯🤯 however may matter less for local dev gubbins. Here you may optimise for speed. This project does not attempt to prove how agents read repositories. It makes documentation context consumption visible and measurable. Before optimising for AI agents, we need evidence of what matters for accuracy, not assumptions. That's documentation engineering. Until then, I will continue to focus on the human. Joyce Stack Hove, Brighton United Kingdom You are receiving this email because you signed up at joycestack.com. If you no longer wish to receive emails from us, you can unsubscribe at any time. Unsubscribe