--- type: "article" title: "Its the knowledge gaps, not the AI." summary: "The ceiling is the knowledge gaps in definition, ownership and access, not by model capability." newsletter: "Joyce Stack" newsletter_handle: "joycestack" newsletter_url: "https://work.joycestack.com/" author: "Joyce Stack (@joycestack)" published: "2026-08-14T12:15:03.148Z" canonical_url: "https://work.joycestack.com/a/S689qdzL" markdown_url: "https://work.joycestack.com/a/S689qdzL.md" chat_url: "https://work.joycestack.com/a/S689qdzL/chat" body_source: "native" likes: 0 replies: 2 body_words: 641 --- # Its the knowledge gaps, not the AI. For the past month or two, my colleague and I have spent time running a bazillion evaluations on trying to figure out how to roll out corporate rule sets across the organization. We tested prompts, RAG, context, memory and guardrails. **The ceiling is the knowledge gaps in definition, ownership and access, not by model capability. ** ### **Challenges with All Approaches** Trying to pin point when things work or not feels impossible given the separate variables. There is much to contend with: - **the consuming team experience** - how are teams going to work with these rule sets day by day, how do we distribute them? - **the authoring experience** - how are rulesets to be written, structured, and broken into pieces before that can be cited by teams. - **the spec frameworks **- OpenSpec vs SpecKit and the nuances that they present. - **individual AI tools - **the combinations are endless Claude vs. OpenAI models, teams local preferences and settings. The list goes on. ### How Old Skool Knowledge Management and Information Architecture Techniques Have Helped My active research focus in recent months has been getting to grips with old skool knowledge management and IA techniques. My sense is that these concepts map to AI native systems: - knowledge object decomposition (breaking a document into independently evolving answerable units) is the upstream input to chunk boundary design (documents with multiple knowledge objects produce noisy chunks in a RAG layer) - controlled vocabularies (elects one canonical term per concept) map to entity resolution making sure every mention of the same thing, however it's worded, gets tied back to one record instead of being tracked as separate, disconnected things. Old wine in new bottles it seems to me. Understanding vocabulary fragmentation across your corpus helps because when a human reads "service" in one standard and "application" in another then they can fill in that context. An agent sees two different things. When you write the word 'key' do you mean encryption key, API key, SSH key or something else? The invisible problems in documentation are invisible precisely because humans compensate for them and we know agents don't compensate. A few things have stood out for me corpus after corpus analysis: - **Tag rules, not documents - **policy documents are not just one thing - they are bundles of rules, each with their own type, obligation level and obligation owner. Metadata at the rule level rather than the document level help an agent understand what's guidance, whats mandatory and who to escalate it to when its unclear. - **Enforce terminology **- vocabulary drift has been super interesting - same concept, three names, no canonical mapping. Agents can't tell if "scope", "boundary" and "containment" are being used interchangeably. My sense is that many teams are relying on accidental good behavior without very through evaluation frameworks. Teams should check cause not just appearance. None of these are new. Taxonomy becomes ontology. Controlled vocabularies becomes entity resolution and metadata schemes become structured context. ## Building the tooling to check my own claims. While many teams are battling with the complexities of learning AI workflows and tooling, I've been drawn to the underlying knowledge. These are new disciplines to me and not something that I've had deep exposure to however I do wonder if every organization should now be hiring librarians? Sharing some skills that you can run against a corpora of markdown: here is [https://github.com/JoyceMarieStack/markdown-ia-skills](https://github.com/JoyceMarieStack/markdown-ia-skills). Its a three-skill sequence (corpus discovery -> content model discovery -> vocabulary governance). The work is ongoing and the findings (and design) are rough in places. That's the honest version. I'd rather show the audit in progress than a polished conclusion I haven't earned yet. *If you run these skills then please let me know. If anything, I hope they are a learning resources. I'd like to hear about it. * *** ## Discussion (2 replies) Numbering shows the reply tree: `2.1` is a reply to `2`. Message text is reproduced verbatim as posted. ### 1. Kin Lane (@kinlane) - 2026-08-14T12:32:48.833Z Great post Joyce! Love the bringing forth your legacy wisdom. The investment in trying to tame the markdown is commendable. I love the "tag rules" -- so important. Amplifying. ### 2. Joyce Stack (@joycestack) - 2026-08-14T13:21:36.298Z Thank you @kinlane Wish I could share some of my report findings because it really is an absolute eye opener for what I feel many orgs are going to have to contend with.