Building an internal knowledge assistant

Every organization loses time to the same problem: the answer exists somewhere: a policy, a past project, a document, or a person. Finding it is slow. An internal knowledge assistant is often the highest-value, lowest-risk place to start with AI, because the value is immediate and the blast radius is contained. It is also easy to get wrong in ways that only show up after launch, once staff have already decided whether to trust it.

What good looks like

• Grounded and cited. Every answer points to the source document, so staff trust and verify it.
• Permission-aware. People see only what they are allowed to see.
• In the flow of work. It lives where teams already are, not in yet another tab.
• Maintained. Someone owns the content, so answers stay accurate as things change.

Source curation is the real work

Which documents go in matters more than which model answers the question. Two conflicting versions of the same expense policy, one archived-but-not-deleted, will produce a confidently wrong answer regardless of how good the retrieval system is. Curation means deciding, source by source, what is authoritative and current, retiring or clearly superseding what isn’t, and assigning someone to own that decision on an ongoing basis, not just at launch. The system will only ever be as trustworthy as the mess it was pointed at.

Mirror permissions, don’t bolt them on

The common mistake is indexing everything and filtering what’s shown at the display layer. That leaves sensitive content sitting in the retrieval index, one prompt-injection or misconfigured query away from surfacing to the wrong person. Permissions need to be enforced at retrieval, mirroring the access controls already in place in the source systems: the same groups and roles that govern the document store or HR system should govern what the assistant can retrieve for a given user, and a permission change on the source system should take effect immediately, not after the next re-index.

Citation UX people actually use

A citation buried at the bottom of a long answer, or one that links to a 40-page PDF with no indication of where inside it, gets ignored. An unverified citation is functionally the same as no citation. Good citation UX links to the specific section or passage, shows a snippet of the source text inline, and surfaces how current that source is. If verifying an answer takes more effort than the answer saved, staff stop checking, and once they stop checking, trust in the tool becomes all-or-nothing.

An eval harness, not a vibe check

Launch-day testing is not evaluation. Build a standing set of real staff questions with known correct answers and expected sources, score the assistant against it before rollout, and rerun it whenever source documents, chunking, or the underlying model changes. Follow the same discipline described in What is RAG. Without this, quality regressions surface as quiet staff distrust long before anyone notices a problem in a dashboard.

Rollout: adoption is the hard part

Thomson Reuters’ 2024 Future of Professionals research found that professionals themselves estimate meaningful time savings from AI tools, on the order of four hours a week in the near term and growing toward roughly twelve hours a week within five years, but the same research found that estimated payoff was noticeably higher at organizations with a visible, communicated AI strategy than at those without one. Communication, not just capability, drives the return. A real rollout plan names champions on each team, gives staff an easy way to flag a wrong or missing answer, and reports time-to-answer and deflection numbers back to the organization so the win is visible, not assumed.

When not to build one

Skip it, or fix the foundation first, if the source documents are too disorganized, outdated, or contradictory to curate quickly. An assistant will confidently repeat that mess at scale rather than fix it. Skip it if the knowledge base is small enough that existing search already works fine; the project should solve a real friction, not add a layer to a non-problem. And skip it if the actual bottleneck is a process, not information: if people already know the answer but are stuck waiting on an approval, no assistant fixes that. In each of these cases, the fix is organizational: assign document owners, retire the conflicting policy, and shorten the approval chain. It is cheaper and faster than building a system to work around it.

Where it connects to Operate

A knowledge assistant is not finished at launch. Content goes stale, source systems change, and retrieval quality can drift as the document set grows. The same operational discipline that keeps any production AI system accurate, described on our Operate page, applies here: an owner for content, a monitored eval set, and a cadence for re-indexing and review. Firms that treat the assistant as a one-time build, rather than a system with an ongoing owner, tend to see it quietly drift back toward the original problem within a couple of quarters: accurate at launch, stale by the time anyone notices.

Why it is a good first system

It is bounded, measurable (time-to-answer, deflection), and it builds organizational confidence in AI before you tackle higher-stakes workflows. It is also the retrieval-layer foundation that later systems build on.

The hard part is rarely the model; it is adoption. See pilot to production, and start with an AI Decision Sprint.

A knowledge-readiness test before retrieval work

Score each proposed source collection from 0 to 2 on five questions: Is there an owner? Is the content current? Are permissions explicit? Can authoritative and superseded versions be distinguished? Are the questions users ask represented in the collection? A collection scoring below six out of ten is not ready for ingestion; retrieval will only make its ambiguity faster.

• 0: absent. nobody can answer the question or prove the control exists.
• 1: partial. the answer exists for some sources or depends on manual judgment.
• 2: explicit. the owner, rule, and evidence are documented and testable.

The readiness score identifies work that belongs before model selection: archive superseded files, assign owners, map access groups, collect real questions, and define what a correct cited answer looks like. That cleanup often creates value on its own. More importantly, it prevents the assistant from becoming a polished interface over an unresolved knowledge-management problem.