Estimated reading time: 6 minutes

Key points:

  • Of the four parts of an AI harness (context, tools, orchestration, feedback), the context layer delivers visible results fastest.
  • An AI knowledge assistant is that context layer in its simplest usable form: an assistant that answers questions based on your own documents, agreements and processes.
  • The quality stands or falls with the source, not with the model. Outdated or contradictory documents give you an assistant that confidently gives the wrong answer.
  • Start with one well-defined knowledge domain and one user group, and arrange from day one who is allowed to see which information.
  • Everything you build for the knowledge assistant (structure, permissions, freshness) is reusable for every next AI application. You are not building a standalone tool, you are laying the first layer of your harness.

Table of contents

  1. Why context is the logical first step
  2. What an AI knowledge assistant is (and what it is not)
  3. The source determines the quality
  4. How to approach it
  5. What this has to do with the rest of your harness
  6. How we can help
  7. Frequently asked questions
  8. Sources

Why context is the logical first step

In our overview article on the AI harness we describe the four parts of every serious AI system: context, tools, orchestration and feedback. If you are just getting started, you do not have to build all four at once. There is an order that holds up in practice almost every time, and it starts with context.

The reason is simple. Tools, orchestration and feedback only make sense once the model knows something about your organization. An agent that is allowed to create quotes but does not know your pricing agreements produces nonsense with a signature under it. The other way around does work: a model that can consult your documentation, agreements and processes is immediately useful, even before it is allowed to do anything.

That context layer in its simplest usable form has a name: the AI knowledge assistant.

What an AI knowledge assistant is (and what it is not)

An AI knowledge assistant is an assistant that answers questions based on your own sources: handbooks, contracts, process descriptions, project documentation, previous quotes, meeting notes. An employee asks "what is our notice period with supplier X" or "how do we handle a return above 500 euros" and gets the answer from the company's own documentation, with a reference to the source.

The difference with a regular ChatGPT session is in that last part. A bare language model knows everything about the French Revolution and nothing about your returns policy. A knowledge assistant retrieves the relevant passages from your own sources for each question and formulates the answer with them. No training on your data, no model that "remembers" your company knowledge: retrieve, read, answer, source attached.

What a knowledge assistant is not: a filing cabinet with a chat window in front of it. If you just dump a pile of unsorted documents into it, you get an assistant that is exactly as confused as the folder structure it runs on. Which brings us to the real work.

The source determines the quality

When setting up knowledge assistants we see the same pattern every time: the technology is arranged in days, the source takes weeks. Three problems come back at almost every organization.

Outdated information. The handbook says A, last year's work instruction says B, and the email containing the actual decision sits in someone's inbox. A person knows which version counts; an assistant does not. Before you point an assistant at your documentation, someone has to answer the question: what is the truth here, and where is it written down?

Implicit knowledge. Much of what an organization knows is written down nowhere. "We always run that past Karin" is not documentation. A knowledge assistant makes painfully visible which part of your operation exists only in people's heads. That is no reason not to start; it is one of the most useful side effects.

Permissions and confidentiality. Not everyone is allowed to see everything, so neither is the assistant. Salary agreements, client contracts and HR files do not belong in the pool everyone can ask questions about. In our own harness we deliberately gave one of our three knowledge layers no AI access for exactly that reason. You draw that line up front, not after something has surfaced in an answer.

Isometric line illustration of a ring binder with document flows and the Dutch label Bron (source), in And AI style on a dark navy background

How to approach it

The approach we use fits in four steps.

  1. Choose one knowledge domain and one group of users. Not "all our knowledge for everyone", but for example: product documentation for the service desk, or project agreements for project leads. Small enough to get the source in order, large enough to be used daily.
  2. Make the source correct. Assign one authoritative location per topic, clean up contradictory versions and record who maintains each document. This is the most work and the least technical.
  3. Arrange the access. Decide what the assistant may see and what it explicitly may not. Apply the reversed burden of proof: information is shielded unless there is a reason to make it available.
  4. Put the assistant in place and measure. Have the group genuinely work with it and collect the questions that got no answer or a wrong one. Every miss points either to a gap in the source or to an improvement in the setup. That is also how, on a small scale, your feedback loop begins.

From that point the system grows with you: more domains, more users, and eventually an assistant that not only answers but is also allowed to do things.

Isometric line illustration of an iceberg with a small tip above the waterline and a large foundation below, with the Dutch label Fundering (foundation)

What this has to do with the rest of your harness

Everything. The structure you put in place for the knowledge assistant (one authoritative location per topic, permissions per layer, an owner per document) is exactly the context layer every next AI application needs. The quoting agent you want next year reads the same sources. The meeting processing that files summaries in the right place writes to them. Whoever starts with a knowledge assistant is not building a standalone tool, but the foundation the rest of the harness stands on.

That is also the order we advise to companies that want to go beyond loose prompts: first the context layer, then agents with tools and orchestration around it.

How we can help

If you want to know whether your documentation is ready for a knowledge assistant, and where the first suitable domain sits, the AI scan maps that out.

If we then build it together, we do that within AI in Business: source in order, access arranged, assistant in use with your team.

Book a no-obligation call and we will look at your current documentation together and what it can already do.

Frequently asked questions

What is an AI knowledge assistant?

An AI assistant that answers questions based on your own documents and agreements, with source references. It retrieves the relevant information per question; your data is not used to train the model.

How is this different from ChatGPT?

ChatGPT does not know your organization. A knowledge assistant is connected to your sources and answers from them. The language skill comes from the model, the content from your documentation.

Does our documentation have to be perfect before we start?

No, but one domain has to be correct. Start with the part you can get in order and expand from there. The assistant will show you where the gaps are by itself.

What about confidential information?

You shield it up front. Decide per source whether the assistant may access it, and keep sensitive domains (HR, contracts, finance) out of reach until there is a well-considered reason and setup for them.

Isn't this just a search engine?

A search engine gives you ten documents that contain the answer somewhere. A knowledge assistant gives you the answer itself, with the source attached, and you can ask follow-up questions.

Sources