📘 Best practices: Maintaining your Knowledge Agent over time

A Knowledge Agent answers your team's questions by drawing from the knowledge your organization has documented in Guru and connected sources. The better that knowledge is — accurate, complete, and up to date — the better the agent's answers will be.

Once your agent is live, maintaining it means staying on top of what your team is asking, understanding where answers are falling short, and using those signals to keep the underlying knowledge current. This article walks through how to do that.

Setting up your foundation

Two things are worth getting in place early. The first is a one-time settings change. The second is an ongoing habit that pays off over time.

Turn on quality automations. Guru can automatically flag content that's gone stale or hasn't been verified in a while, so outdated information gets caught before it affects your agent's answers. Auto-verify is on by default. To also catch content that may need attention, enable auto-unverify in the Quality tab. See Setting up Knowledge Agent Quality for more.

Check that your sources cover what your team is asking about. Your agent can only draw from content it has access to — the Guru collections and external sources connected to it. If a whole topic area isn't covered by a connected source, the agent won't be able to answer questions about it. It's worth reviewing your connected sources periodically and asking whether they reflect where your organization's knowledge actually lives. See Managing Knowledge Agent sources for how to add or update connections.

Reviewing agent activity in the AI Agent Center

The AI Agent Center is where you can see how your agent is performing - what questions people are asking, how the agent responded, and what feedback users have left. Think of it as a window into the real-world experience of everyone using the agent, rather than a to-do list of things to fix.

As an owner, you might check in when you want a general sense of how things are going, or when a specific question or piece of feedback catches your attention. You don't need to review every answer — the goal is to notice patterns over time. Filters like Flagged, and Not Answered can help focus your review.

When something specific warrants a closer look, Answer Details shows you exactly how the agent constructed a particular response: which content it drew from, and where the answer may have gone wrong. You can access Answer Details by selecting any answer in the Agent Center. This is particularly useful when a user reports that a topic isn't being handled well and you want to understand why before deciding what to do about it.

Building a regular picture with automations

Looking at individual answers tells you what happened in a specific moment. Understanding how your agent is performing over time - and where gaps are building up - means looking at patterns across many questions.

A good way to do this without manual effort is to schedule a weekly Knowledge Agent automation that reviews your Agent Center data and sends you a summary. You set up the automation once — including the prompt that tells it what to look for — and it runs on the cadence you choose. Summaries can be delivered to a Slack channel or reviewed directly in the Agent Center if your team doesn't use Slack.

The prompt you write determines how useful the output is. A broad prompt returns a long list of individual questions. A more specific prompt returns a grouped, prioritized summary that's easier to act on.

This gives you a weekly view of where the knowledge base has room to improve — organized by what matters most rather than by when questions were asked.

Reading the patterns

Once you're getting regular summaries, the patterns that emerge tell you something useful about the state of your organization's knowledge — not just how the agent is performing.

Questions on the same topic consistently go unanswered.
The knowledge likely doesn't exist in a form the agent can use. It may not be documented anywhere yet, or it may live in a source that isn't connected to the agent. This is one of the most common patterns, and it usually points to an opportunity: knowledge that exists somewhere in your organization but hasn't been captured yet.

Questions are being answered but frequently get negative feedback.
The content exists but something isn't quite right. It might be outdated, incomplete, or technically accurate but not useful for the way people are actually asking the question. This pattern often surfaces content that was written for one purpose and is being stretched to cover something slightly different.

The same question gets inconsistent answers depending on how it's phrased.
Usually a sign of duplicate or conflicting content — multiple sources covering the same topic with different information. The agent draws from whichever one best matches the phrasing of the question, which produces inconsistent results. Consolidating that content into a single authoritative source tends to resolve it.

A topic that used to get good answers starts degrading.
Something changed - a process, a policy, a product, or the source content itself. This pattern is easy to miss because the agent was working before. Automated verification helps catch it, but it's worth paying attention to topics where answer quality shifts after a period of stability.

The same topic keeps appearing week after week.
When a gap persists despite updates, it usually signals that the knowledge in that area needs more sustained attention — not just a one-off card, but a clearer sense of who owns that topic and how it stays current as things change.

Good sources are being used but the answers are still off.
Most degraded answers trace back to content — missing, outdated, or conflicting. But occasionally the content is fine and the problem is in how the agent is using it. Answer Details is the tell: if the sources being cited look accurate and relevant but the answer still isn't right, that's a signal the issue may be in the agent's skill configuration rather than the knowledge itself.

This is relatively uncommon, but when it does happen, Operator Mode is where you address it. You can bring a specific answer ID into Operator Mode and work through what went wrong conversationally — the agent traces the failure and can draft a fix to the relevant skill, which you review and approve before anything changes. It's worth checking content first, but when content isn't the issue, Operator Mode gives you a way to diagnose and adjust the skill without leaving the agent.

Using what you learn to strengthen your knowledge base

The patterns you notice over time are more than a maintenance checklist — they're a map of where your organization's knowledge is strong and where it has gaps. Topics that consistently surface as problems often reflect areas where knowledge exists in people's heads but hasn't been captured in a way the broader team can access.

As a Knowledge Agent owner, one of the most impactful things you can do is use that visibility to connect the dots. If a topic keeps coming up as a gap, the next step is usually a conversation — with a team lead, a subject matter expert, or whoever owns that area of knowledge — about getting it documented. That might mean asking someone to write or review a Guru card, or identifying a shared resource (like a folder of internal documents) that could be connected to the agent as a source.

The agent improves as the knowledge behind it improves. And over time, keeping that knowledge current becomes a shared habit across the team rather than a task that falls entirely to the owner.