Setting up Knowledge Agent Quality

Download the setup guide here.

Knowledge Agent quality lets your agents automatically verify and unverify content based on usage signals and rules you define. Instead of manually reviewing every card, your agent evaluates content on a nightly schedule—surfacing what's trustworthy, flagging what's stale, and giving you a transparent log of every decision it made.

👥
Access Required
You must be an Owner on a Knowledge Agent to access and configure its Quality tab.

Setting up the quality workflow

Follow these steps to get Knowledge Agent quality up and running.

Step 1: Choose or build your Knowledge Agent

Start by identifying the agent whose content you want to review. You have two options:

Use an existing agent. Choose an agent that already has unverified content that needs to be reviewed—for example, an onboarding agent pulling from Google Drive, Guru Cards, and Asana.
Build a new agent. Create a Knowledge Agent scoped to the specific collections you want to audit. This is especially useful for systematically reviewing content within Guru.

✍️
Note
For sources that are not Guru Cards, content must have been used by the agent to answer a question before it can be quality-reviewed.

Step 2: Configure source limitations

You can control which sources your agent draws from when answering questions, filtered by verification status. This setting is in the agent's Quality tab under "Which sources connected to this agent should be used?"

Your options:

All connected sources. The agent uses both verified and unverified content.
Connected sources that are verified or have no status. The agent uses verified Guru content and external sources, which have no verification status.
Only verified connected sources. The agent answers exclusively from verified content.

💡
TIp
The third option is recommended when accuracy is critical—it ensures users only receive answers grounded in content your team has explicitly trusted. This will limit available sources used to answer questions, however, so should be turned on with that consideration.

Step 3: Enable and review verification rules

In the Knowledge Agent's Quality tab, toggle Auto-Verify on.
Review the default rules Guru provides. You can edit or delete any rule to fit your use case.
Let the automation run overnight to see it in action before making further adjustments.

Default auto-verify rules include:

If the document has been thumbs-upped in an answer more than 5 times in the past month.
If the card was used in an answer, the asker was in the verification group, and the answer received a thumbs up or no feedback.

✍️
Note
If you're conducting a content audit or looking to remove inaccurate information, toggle Auto-Unverify on as well and review its rules.

Step 4: Test on specific content

Before trusting the automation at scale, manually test it on cards you already know are accurate or inaccurate. To run a manual quality check on a card:

Open the card and click the ⋮ (more options) menu in the top right corner.
Select Check this Card's quality.
In the Quality check panel, choose the Knowledge Agent to evaluate the card and select whether to check if it should be Verified or Unverified.
Click Run check.

The agent returns its determination (verified or unverified), a confidence level (High, Medium, or Low), and a plain-language explanation of which rule applied and why.

✍️
Note
If you want to test with external sources, let the automation run overnight first—external content must have been used in an agent answer before it's eligible for quality review

Step 5: Refine your rules

After reviewing your test results, return to the agent's Quality tab to adjust rules as needed. You can:

Edit a rule. Click the ⋮ menu next to any rule and select Edit.
Delete a rule. Click the ⋮ menu and select Delete.
Add a rule. Write new rules in plain language, for example: "If a Card has been viewed more than 5 times in the past month, then verify it."

💡
Tip
Start with the defaults, observe how they perform, and add specificity over time

Step 6: Expand to more sources and agents

Once you're confident in your rules:

Add more sources gradually. Layer in additional sources and verification criteria over time.
Enable quality on more agents. Roll out Auto-Verify to other Knowledge Agents across your workspace.

Reviewing the quality log

After your rules run, you can review what happened in the Quality tab of the AI Agent Center. Select an agent from the Review answers for dropdown, then click the Quality tab.

The log shows a summary row for each nightly run, including the date, agent name, and a count of records processed, verified, unverified, and no-actioned. Click any row to expand it and view the individual documents reviewed.

Each expanded entry shows:

Source/Card. The specific card or document evaluated.
Action. Whether the agent verified, unverified, or took no action.
Confidence. High, Medium, or Low, based on the strength of the signals available.
Reason. A plain-language explanation of which rule applied and why.

Quality rules run every night. After a source is reviewed, it won't be reviewed again for 7 days. The Quality Log respects content permissions—you may not see all records processed by an agent if you don't have access to the underlying content.

Running a manual quality check

In addition to the nightly automation, you can run an on-demand quality check on any card at any time. This is useful when a stakeholder flags a specific card, you've added a new rule and want to test it immediately, or you're onboarding a new source and want to spot-check content before it surfaces in answers.

To run a manual check:

Open the card and click the ⋮ (more options) menu in the top right corner.
Select Check this Card's quality.
Choose the Knowledge Agent you want to evaluate the card against.
Select whether to check if the card should be Verified or Unverified.
Click Run check.

The agent returns a confidence level and a plain-language explanation of its determination.

📘 Best practices

Start with Auto-Verify only. Get a feel for how the agent evaluates content before enabling Auto-Unverify, which will actively remove verified status from cards.
Watch the "No action" count. A high no-action rate often means your rules are too narrow or your content hasn't been used enough to generate signals. Broaden your criteria or let the agent answer more questions first.
Pair with source limitations. Setting an agent to answer only from verified sources becomes much more powerful once quality rules are actively curating which content earns that status.
Use a dedicated audit agent. For a one-time content audit, build a new Knowledge Agent scoped to specific collections. You can repurpose it afterward.