When Ticket reviews is on (Settings → AI assist, on by default), an AI reviewer reads each conversation about a day after it closes. It skips spam, feature requests and conversations with nothing to judge, and a reopen before then cancels the review. It records how the conversation ended, who solved it, and how Kai Support, Kai Resolve and each teammate did. You'll find the results under Reports → AI Agents → Overview (the AI quality page).
Every reviewed conversation gets exactly one outcome, judged from the customer's side:
Resolved: the customer's need was met. Either they confirmed it, or the last real reply fully answered them and they never came back with an objection. Silence after a complete answer counts as resolved.
Partially resolved: part of the need was met, a workaround was offered, or the answer was incomplete.
Unresolved: the need wasn't met, and the last move was the team's or the bot's. Examples: a wrong or missing answer, a promise nobody followed up on, or a teammate who claimed the ticket and never replied.
Customer left: the ball was in the customer's court. They were asked for information or a decision and never gave it, or they left before saying what they needed. This is nobody's failure.
Handed off, no reply: the bot handed the conversation to a person or another team, and no teammate reply is visible. Usually the conversation continued somewhere else, or was closed right after the handoff. This is nobody's failure either. If a teammate claimed the ticket and then never replied, the outcome is Unresolved.
If a ticket was reopened, only the part after the reopen is judged.
Kai Support, Kai Resolve, Teammate, Customer self-served or Nobody. A teammate's "thanks, glad that helped" after Kai's answer doesn't make it a teammate resolution. Only a reply that added substance does: new information, a correction, or an action Kai couldn't take. Internal notes never count, because the customer can't see them.
Resolved conversations divided by all reviewed conversations except Customer left and Handed off, no reply. Partially resolved doesn't count as resolved. Leaving out the two neutral outcomes means a bot that hands off exactly when asked doesn't drag the rate down.
Kai Support's own replies are scored 1–5. Its mistakes are tagged as Wrong answer, Made something up, Missed existing knowledge, Escalated unnecessarily, Should have escalated, Wrong language, Repeated itself or Policy violation. Handing off at the right moment counts as a correct answer. So does saying "no answer found" and pointing to the right next step, when the answer really isn't in your knowledge base.
This review scores whole conversations. A single agent run has its own outcomes (Resolved, Escalated, Abandoned…), explained in How an agent run can end.
