The accuracy component of Automated Resolutions (AR) is now measured by incorporating Processes, Actions, Coaching, and Company Descriptions as additional grounding factors.
Previously, AR evaluation only considered knowledge content to ground the evaluation of accuracy. Now, additional grounding factors are considered.
These enhancements provide a more precise accuracy assessment, reducing cases where conversations were previously judged as inaccurate—particularly when the AI agent leveraged processes, actions, or coaching during a conversation.
AR should generally stay the same or improve
- More grounding factors now contribute to verify accuracy.
- Conversations that previously lacked knowledge references may now be correctly classified as accurate based on new grounding factors.
- Contradictions between knowledge and other elements (processes, actions, coaching, or company descriptions) may result in a conversation being considered inaccurate and therefore unresolved.
- False positives, where the LLM predicts a conversation was resolved despite the user explicitly saying in the survey that it wasn't, and false negatives, where the LLM predicts a conversation was not resolved despite the user explicitly saying in the survey that it was, have both decreased as a result of this calculation change.
Learn more about how AR is calculated.