Test design — not executedCustomer support agents
How to evaluate customer support agents
Use a bounded task with observable results. The candidates below are research starting points, not tested recommendations.
Source-linked research and editorial guidance. Capabilities are not independently tested unless a profile explicitly includes a reproducible test record. Sources checked November 13, 2026.
- Inputs
- An approved policy set, fixture customer records and tickets with known resolution paths.
- Expected deliverable
- Grounded answers, correctly executed permitted actions and escalations with context.
- Procedure
- Include answer-only tickets
- Include identity-sensitive actions
- Include exceptions
- Run blind cases
- Review policy adherence and customer outcomes
- Approval boundary
- Separate answer permissions from refunds, account changes, cancellations and disclosure of private data.
- Acceptance criteria
- Resolution quality, policy compliance, escalation quality, action correctness and cost per accepted resolution.
- Failure modes
- Calling a conversation resolved when it only ended, applying a policy to the wrong customer and suppressing escalation.
Candidates to investigate
Editorial topic relevance — task support not verified. Research starting points — none of these has been run against this task.
Sierra
Builds customer-facing agents across chat, voice, messaging, email and ChatGPT.Decagon
Customer concierge platform for building and optimizing AI agents.Ada
Customer-service agents with task tools, playbooks and integrations.Forethought
Support-agent suite covering resolution, triage, QA and knowledge insights.Yuma AI
Ecommerce support agents handling order status, returns, refunds and order changes.Sources
- sierra.ai (opens in a new tab)
- decagon.ai (opens in a new tab)
- ada.cx (opens in a new tab)
- forethought.ai (opens in a new tab)
- yuma.ai (opens in a new tab)
Something out of date? Suggest a correction