Most advice about AI in customer support starts with the savings. This guide starts with the failures, because the failures are where companies lose money and trust. Regulators and courts have already documented three recurring problems: customers trapped in loops with no exit, assistants that state policies that do not exist, and companies that assume the bot’s mistakes are somebody else’s responsibility. Below is a practical audit you can run before any support assistant goes live, with evidence from public sources for why each check matters.
If you want the broader picture of how the technology works, read how AI is transforming customer service first. If you are already planning a rollout, pair this audit with how to launch an AI customer support chatbot.
What the public record shows
The strongest case for automation is real. A study of 5,179 support agents published by the National Bureau of Economic Research found that a generative AI conversational assistant raised issues resolved per hour by 14% on average, and by 34% for novice and low-skilled workers, with minimal effect on experienced agents. Note the design: the tool assisted human agents rather than replacing them. In February 2024 Klarna reported that its assistant, built with OpenAI, handled two-thirds of its customer service chats in its first month, 2.3 million conversations, and said customer satisfaction was on par with human agents. These are company-reported figures, and they describe results at the time of the announcement.
The cautionary record is just as concrete. In 2023 the US Consumer Financial Protection Bureau reviewed chatbots in consumer finance. It found that each of the ten largest US commercial banks had deployed one, and estimated that about 37% of the US population, over 98 million people, interacted with a bank chatbot in 2022. The agency warned that chatbots can produce “doom loops,” in which a customer’s issue falls outside the bot’s limited capabilities and there is no way out to a human. It also stated that chatbots must comply with federal consumer financial laws and that institutions may be liable when they fail to.
In a 2024 decision in British Columbia, a tribunal ordered Air Canada to pay a customer $812 after its website chatbot gave wrong advice about bereavement fares. The airline argued the chatbot was a separate legal entity responsible for its own actions. The tribunal member called that “a remarkable submission” and wrote that the chatbot is “still just a part of Air Canada’s website,” so the company is responsible for all information on it. The ruling concerns one small-claims case and is not binding precedent everywhere, but the reasoning is easy to repeat: you own what your assistant says.
Audit 1: the exit test
Every support assistant needs a guaranteed route to a person. Test it as a frustrated customer would.
- Ask the assistant for a human in the first message. It should offer a route immediately, not after three clarifying questions.
- Ask the same question three times with different wording. If it repeats the same answer, the loop detector should trigger a handoff.
- Raise a topic outside its scope, such as a legal threat, a complaint about a fee or a bereavement situation. These should route to people by rule, not by the model’s judgment.
- Check the handoff itself. The agent who receives the case should see the conversation summary and what the customer already tried, so the customer does not start over.
- Check out-of-hours behavior. A handoff that leads to a closed queue is still a doom loop.
Write down the result as a pass or fail for each step. A single fail on the first or third step is a launch blocker.
Audit 2: the wrong-answer test
Assistants that answer from documents can still state things that are not in the documents. Build a test set before launch.
- Policy questions with edge cases. Collect 50 to 100 real tickets where the right answer depends on a condition (dates, plan types, regions). Have a policy owner write the correct answer.
- Questions with no answer. Include questions your documents do not cover. The correct behavior is to say so and route, not to improvise.
- Conflicting sources. Air Canada’s customer was told one thing by the bot and another by a different page of the same site. Seed your test set with known contradictions in your own knowledge base, then fix them.
- Money and commitments. Anything that promises a refund, a discount or a deadline should require a rule-based confirmation step rather than free-text generation.
Measure the share of answers a policy owner marks as correct, incorrect and unsupported. Review a sample weekly after launch, not only before it. See the glossary entry on hallucination for why a fluent answer is not evidence of a correct one.
Audit 3: ownership and records
Decide, in writing, who answers for what the assistant says.
- A named owner for the assistant’s behavior, with authority to pause it.
- A kill switch that disables generated answers and falls back to human-only or scripted flows within minutes.
- Conversation logs retained according to your privacy rules, so complaints can be investigated. Check which personal data the assistant can see and that it only shows a customer their own account.
- Disclosure. Tell customers they are talking to an automated system. Customers who feel deceived escalate faster, and some jurisdictions require disclosure.
- Regulated topics. In finance, health, insurance and utilities, involve compliance before launch. The CFPB’s point that chatbots are covered by the same laws as human channels applies by analogy elsewhere.
The NIST AI Risk Management Framework offers a vocabulary for this work: govern, map, measure and manage. You do not need to adopt it formally, but its structure is a useful checklist for who owns what.
Audit 4: metrics that expose problems
The headline number in many vendor reports is containment, the share of conversations the assistant closes without a person. Containment alone rewards the doom loop, because a customer who gives up counts as contained. Pair it with measures that catch abandonment.
| Metric | What it catches |
|---|---|
| Repeat contact within 7 days | Issues that were closed but not solved |
| Escalation rate by topic | Subjects the assistant cannot handle |
| Customer satisfaction, bot vs. human | Quality gaps hidden by volume |
| Share of chats where the customer asks for a human | Frustration with the automated path |
| Wrong-answer rate in weekly review | Drift as policies change |
Klarna’s announcement included a 25% drop in repeat inquiries, which is the kind of outcome metric worth tracking, though it is the company’s own figure. Segment every metric by language and topic, because averages hide the groups the assistant serves worst. The CFPB noted that limited-English-proficiency customers can be particularly poorly served by narrow chatbots.
A worked example
Suppose a subscription company launches an assistant for billing questions. The audit finds that the exit test passes in chat but fails on weekends, when the handoff queue is closed. The wrong-answer test shows the bot quotes a 30-day refund window from an old help page, while the current terms say 14 days. The metrics plan tracks only containment. Before launch, the team adds a weekend callback option, retires the old page, routes all refund promises to a confirmation step, and adds repeat-contact rate to the dashboard. None of these changes needed a better model. They needed a process around it.
What to watch next
- Further rulings on whether companies are liable for statements made by their assistants.
- Regulator guidance on disclosure and human-escalation rights in customer service.
- Published results from companies that mix assistants and human agents, with outcome metrics beyond containment.
- Evidence on how assistants perform for non-native speakers and for customers with accessibility needs.
Key takeaways
- Treat the assistant as part of your website and your staff: you are responsible for its statements.
- Test the exit path, the wrong-answer rate and the handoff before launch, and keep testing weekly.
- Use rules, not free-text generation, for refunds, discounts and deadlines.
- Pair containment with repeat-contact and satisfaction measures.
- Name an owner and keep a kill switch.
Frequently asked questions
Is a company liable for what its chatbot says?
In the Air Canada case a Canadian tribunal said yes, treating the chatbot as part of the company’s website. It is one decision, but it shows how a tribunal can reason.
What is a doom loop?
The CFPB uses the term for a chatbot flow that keeps a customer cycling through repeated, unhelpful responses with no way to reach a person.
Is containment rate a good success measure?
Only with companions. A customer who gives up looks contained, so pair it with repeat contacts, satisfaction and escalation by topic.
