Chatbot Metrics That Matter: Containment, Satisfaction and Handover

Most chatbot dashboards open with a big number: conversations this month, messages handled, hours saved. It looks impressive and tells you very little about whether customers got what they came for.

The chatbot metrics that matter measure outcomes. Was the customer's problem solved? How did they feel about it? Did the bot hand over when it should have, and did a person reply in good time? Did the conversations that should end in a booking, a lead or a payment actually get there? This guide defines each metric, shows how to measure it, flags the vanity metrics to drop and sets out a monthly review routine.

Start with What the Chatbot Is for

Metrics only make sense against a job. A bot that answers order status, one that books appointments and one that qualifies sales leads succeed in different ways, so list the jobs first, as you would when choosing a conversational AI chatbot. Then fix three definitions before you count anything:

• A conversation. Decide when one ends, for example after a set period of silence, so a WhatsApp chat that resumes the next morning is counted the same way every month.
• What is in scope. Exclude test traffic, spam and internal users, and decide whether a greeting with no question counts.
• The outcome of each intent. For order status, a status shown to a verified customer; for bookings, a confirmed booking; for leads, contact details and a need recorded in the CRM (customer relationship management system).

The Core Chatbot Metrics, Defined

Seven measures cover most of what a service team needs to know. Write the definitions down, because two tools, or two analysts, rarely count the same way by default.

Containment and Resolution: Read Them Together

Containment is the easiest metric to report and the easiest to flatter. A customer who gives up after two unhelpful answers ends the conversation without an agent, so they count as contained. Resolution asks the harder question: was the need actually met?

Evidence can come from a closing question such as "Did that answer your question?", from a completed flow, such as a confirmed booking, or from the absence of a repeat contact on the same topic within a window you choose, for example seven days. Whatever tool you use, check exactly how it defines containment before you compare one month with the next.

Satisfaction That Means Something

Ask one short question at the end of the conversation, a thumbs up or down or a rating from one to five, with an optional comment. Report the share of satisfied answers together with the response rate, and split it by whether a person was involved. The comments are often more useful than the score, because they say which answer annoyed the customer.

Handover Rate and Time to a Human

Handover is not a failure. Complaints, disputes and sensitive topics should reach a person, and a bot that never hands over may be trapping customers. Report handovers by trigger, track the time from handover to the first human reply and count the customers who leave while waiting. Our guide to chatbot human handover covers the triggers and measures in detail.

Fallback Rate and the Training Queue

Every fallback is a question the bot could not handle. Group fallbacks and low-confidence answers by topic each month, and the largest groups become your next content and training priorities. A sudden rise usually follows a change nobody told the bot about: a new product, a price change or a holiday timetable.

Conversions That Reach the System of Record

For sales and bookings, count the outcome where it is recorded, not only in the chat. A lead counts when it reaches the CRM with contact details and a need, a booking when it is confirmed and a payment when the money arrives. With the bot connected to your ERP (enterprise resource planning system) or CRM, such as StarBiz or Odoo through OdooStar, you can follow chat leads through to orders and see which conversations create revenue.

If you also report cost per resolved conversation, include platform fees, agent time on handovers and channel charges. On WhatsApp, Meta charges for delivered template messages at rates that depend on the template category and the recipient's country, while replies inside an open customer service window are free, as its WhatsApp Business Platform pricing page explains.

Vanity Metrics to Drop

Some numbers look good in a slide and lead teams in the wrong direction. Retire these, or at least stop leading with them:

• Total conversations or messages. Volume rises when the bot is confusing as well as when it is popular.
• Average conversation length. A long chat can mean an engaged customer or a stuck one.
• Number of intents or articles. Coverage on paper says nothing about answers in practice.
• Accuracy without a method. An accuracy figure means little unless you know what was tested, on which conversations and by whom.
• Containment on its own. Without resolution beside it, containment rewards a bot that wears customers down.
• Hours saved from assumptions. Savings built on a guessed handling time per chat are easy to inflate; if you report them, show the assumptions.