Introduction
To understand the value your AI Chatbot is providing, it is important to monitor key performance indicators (KPIs). These metrics help you evaluate how effectively the chatbot supports guests, resolves questions, drives engagement, and contributes to booking and revenue opportunities.
Most of these metrics can be monitored directly from the Insights tab within the Myma.ai Dashboard.
1. Response Time
This measures how quickly the chatbot responds to user inquiries. Faster response times are crucial for a positive user experience, with a good goal being less than 5 seconds.
2. Resolution Rate
Tracks the percentage of conversations where the chatbot successfully resolves the user's query without human intervention. Aim for a 70-80% autonomous resolution rate. For anything that the chatbot is unable to answer, it should create a ticket and assign it to the right staff member.
3. Fallback Rate
This metric shows how often the chatbot fails to understand or handle a user’s request and escalates it to a human agent or falls back on a generic response. A lower fallback rate indicates better training and adaptability.
4. User Satisfaction
Myma.ai's Chatbot measures customer satisfaction during each conversation by allowing users to leave a thumbs up or thumbs down, along with optional comments. A satisfaction score of 90% or higher, based on positive feedback, is generally considered a strong indicator of effective service.
You can also make use of the statistic below, with anything being under 5% is ideal.
5. Guest Sentiment
Guest Sentiment analyzes the overall tone of conversations and categorizes them as:
Positive
Neutral
Negative
This provides a broader view of how guests are feeling during their interactions with the AI and can help identify trends in guest experience and satisfaction over time.
6. Engagement Rate
Represents the percentage of users actively interacting with the chatbot. High engagement can indicate the chatbot’s ability to attract and retain user attention. You can measure this as a %age against the website traffic. Anything greater than 1% is good. You should also check how many conversations are outside your normal business hours, which will usually go unanswered.
The two tiles below will also be able to assist you in this regard by showing these statistics live.
7. Conversation Completion Rate
Percentage of conversations where users complete the intended task, such as booking a reservation, finding information, or completing an inquiry.
8. Guest Stage
Guest Stage identifies where the guest appears to be in their journey based on the context of their conversation.
Guests are categorized as:
Planning: The guest is researching, considering, or preparing for a future stay.
Staying: The guest appears to be currently staying at the property.
Past Guest: The guest appears to have already completed their stay.
Undetermined: There is not enough information available to confidently determine the guest's stage.
Guest Stage provides additional context around conversations and helps properties understand whether guests are researching a stay, currently on property, or engaging after their visit.
9. Website Referrals & Leads
Measure how much value that is generating. The chatbot can direct guests to relevant pages across your website and capture leads for areas such as:
Group bookings
Meetings and events
Reservations
Activities and experiences
Other property services
Monitoring these interactions can help demonstrate how the chatbot contributes to guest engagement and potential revenue opportunities.
10. Revenue Attribution
Revenue Attribution provides additional visibility into how AI conversations may contribute to booking and revenue-generating opportunities throughout the guest journey.
Impact on Room Booking Intent
Identifies conversations where the AI interaction may have influenced or strengthened a guest's intent to book a room.
This helps demonstrate how the AI supports guests as they research, compare options, and make booking decisions.
Booking Engine Redirects
Tracks conversations where the AI directs a guest to the property's booking engine.
This helps measure how effectively conversations move guests from researching or asking questions toward taking a booking-related action.
Where available, associated potential room nights may also be displayed.
Impact on Ancillary Revenue Uplift
Identifies conversations where the AI may have contributed to additional revenue opportunities beyond the room booking.
This may include guest interest in:
Dining
Spa treatments
Activities and experiences
Room upgrades
Transportation
Other property services
Together, these metrics provide a clearer narrative around how AI conversations may influence both room bookings and additional guest spending.
11. Average Conversation Length
Tracks the average time or number of messages exchanged per conversation. Shorter interactions may indicate efficiency, but very short conversations could also indicate unresolved issues.
12. Conversion Rate
Measures how many booking referrals the Chatbot can generate and the value of those referrals.
11. Training Data Utilization
An internal KPI measuring how effectively the chatbot utilizes training data to answer queries helps gauge the quality of its machine learning model. This is something that the Myma.ai team monitors closely.
Setting and tracking these KPIs helps you understand how well the chatbot is performing, where it can improve, and ultimately, how to provide better service.
Bringing the Insights Together
No single metric tells the complete story of chatbot performance.
By reviewing engagement, resolution, guest sentiment, Guest Stage, booking intent, referrals, and revenue attribution together, you can build a clearer picture of:
Who is interacting with the AI
Where guests are in their journey
What guests need assistance with
How effectively the AI is resolving their questions
Whether conversations are moving guests toward booking actions
Where AI may be contributing to room and ancillary revenue opportunities
Regularly reviewing these Insights can help identify opportunities to improve the guest experience, strengthen AI performance, and better understand the overall value the chatbot provides.