Skip to main content

Validating and Troubleshooting AI Responses

How to Troubleshoot and Identify the Source of Information

T
Written by Team at Myma.ai

View Published Content

The AI learns from content that comes from your website, uploaded documents, FAQs, room descriptions, and other sources within the Knowledge Hub.

To see everything that’s currently published, navigate to the below:

On this page you will be able to view all the information that is published at that time. Which the chatbot currently uses to answer guest questions.

Reviewing this page can be very helpful for verifying the content that is live and ensuring that your chatbot provides accurate and up-to-date information.

Each of the minitabs in this section will correspond to where the information is stored within the Knowledge Hub so you can locate and maintain it as neccessary.

For example, when verifying pool related contents:

In this section, you could see that it's within the Rooms section so for example if you were to find a mistakes in this section would then navigate to the below for any adjustments.

Special Case: External Sources

If External Sources is set up for your property, the way the information is stored is via the Document sections. The data is first pulled from the website that was set up in the External Sources section and then reupload as a document under the Document section.

  • External Sources view:

  • Documents view: You could see that the source of the document is Web Page

Hence, when you search for a key word under View Published Content and if it shows up under the document tab with the line that says:

It means that the content was pulled from https://www.[DEMORESORT].com/

To confirm what contents could be pulled from the setup website, right click the page that was used to set up External Sources and select View page source

This will open a new tab and whatever is visible on this tab will also be available to the chatbot. This allow you to confirm whether or not the chatbot was able to pull the necessary data as not all data on the normal view of the webpage that is visible to you can be visible to the chatbot. This could be due to security settings or the way the contents on the pages are presented.

As the data on the View page source page is unconventional; hence, in order to confirm if a piece of information is on this page you would need to utilise Ctrl + F (on Window) or Cmd + F (on Mac) to search for the keyword.

For example:

When searching for "knowledge hub"

  • Example Trouble-shooting Flow for External Sources:

  1. You search for the keyword that is related to the question being asked or the incorrect answer in View Published Content and finnd that it originates from the Documents section with at the top of the location where the information is located:

  2. You visit https://www.[DEMORESORT].com/ to identify from where on the website the chatbot get the incorrect information from via View page source (from rightclick menu).

  3. You search for the keyword on the new tab that is opened and locate the location of the information.

  4. You rectify the information on the webpage and resync External Sources.

  5. Retest after the resync is successfully published using incognito window.

Conversation Logs

Sometimes, there is a need to investigate the AI response to determine the source of the response. This can help determine if the source of the response is coming from an unpublished page, an outdated FAQ, or content hidden within a document, making it difficult to trace the exact source manually.

For every response, you can view the source in Unified Inbox. Locate the conversation, look under the Unique Chat ID, and select the arrow icon to view where the information originated.

After selecting this, a window will pop up showing every query that was asked, along with all the information the chatbot used to generate its response. From here, you can search for specific details or missing information by pressing Command + F on Mac or Ctrl + F on Windows.

Once you’ve found it, you can locate where the information is stored and update or remove it as needed.

Correcting the Information

Once you’ve located the source of the incorrect content, you can take the appropriate action:

  • Edit the text within the knowledge hub or connected document to update it.

  • Remove the outdated information if it is no longer relevant.

  • Unpublish or disconnect old sources that the chatbot should no longer reference.

After making the necessary adjustments, perform a quick test chat to confirm that the AI is now responding accurately.

Summary

This new feature is designed to make troubleshooting inaccurate chatbot responses simpler and more transparent. By using the Unique Chat ID and arrow icon within the Unified Inbox, you can easily trace responses back to their original sources, verify accuracy, and make corrections within minutes instead of hours.

Did this answer your question?