Why AI projects in the hotel industry based on ChatGPT are failures and will always remain failures.
Introduction
The thunderous arrival of ChatGPT has spread the fantasy of an already omniscient Artificial Intelligence, connected to a website, to automatically generate a virtual assistant capable of providing customer support. But this truly is a fantasy, which leads to a long list of issues.
Generative AI potentially has the answers to solve these issues, but these answers are often approximate, even false. When it’s your hotel’s virtual assistant which is communicating false information, you can imagine the consequences.
Generative AI is indeed a major factor and a big step forward. However, over a year after its appearance, the failures of all hotel projects based purely on generative AI show clearly that everything is not that simple.
Chapter 1. Intelligence vs. Knowledge
Intelligence is the ability to understand, resonate, solve a problem and adapt to new situations. Knowledge is a collection of information, ideas and skills acquired through experience or education.
In summary, knowledge and intelligence complement each other and work together to understand and navigate in an ever-changing world.
In a virtual assistant project, AI is certainly important, but the bulk of the work lies in managing the knowledge acquisition process and managing the data. Undoubtedly it has to be both knowledge and intelligence combined, yet strangely very few seem to realise this!
It is frightening to see so many IT managers convinced that they just need to put the data from the website and possibly their SalesForce knowledge base, together into a big pot and stir well, and that’s it ?!
No. Unfortunately, that doesn’t work. To convince yourself of this, all you need to do is ask ChatGPT questions on a subject that you understand well, to realize that the answers provided are regularly approximate, and often completely false. This is called hallucination. Sadly, hallucination can be convincing.
Indeed ChatGPT, like the sophists (or gaslighters), can produce convincing answers without aiming for the truth. The answers are based on statistical models with no real understanding of the truth or the implications. Additionally, just like sophistical rhetoric, these responses can influence actions, even if they are not based on factual evidence or a deep understanding of the topic.
- An example of sophism: My cat is a mammal, my dog is a mammal therefore my cat is a dog.
- An example of hallucination: The twin room is 25m2, the double room is 25m2 so the two rooms are identical.
This phenomenon is particularly dangerous because the poorer the database, the stronger the hallucination phenomenon will be. In the case of hotels, these databases rarely exceed 500 information points when in reality, customers are already asking for more than 2,500 and their thirst for information is constantly growing!
And what will happen when a guest has checked with the hotel chatbot and shows up at the hotel reception, in a wheelchair, even though the hotel is not equipped to accommodate this guest? Who will be deemed responsible for the hallucination? The hotel, the development team, the data managers, ChatGPT? Tell your children to become lawyers, there will be enough legal work for many years to come.
Exemple: Air Canada must honor refund policy invented by airline’s chatbot.
Chapter 2. I have no opinion, I only have data
A. The knowledge base
The quality of the data, its structure, order and formatting are as important as their relevance in determining the quality of the results. Business logic is also a key to success.
- To meet customer expectations, a hotel must be able to display at least 2,500 points of information (points which go far beyond the price of breakfast, the size of the room, the check-in time, etc).
- Between 60 and 70% of these points (i.e. around 1500) are absolutely not formalized (such as: the dress code of the restaurant, the presence of Wifi at the beach, the depth of the swimming pool, the type of electrical outlet, where to find the beach towels, etc). Clearly, the information is only present in the heads of staff members. When you think about the turnover of staff in the hotel industry, it’s scary.
- A hotel website has a maximum of 300 information points. The OTAs (Booking, Expedia) have only a few more (For this reason, hoteliers should focus more on Google and Less on OTAs).
Given these figures, how can we explain how ChatGPT (or an equivalent), would be able to answer a question on a topic which it has not the faintest knowledge about?
Populating an existing database like Q-Data from Quicktext AI solutions and setting up the processes to maintain it, is already a challenge for most hotel IT and marketing departments. But it is possible! Building a knowledge base like Q-Data, from scratch, when you are a hotel group, however large the group may be, is totally impossible.
So the challenge is much more the improvement of knowledge, than that of intelligence. It is for this reason that Velma from Quicktext, is globally recognized as the most efficient AI in the hotel industry, not only through Velma’s cognitive capabilities specialized in the hotel industry, but also thanks to the deepest structured database on the market (more than 1900 points and counting).
B. The prompt, in other words what your customers are asking for
How and on what basis, do you improve your knowledge base?
This is where human intelligence, combined with data, will allow us to determine which elements of knowledge to collect.
Good news! Since Velma has existed at Quicktext (the hospitality AI SuperApp), we have accumulated more than 30 million questions and queries on every type of hotel, location and season, from customers all over the world.
This knowledge allows us to anticipate these questions, in other words, the prompts.
C. AI modeling
Model tuning and behavior are much more important than you think. It is necessary to, at the same time:
- Anticipate prompts (customer questions),
- Anticipate the answers (the knowledge base),
- Manage connectivity (reservations engine, CRM, call center, task management software, etc.),
- And finally, master the AI which will ‘grind’ it all together (As you can see from the list above, this last part is only one component of the equation.)
ChatGPT might seem like the obvious solution, because it’s quick and easy (7 hours to set up), but unfortunately laziness comes at a price: far too many errors. Here are the causes…
- Missing content: it is impossible to respond without the available data (the famous 2500 potential information points).
- Ranking: The response is present, but not classified or qualified enough to be included in the returned results.
- Out of Context: the data is extracted from the database, but cannot be included in the generated response due to too much information.
- Non-extracted data: the model fails to extract the correct information due to excessive data volume or contradictory information.
- Wrong format: the question involves extracting information in a specific format (photo, button, table, carousel) that the model ignores.
- Incorrect specificity: response that is too specific or not specific enough, due to lack of clarity in the wording.
- Incomplete: exact answers, but with missing information, even if this is present in the database.
But it’s not over, we will also have to manage:
- Scalability: as the volume of data increases, maintaining efficient and rapid indexing quickly becomes an exponential task.
- Updates: especially where documents are frequently added, modified or deleted, ensuring these updates are done, without compromising system performance, is a daily challenge.
- Response times: 10 seconds is a long time when you wait.
- Not to mention the connection to business softwares.
D. The facts and figures
The Quicktext solution therefore consists of opting for a hybrid method: classic AI, combined with generative AI, both specific to the hotel industry.
A little math to really understand.
- ChatGPT in the hotel industry generates 30% hallucinations (this is the minimum observed)
- 80% of customer requests are simple and do not need generative AI. (Example: What time is breakfast?)
In the case of hybrid AI (like Q-Brain+) 80% classic, 20% generative, only 20% of requests will potentially be subject to hallucinations. With mastery of your AI (having your own LLM), this level of hallucination drops to less than 10%.
So the hallucinatory volume is 10% x 20% = 2%.
In summary with a generative model, hallucinations will be around 30% whereas with a hybrid model 2%.
Chapter 3. Let us not forget anything
To summarize, creating a virtual assistant based on ChatGPT or equivalent without having the other necessary elements amounts to being condemned to:
- Not mastering prompts (questions),
- Not mastering the black box (generative AI),
- Not mastering the knowledge base (the answers).
How can we believe that such an assortment of approximations can generate a relevant system?
Good news! you’ve just hit the jackpot by answering that question.
And in case that wasn’t enough!
How is it possible to solve the following issues with pure generative AI?
- The notion of multi-property and multi-criteria search (we are very curious to see that)
- Updating data (especially if it is not structured)
- Cultural adaptation (and this is definitely not a simple translation)
- Connectivity to the APIs of Booking engines, PMS, CRM, task management
- Management of notifications when human interaction is necessary
- Recovery of conversational data
- GDPR, CCPA
- Etc
Chapter 4. Double hybridization
You now understand why hotel chains and “experts” who have decided to go exclusively down the path of generative AI for the hotel industry are experiencing disappointment after disappointment. The few, those still refusing to admit it, are those who have decided not to measure the results. No thermometer, no fever.
The solution therefore lies in double hybridization.
- In intelligence: classic + generative
- In knowledge: structured database + external data sources (but in moderation).
This is a huge task. At Quicktext, we know just how immense this task is, as we have been working on it for 7 years.
And always remember, the best way to answer a question is no-one needing to ask that question.
Meditate on this, it will be the subject of a future article.
