Generative AI - Limits and Opportunities in Chatbots

As a business, how can one manage AI mistakes in a profitable manner? AIs in general, and (large) language models in particular will always make “mistakes”, just as humans make mistakes. This is a mathematical fact, since the whole underlying structure of any AI model out there is based on probabilities.

The above question is actually too general, considering the wide variety of AI applications out there. For the sake of this article we are going to focus on customer support chatbots, focusing on the high level business logic that underlie them. We are going to look at the following questions:

1. Would you use a chatbot that is only 80% right? What about an engineer that delivers results only 80% of the time?

2. How about a customer support officer that can help only in 80% of cases?

Running a business is all about dealing with imperfections. Failures and mistakes are part of every operation, regardless of its nature, from software development to selling chewing gum. That is why any successful business tends to focus on the optimization or the continuous optimization of its processes, within certain cost parameters. Such optimization is driven by the business ability to manage the knowledge it has and acquires throughout its activities. For example, its ability to sell and grow revenues will depend both on the existing sales capabilities of its team, but also on its ability to process the knowledge it gains through the ongoing sales processes and use such knowledge to grow the existing capabilities. The same goes for other activities too, such as customer support.

This is where language models come into play, as they are perfectly suited for analyzing existing information (knowledge). Better still, in the case of large language models, their generative capabilities make them perfectly suited for live deployment within daily business activities - hence the launch of Clipport service as an AI Concierge. And while such models will make mistakes, they do present clear business advantages for deployment:

1. They allow any business to maintain and augment its know-how - no more worries about losing experience and knowledge because of (high) employee turnover.

2. Fully trainable and able to improve continuously. The more specific knowledge is accumulated, the better such models become when fine-tuned and redeployed, leading to even lower costs - that’s because a larger amount of information for training may require smaller models for finetuning, which require lower hardware infrastructure costs.

3. Fully scalable, both upwards and downwards, leading to a very flexible cost structure.

4. Accelerate employee onboarding and performance improvement. We remain firm convinced that people remain by far the best asset of any company out there, but not every employee is an asset. Best people in any industry focus on growing, on learning more, on performance and results, even when there is nobody offering them such support. Large language models allow a business to grow such employees’ capabilities by a few orders of magnitude.

The reality is that perfection sells - in every corner of our society, yet it is dealing with imperfections that drives the best results and profitability for any business. We have finally launched the Clipport service this month, an AI Concierge focusing especially on the hospitality industry, but we discovered something very interesting over the last three years while trying to build its brains. Its capacity for improving is not driven by being the best AI models, but by being the best at trying to capture user input and improve on it.

We have also realized that the limits of using AI in any business are driven by people’s understanding of what they need - hence “prompt engineering” becoming a new hot area on social networks. However, it is such prompt engineering, used programmatically or directly or on the web, that is going to decide the limitations of a business, especially when combined with access to goal specific information. For example one could easily use large language models to write generic emails, blog posts and other content - yet without specific input or requests put into such language models, one is only going to get generic results, which don’t mean much in today’s hypercompetitive world. 

It is the continuous creation of new data that will always continue to improve the capabilities even of the large language models. No matter how large the model, at the end of the day it is trained on yesterday's data, dealing with yesterday’s thinking. Tomorrow will have many unknowns, hence the need for continuous training and fine-tuning. As an example, sales people know this best, because no matter what their last sale was, the next opportunity will be different.

Once we start fine tuning our models though, we can start talking about opportunities - and the more data available for fine-tuning, the stronger the opportunities available, such as:

• Lower tech infrastructure costs. This is because the less available data for fine-tuning, the larger the initial language model needs to be, in order to be able to deal with natural language in a real life scenario out of the box.

• Higher quality interactions. Even ChatGPT needed to actually be fine tuned on specially curated text. The more specific a situation, the more training/fine-tuning is required for a model.

• Never forgetting - knowledge is never lost. This can be priceless for many businesses, which have to deal all the time with employee turnover, especially in the case of longer term employees, or performant ones.

• Better business decisions. Higher quality interactions lead to better data, which can be used for better analytics, which eventually can lead to better decision making.  

Understanding how to use generative AI for business remains a massive opportunity, comparable maybe to the industrial revolution, yet a lot stronger. While the AI will make mistakes or miss some information parts, its ability to consistently track such mistakes and interactions more than make up for the mistakes, because that’s how the AI improves itself - just like a person, but continuously and without a break.