Large language model–driven artificial intelligence (AI) chatbots burst into prominence in recent weeks, capturing enterprise leaders’ attention across various industries. One such chatbot, ChatGPT, made especially notable waves in the tech world, garnering over 1 million users within a week of its launch. 

ChatGPT and other turbo-charged models and bots are set to play a crucial role in customer interactions in the coming years, according to Juniper Research. A recent report from the analyst firm predicts that AI-powered chatbots will handle up to 70% of customer conversations by the end of 2023. 

>>Follow VentureBeat’s ongoing ChatGPT coverage<<

This highlights the growing reliance on AI to enhance customer experience (CX) and streamline interactions. With chatbots becoming increasingly human-like in their conversations, there are numerous opportunities for businesses to use this technology to improve marketing strategies, deliver personalized services and generally drive efficiencies.

While speech recognition and natural language processing (NLP) have a long history in customer management and call center automation, the new large language model (LLM)-driven chatbots could significantly change the future of CX, according to veterans in the field.

“LLMs are fundamentally changing the way search algorithms work,” Sean Mullaney, CTO of search engine SaaS platform Algolia, told VentureBeat. Traditional search engines match individual words from a query with the words in a large index of content, he said, but LLMs effectively understand the meaning of words, and can retrieve more relevant content.

With the advent of LLM-based chatbots and virtual assistants, customers can now interact with businesses in a more natural and conversational manner. This has been a significant step forward in providing a better CX throughout the customer journey. As a result, LLMs have become a go-to solution for companies looking to enhance their customer support, sales and marketing efforts.

But implementing the new bots will not be without challenges. Success is not a given, as first-gen chatbots have already shown.

Despite their versatility, many first-gen chatbots struggle to understand complex requests or questions and are limited in maintaining context throughout an interaction. This has resulted at times in a stilted or rigid customer experience, as the chatbots are often restricted to a limited set of interactions. In many cases, interactions are ultimately routed to a human.

A recent survey conducted by AI company Conversica shows that first-gen chatbots experienced by users are not living up to customer expectations. The firm said four out of five buyers abandon the chat experience if the answers don’t address their unique needs. 

“First-gen chatbots rely on predetermined scripts that are tedious to program and even harder to maintain,” said Jim Kaskade, CEO of Conversica. “In addition, they don’t understand simple questions, and limit users to responses posed as prewritten messages.” Enterprise-ready, AI-equipped applications with LLMs like GPT can make a difference, he continued.

ChatGPT alters conversational AI landscape

By incorporating different conversational styles and content tones, LLMs inspired by ChatGPT can give businesses the ability to present their content more engagingly to their customers. LLMs can also learn and adapt based on customer interactions, continuously improving the quality of their responses and overall CX.

Dan O’Connell, chief strategy officer at AI-powered customer intelligence platform Dialpad, believes that LLM-based chatbots such as ChatGPT can serve as editing/suggestion tools for agents in terms of helping them better engage directly with customers. They “can be used in a variety of ways to save time and append records, but to also effectively identify topics, action items, and map sentiment,” O’Connell told VentureBeat. 

Hi, I’m ChatGPT. Ask me anything!

Traditional chatbots allow interaction in a seemingly intelligent conversational manner, while the GPT-3’s NLP architecture produces an output that makes it seem like it “understands” the question, content and context. However, the current version of ChatGPT also has its drawbacks, such as generating potentially false information and even politically incorrect responses. The OpenAI team has even advised against relying on ChatGPT for factual queries.

Even ChatGPT’s creators admit to limitations on its usefulness, as seen in OpenAI leader Sam Altman’s Twitter postings.

“The problem with models like ChatGPT is that ChatGPT ‘memorized’ everything it could find on the internet into only 175 billion numbers (5,000 times fewer than the human brain). So ChatGPT is never 100% sure of the answers it gives you,” said Pieter Buteneers, director of cloud communications platform Sinch Labs. “It is impossible to remember every minute detail, especially if we’re talking about storing all the knowledge on the internet. So in every situation, it will just blurt out the first thing that comes to mind.”

Despite its drawbacks, upstart ChatGPT has one major advantage over other chatbots: it excels at understanding user intent, maintaining context and remaining highly interactive throughout the conversation. In addition, ChatGPT’s potential for NLP and ability to efficiently respond to queries have made enterprises rethink their current chatbot architectures aimed at enhancing CX.

Jonathan Rosenberg, CTO and head of AI at contact center platform provider Five9, said utilizing AI algorithms such as zero-shot learning — as ChatGPT did — will be the key to developing LLMs with exceptional capabilities. Zero-shot learning is an instance where a machine learning model is confronted with input that was not covered during machine training.

“What makes GPT-3 different is that it became big enough to do things its predecessors could not — which is to generate coherent output to any question, without being explicitly trained on it,” Rosenberg told VentureBeat. “It’s not that something is radically different with the design of GPT-3 compared to its predecessors. Instead, zero-shot learning wasn’t accurate enough until the model size exceeded a certain threshold, at which point it just started working much better.”

“Models like ChatGPT will not be able to replace everything companies do within the contact center with traditional conversational AI,” said Kurt Muehmel, everyday AI strategic advisor at AI-powered analytics platform Dataiku. “Companies that deploy them need to build processes to ensure that there is a steady review of the responses by human experts and to appropriately test and maintain the systems to ensure that their performance does not degrade over time.”

However, businesses must view chatbots and LLMs like GPT not as mere gimmicks but as valuable tools for performing specific tasks. Organizations must identify and implement use cases that deliver tangible benefits to the business to maximize their impact. By doing so, these AI technologies can play a transformative role in streamlining operations and driving success.

“Where the opportunities with ChatGPT lie is that this technology can understand more emotional nuance within the text. This won’t entirely replace what companies are doing within the contact center because the human element still needs to play a critical role,” said Yaron Gueta, CTO of Glassbox. “Where it will have the most benefit is companies will be able to have far less call deflection between the chat channel and call center, as ChatGPT can make the end-user experience better within chat interactions.” 

Tuning and maintaining conversational AI models

The versatility of conversational models like GPT is demonstrated in a wide range of potential applications, including computer vision, software engineering, and scientific research and development.

“The challenging part is fine-tuning the models to solve specific customer problems, such as in ecommerce or customer support where the answers are unavailable from the base training. In addition, these use cases need proprietary company data to fine-tune them to meet domain-specific use cases like product catalogs or help center articles,” said Algolia’s Mullaney. 

Likewise, Yori Lavi, cloud expert at data analytics platform Sqream, suggests that it is vital to remember that the training, testing and ongoing monitoring are critical. Importantly, he said, models like GPT often need to be made aware of the value/risk of its answers. 

“High-risk decisions made by chatbots should always be verified/assessed. Therefore, to enhance your CX, companies should work on creating chatbots that can find answers to complex needs and build on previous questions/context to fine-tune their results,” said Lavi. 

Leveraging advanced LLMs for better CX 

Deanna Ballew, SVP of product, DXP at digital experience platform maker Acquia, believes that advanced LLMs like ChatGPT will become a dataset and capability of conversational AI, while other technologies will advance ChatGPT to train on. 

“We will see much experimentation in 2023 and new products emerging to add business value to ChatGPT. This will also extend into how support agents respond to consumers, either using automated bots or quickly getting an answer by leveraging ChatGPT on their own dataset,” said Ballew. 

Likewise, Danielle Dafni, CEO of generative AI startup Peech, says the increasing use of these models in customer service and support means companies will need to continue to invest in developing more sophisticated chatbots, leading to improved CX. There is a payoff, however.

“Companies that adopt these models to improve their existing chatbot’s ability to recognize and respond to emotions in interactions and other capabilities will be well-positioned to provide improved customer support and experience,” Dafni told VentureBeat. 

“ChatGPT and traditional LLM chatbots will continue to advance and become more sophisticated in their ability to understand and respond to customer interactions. With wider public awareness, more customers will expect the GPT-level of conversation ability from chat functions, leaving first-gen scripted bots in the dust,” predicts Conversica’s Kaskade. 

He said the current developments are just the tipping point for adopting web chat solutions with generative AI abilities. He predicts these will be ubiquitous across B2B and B2C in the next three years.