Why use AI in support:

The rapid development of generative artificial intelligence, such as ChatGPT, has profoundly transformed the field of customer support. But can AI really replace a human agent?

At this stage, there seems to be a consensus that the technical question has largely been addressed, but the solutions available on the market very often remain incomplete. They all recognise customer satisfaction as a major issue, but very few actually contribute to it, preferring instead to optimise support costs. This measure, although seemingly simple, is extremely complex to evaluate: it is generally based on a sample that is not very representative of the overall population, calculated using heterogeneous methods. The absence of an absolute definition of this metric means that anyone can manipulate it in order to validate any approach.

End-users want human interaction:

The notion of customer satisfaction is a crucial issue in the field of customer service, and the various data show how complex it is to understand consumer preferences. Let's imagine that our objective is to invalidate a Chatbot approach, but with a simple multiplication we achieve our goal thanks to our famous customer value.

If we take the example of Toyota's bot, which according to COMETE.AI satisfies 98% of its users, it seems to be a success. However, a PwG survey reveals that 80% of respondents would prefer to interact directly with a human customer service agent rather than a chatbot or other automated response service. These results are in line with those of Forbes, which reports that 86% of consumers prefer human interaction to chatbots. 

So we can deduce that the 98% user satisfaction rate only applies to 20% of the overall population (the other 80% of the population were unable to really express their preferences). The result is a customer satisfaction rate of 19.6% - it is legitimate to question the effectiveness of this technology as a satisfactory compromise. 

AI in support

AI in support is primarily used through chatbots:

The vast majority, if not all, of the solutions available follow the ChatBot model. However, Chats represent a very small proportion of the volume of tickets handled by our agents, with over 70% of requests made by email.

ChatBots using AI represent an ingenious solution for getting round most of the problems associated with processing tickets. Artificial intelligence is content to provide partial answers, prompting the end-user at every point of uncertainty to think things through. These ChatBots are nothing more than intelligent, dynamic FAQs, capable of responding satisfactorily only to requests for simple information, which represent a small percentage of standard customer service traffic. 

In the context of emails, the AI only has one chance to provide the right answer, and must only solicit the user when it is really necessary.

AI is still relatively rare as it's a real challenge to automate:

Installing artificial intelligence is a major challenge for most companies. Only a few large accounts have the resources to tackle this problem head on. The data work required to automate this process is crucial, but is often hampered by the complexity of data recovery and strict compliance with data protection regulations (GRPD). Implementing such a solution is far from straightforward, and usually requires a substantial budget and technological restructuring, which often leads to this service being outsourced. The main difficulty lies in the scalability of the tool: automating processes in a closed data environment is relatively simple; however, processing tickets from a multitude of customers with varied processes, tools and data structures is a far more complex task. This is where the real challenge of BPO (Business Process Outsourcing) lies. This is the context in which Onepilot's ambition lies: to make artificial intelligence available to as many people as possible for the efficient processing of all tickets.

AI at Onepilot

Our vision:

At Onepilot, we are convinced that human intervention remains essential to guarantee genuine customer satisfaction. AI, while a valuable tool, cannot be considered a complete solution in itself.

Our approach is to optimise the work of our freelancers rather than replace them. In effect, our AI acts backstage, 'whispering' to our agents the right answers at the right time. However, it is still under human control that the response is formulated for the end user. 

For our customers, this means total peace of mind: they entrust us with their requests and we take care of the rest, from modelling the processes to implementing the appropriate artificial intelligence models. Our main concern is productivity, but always with the customer in mind. We carefully monitor the time taken to process tickets, which enables us to adopt dynamic and fair pricing. The shorter the time taken to resolve a problem, the lower the final cost to the customer. In this way, any value created internally is passed on to our services, offering our customers an exceptional level of quality at a competitive price.

Some figures:

A typical month at Onepilot is characterised by a volume of more than 500,000 emails to be processed, requiring the intervention of more than 1,200 agents. The introduction of Autopilot, with its message recommendations, has proved highly promising since its launch in November 2023. This new tool enabled us to cut the processing time for half the tickets on which it was deployed by a factor of 6, from 6 minutes to just 1 minute on average. On the strength of these results, we are now embarking on the expansion phase across all our customers. By the end of the year, our aim is to manage 50% of e-mail tickets via Autopilot. This transition will result in a doubling of our ticket processing capacity, while guaranteeing unchanged end-user satisfaction, without any action being required on the customer's side. 

Unlocking the fair use of AI: Our benefits

The deployment of our core AI algorithm "Autopilot" represents a major step forward in solving the problems associated with ticket processing. It enables us to automatically interpret our customers' knowledge bases and integrates perfectly with the main ticketing tools. It also enables us to retrieve essential contextual information such as order status or delivery times. This solution is based on several million messages carefully annotated by our agents, as well as hundreds of knowledge bases that are maintained and regularly updated, meticulously describing all our customers' processes using a proprietary data structure adapted to AI.

Graph 1: 70% of Onepilot tickets are e-mails, while over 56% of Onepilot tickets require interaction with an external tool (most often a back office or carrier).

We have developed a panel of AI tools to improve the daily lives of our agents while guaranteeing a high level of quality:

As soon as a ticket is received, Autopilot automatically scans the knowledge base associated with the customer so that the agent is in the best possible position to respond. If the algorithm can resolve all the necessary steps, it suggests a response created manually by one of our Account Managers, also offering an alternative generated by LLM to increase personalisation. If the algorithm does not have enough information to complete the process, but has managed to process certain steps, the agent is directly directed to the first step not processed automatically. In addition, for each step skipped, the agent receives a summary justifying the algorithm's choices. 

As we are concerned about the protection of personal data, we have put in place rigorous measures such as the real-time anonymisation of messages via very precise named entity detection. The data, which no longer contains any personal information, can therefore be saved indefinitely and sent to the LLMs of our choice. To guarantee the quality of our agents' work, another specialised AI algorithm analyses all the messages sent and selects those it considers to be of insufficient quality. Each week, this algorithm forwards up to 10% of the messages to our team, who manually check these tickets and annotate their quality. This tool is used to guarantee a high level of customer satisfaction and will be used in the medium term to control Autopilot.

A little more detail on our core algorithm: Autopilot

Our approach is based on the combined use of large-scale language models and classic machine learning techniques. To achieve this, we have set up a sequence of models that are re-trained on a weekly basis, incorporating the latest tickets available. 

Thanks to a data augmentation algorithm, we are able to generate synthetic data, thus enriching our learning set, even with relatively modest samples. Just three points can be enough for our algorithm to understand a process. This approach has a number of advantages, including the ability to integrate new customers quickly and to take account of changes in customer processes in a responsive way. Each model is specialised in a precise task and limited in its domain to guarantee its relevance. We combine LLMs, RAG (Retrieval-Augmented Generation) and supervised classification to produce high-performance models. At each node of the knowledge tree representing the customer process, several models are associated and selected by a voting mechanism. Thus, for each customer, several dozen models are trained automatically, according to architectures and constraints defined as part of our proof of concept.

We measure the relevance of our algorithm via the % of suggestions used by our agents, and during the POC phase we reached 85% of suggestions that were the right ones. This measure contrasts with customer satisfaction in that it is absolute and verifiable.

Graph 2: With an average accuracy of 85% and despite a limited scope of 2% of predicted tickets, our "Autopilot" proof of concept has already saved our agents more than 3,500 hours. Our ambition is to multiply this figure by 25 as we scale up the tool.

Conclusion:

The integration of Autopilot into Onepilot has been a veritable revolution in the field of ticket processing. The impressive results achieved from the outset of this initiative bear witness to its undeniable effectiveness. Building on this momentum, we aim to redefine the standards of ticket management for our customers, offering them an even smoother and more efficient experience. With Autopilot now available in all languages, we are ready to take on any challenge, while maintaining our absolute commitment to end-user satisfaction.