AI chatbots: Can frustration in customer service be avoided?

AI-supported chatbots are revolutionizing customer service: they enable companies to offer their customers fast, personalized support around the clock - and save millions in costs in the process.

Around the world, companies such as Klarna, Nykaa and Amtrak are reporting impressive efficiency gains and growing customer satisfaction.

However, as with every technological revolution, success depends on how consistently quality and a focus on people are placed at the center.

Those who actively tackle the challenges will benefit twice over - through satisfied customers and sustainable competitive advantages.

In the medium term, a gap between expectations and reality still needs to be bridged in many places, as hardly any chatbot simply works out-of-the-box .

In addition to technical tests, it is particularly importantto check the customer journey for friction points using user acceptance tests (UATs).

Many bots are still failing at their tasks.

What does this mean for business?

From time waster to customer chaser

Chatbots are supposed to save time, but often don't.

Below we will look at examples of successful chatbot implementations.

Currently, the data is still sobering:

• 72% feel chatbot use is a waste of time.
• 63% do not solve their queries.
• 50% end up in endless loops.
• 78% of inquiries ultimately require human agents.

Frustration is inevitable - user-centered quality assurance can help.

Bots lose context

One of the biggest problems with chatbots is their lack of "understanding" of human language and contextual blindness. The dynamics of the sector will provide a remedy. However, many chatbots still fail to grasp the nuances and meaning of an entire conversation.

Often, outdated NLP (Natural Language Processing) models without an adequate context window or a lack of conversational memory are the core problems, but not the only ones.

Other typical deficits:

• 35% of errors are caused by API errors in data delivery, meaning that even if the bot understands correctly, the information it delivers is incorrect because the connection to the data sources does not work.
• 27% of requests trigger generic "I don't understand" responses. This is the classic frustration: the bot doesn't know what to do, can't process the request and sometimes even disguises this - for example by explaining to the user how they could solve the task themselves.
• 19% are based on outdated product information. Imagine your chatbot gives your customers incorrect information about overdraft fees, for example.

Just because the technology used is best-of-breed doesn't mean it works as intended. These are just a few examples that show A human customer advisor would often be more effective.

An AI chatbot that cannot categorize the course of the conversation, does not learn and does not help is a point of friction.

If companies rely on AI chatbots, they must constantly invest in technology, data management, clear escalation paths and consistent maintenance of IT systems in order to truly replace good customer advisors.

Efficiency gains may therefore require greater complexity and better organization.

A key issue here is "escalation".

The escalation gap

When the bot gets stuck, 67% of customers expect a seamless transfer to human employees.

But the reality is different: 46% have to start conversations from the beginning and 54% find no option at all to get through to a live agent. 62% of users abandon interactions if the intention and context of the conversation are not understood. Users cannot pass on their concerns, "escalate" - they simply have no option to do so.

If the bot is a dead end instead of building a bridge to the human, all automation efforts come to nothing.

The short-term consequences of inadequate end-to-end customer journey tests as part of an acceptance test are being felt by a German neobroker, for example, particularly along the lines of the problem descriptions above. The neobroker relies on AI and external call centers instead of the previous internal customer service in order to supposedly save costs - read more at this link.

Whether it was nevertheless the right step into the future to rely on an AI-centered customer service system early on, or whether this has primarily led to a loss of customers and reputation, remains to be seen.

With our end-to-end customer journey tests, you can avoid such problems. Get in touch.

Why empathy is crucial in AI

It may sound paradoxical, but even in the world of AI-powered automation, empathy is a crucial factor.

• 38% of users are bothered by a lack of contextual understanding.
• 47% criticize the impersonal tone of voice.

The consequences are serious:

• 30% switch brands after negative experiences.
• 73% cancel transactions after bad bot experiences.

The potential damage is immense - and so is the risk.

Chatbot failure: further examples

The most extreme cases of chatbot failures show how dangerous it can be to neglect emotional intelligence and sound risk management during development.

Who hasn't seen examples like these?

• Airline chatbot gives incorrect information about special fares: Air Canada chatbot misinformed a customer about bereavement fares, leading to a lawsuit (Canada, 2024).

• Delivery service chatbot insults its own company: Adelivery service chatbot responded ironically to the question about the best delivery service with "We are the worst delivery service in the world" (e.g. Domino's Pizza Chatbot 2023).

• Bank chatbot gives illegal financial advice: Abank chatbot gave customers tips on tax evasion or illegal financial transactions (various documented cases, including in the UK).

• Health chatbot misdiagnoses: Amedical chatbot gave patients incorrect diagnoses or advised dangerous self-treatment instead of referring them to a doctor (e.g. Babylon Health, UK).

Comprehensive tests and safeguards against such misconduct are essential.

Hybrid models make economic sense

What's more, the apparent cost savings of pure chatbot solutions are often a fallacy.

A pure cost-benefit analysis shows that chatbots often reduce the resolution rate, impair customer loyalty and can even increase support costs:

We work with estimates and mean values based on the sources below.

Remember:

• 47% of customers switch brands after just two negative experiences.
• Poor bot performance leads to an up to 30% higher churn rate.

Don't harm yourself.

Comprehensive quality assurance is the be-all and end-all for chatbot implementations.

Solution: Quality-assured human-AI collaboration

At least in the medium term , the focusshould not be on more AI at any price, but on better human-AI collaboration and targeted testing.

We believe that hybridmodels that combine the speed of AI with the empathy of humans are the future:

• Real-time escalation paths with comprehensive context handover are essential.
• Sentiment analysis for the early detection of frustration makes it possible to offer human support at an early stage.

Promising technologies includeBERT models (Bidirectional Encoder Representations from Transformers), which can reduce the above-mentioned errors by up to 40%, and RAG architectures (Retrieval Augmented Generation), which improve the timeliness of information.

Here we see how targeted software testing and model validation in the area of AI development, as well as testing along the customer journey, are crucial to making these technologies robust for practical use. If you avoid friction points and really help the customer, they will stay with you.

Dispensing with the human component means more effort in quality assurance.

Technical tests and user acceptance tests (UATs) are necessary for a smooth customer journey.

Many chatbot success stories

Depending on the business model, the transformation of customer service to AI augmentation may be more difficult or easier. There are already huge opportunities in B2C in particular.

Some success stories:

• The Australian insurer nib Health saved 22 million US dollarsby AI-optimizing its chatbot Nibby and at the same time increased itscustomer satisfaction (CSAT) by a whopping 15% - more at this link.

• Swedish payment provider Klarna officially announced in February 2024 that its AI assistant handles two thirds of all customer service chats, which is equivalent to 700 full-time agents. Customer satisfaction is at the same level as with human agents. The average processing time has fallen from 11 to under 2 minutes. The use of AI saves around 40 million US dollars a year. Read more here.

• Autodesk, a US software company, implemented the virtual agent "AVA", which reduced the resolution time for customer inquiries by 99% - from hours or days to just a few minutes. AVA handles tens of thousands of requests per month, increasing customer satisfaction and reducing the cost per resolved case by 90%. You can find more information here.

• Nykaa, a large Indian beauty e-commerce company, automated its customer service with the help of Verloop.io, saving over 32,000 man-hours per month. Over 90% of users rated the AI chatbot as "very positive" or "excellent". The AI processed around 1.6 million individual conversations in the first 30 days. Details can be found here.

• Amtrak, the US rail operator, saved around 1 million US dollars in customer service costs in the first year thanks to the chatbot "Julie". Bookings via the chatbot increased by around 25% and user satisfaction with certain types of request rose by 50%. Julie handles over 5 million customer inquiries a year and is continuously improving the efficiency of customer service. Read more here.

Further success stories are expected in the coming months.

We will be happy to help you implement your AI chatbot solution in line with the market and in a timely manner.

The path to accepted automation leads via quality assurance

The vision of fully automated customer interaction is ambitious.

Certain factors will determine whether this vision becomes a success:

Human-centeredness and quality assurance are success factors

At Testsolutions, we are convinced that companies that prioritize hybrid models, transparent escalation options and continuous training and testing, at least in the medium term, can increase the acceptance of their AI systems and realize the promised added value for customers and companies alike.

As long as chatbots do not advance customers, they will remain potential cost drivers rather than cost reducers.

After an already complex implementation with technical and organizational changes that cannot be easily reversed, this also leads to higher costs in the long term.

The solution in the medium term is not just more AI, but quality-assured human-AI collaboration.

Recognize the weak points of your chatbot before your customers do and develop the best possible solution.

As specialized software testers, we uncover risks and optimize the performance of your bots for real customer satisfaction and efficiency. Request your free initial consultation now !

Source selection: Market volume of chatbots, Chatbots as a labor facilitator, The AI Chatbot Crisis: When Good Intentions Meet Poor Implementation,DPD customer service chatbot swears and calls company 'worst delivery firm',Sorry, I Don't Understand' - Top AI Chatbot Fails and How to Prevent Them , Gartner on the Role of Chatbots in Customer Services Experience, AI Customer Service Statistics in 2025,When AI Chatbots damage loyalty: A lesson in human-centered design, Top 150+ Customer Service Statistics & Trends [2025],Consumers frustrated by inability to switch from self-service to live agent, survey finds,December 2024 - AI Misinformation Monitor of Leading AI Chatbots, BEST Chatbot Statistics for 2025 | Master of Code Global, Fink, The little bot tastes best - the simpler, the quick win, One Negative Chatbot Experience Drives Away 30% Of Customers, What Consumers Think About AI Customer Service Gone Wrong, Dark Side of Chatbots, Top Customer Service Chatbots, 12 chatbot case studies, Amtrak's chatbot, IBM Watson and Autodesk reinvent customer service, Hidden AI Use Case, MIT Error-riddled data sets are warping our sense of how good AI really is

We work with estimates and averages based on the above and internal sources.