World Aviation Festival 2026: In Dialogue with the Airline Industry

TestSolutions participated in the 2026 World Aviation Festival in Lisbon with its own booth and a technical presentation. The focus was on professional dialogue with the airline industry and the question of how software quality can be ensured in an increasingly AI-driven airline IT environment.

A Quality Assurance Specialist at an Airline Tech Trade Show

A trade show dedicated almost exclusively to technology for airlines is an unusual setting for a software testing service provider. That’s exactly what made it so appealing to us. On the one hand, we wanted to engage in technical discussions with airlines and technology providers; on the other hand, we wanted to demonstrate that quality assurance is a central component of modern airline systems. Our booth was Booth 1-230.

Our Topics

Our presentation centered on five key areas that reflect the breadth of our offerings:

• Testing Customer-Facing AI: How AI-powered customer channels can be systematically tested once responses are no longer predictable.
• Test Factory: An operational model in which we provide ongoing and scalable quality assurance for airline clients.
• Test Automation with AI: How AI can support the creation, maintenance, and evaluation of automated tests.
• Airline Retailing: A topic that is currently a major focus for the industry and places high demands on offering and booking processes.
• Test Data for GDS Systems: A solution that supports the generation and management of test data for Global Distribution Systems, particularly in the context of PNR.

The Technical Presentation: Testing AI Chatbots

In addition, we gave a roughly fifteen-minute presentation demonstrating how a customer-facing AI chatbot can be systematically tested. The presentation was titled “Testing Customer-Facing AI Chatbots with Golden Data Sets and LLM-as-a-Judge” and was delivered by Anupam Krishnamurthy, Head of AI Testing, and Balamurali Boggu, Airline Domain Expert.

The presentation began with a concrete example from everyday airline operations: At 3 a.m., a passenger messages his airline via chat, stating that his flight from Lisbon to Amsterdam landed 3 hours and 10 minutes late, and asks for compensation. The chatbot’s response can take one of three forms.

One response flatly rejects the claim and is incorrect in most cases. A second response confirms compensation of 400 euros in accordance with EU passenger rights and is correct in most cases, unless, for example, the delay was caused by a bird strike. A third response refers the passenger to a colleague in the call center, which is certainly safe but raises the question of who bears the cost of providing incorrect information. The case of a Canadian airline whose chatbot had promised a refund that the company subsequently denied illustrates the consequences. Such an error is repeated by a chatbot every night and in every language.

Against this backdrop, the presentation highlighted an important distinction: A chatbot can be compliant—for example, with the requirement to label it as AI under the EU AI Act, which takes effect on August 2, 2026. However, no law verifies whether it provides the correct answer. That is why we need technically defined performance standards and systematic testing—just as airplanes are inspected before every takeoff.

Two approaches took center stage:

1. Golden Data Sets: A curated collection of questions with verified reference answers forms the basis for reproducible tests. This makes it possible to track whether a chatbot’s quality improves or deteriorates between two releases.
2. LLM-as-a-Judge: Another language model evaluates the chatbot’s responses based on defined criteria. This significantly speeds up the evaluation process but does not replace expert review.

An important point in the presentation was how to handle the uncertainty inherent in such evaluations. Since AI systems respond stochastically, the same question can lead to different results. Therefore, even with automated evaluation, human oversight remains necessary. LLM-as-a-Judge speeds up the evaluation, while humans retain professional responsibility for the outcome.

Balamurali Boggu and Anupam Krishnamurthy present on testing customer-facing AI chatbots

Many Discussions with Experts

We had numerous valuable conversations during the trade show. A clear picture emerged: Those who understood what we do consider quality assurance to be a key factor in airline projects and have identified specific needs.

The exchange with technology providers was particularly insightful for us. Many are currently developing an AI layer for FlightOps or MRO systems and are looking for partners who not only test but also understand the technical intricacies of the airline industry. This is precisely where our strength lies: Our team in the Airline Business Unit consists primarily of experts who come from the aviation industry. They know the processes, the systems, and the requirements behind them.

At the same time, we have initiated discussions with several airlines, which we are now continuing.

A Voice from the Company

Christian Knoop, CEO of TestSolutions, sees the trade show appearance as confirmation that the company is on the right track:

“The airline industry is facing major changes, driven by retail, automation, and AI. Quality determines whether these innovations will succeed in practice. That’s exactly where we want to step up and take responsibility with our expertise and practical experience.”

Outlook

The World Aviation Festival has shown us that the need for technically sound quality assurance in the airline industry is growing and that technology providers are increasingly relying on partners with industry expertise. We’ll carry these insights forward into our future work and upcoming industry events. We’re particularly looking forward to the IATA World Passenger Symposium (WPS) on October 28 and 29, 2026, in Macau. As at the World Aviation Festival, Christian Knoop (CEO), Vittorio Monteiro, and Sabri Deniz Martin will be in attendance and will follow the “Retail and Distribution (Offers & Orders)” track.

Contact Us

If you’re interested in testing topics related to airline IT, customer-facing AI, or test data for GDS systems, please feel free to contact us. We look forward to a professional exchange.