Testing AI Governance: EU AI Act compliance requires evidence, not stacks of paper.

In most AI projects, governance and testing tend to be handled separately. Documentation is required for compliance purposes. Testing validates the model. However, few people link these two areas. The question we often hear isn’t whether the governance layer needs to be tested. It's: 'Who should take this on, and how, without having to build a parallel process from scratch?'

As with AI testing in general, our answer is the same. Most of the prerequisites are probably already in place.

Documentation reflects intent. A test result is a fact.

But documentation is not the same as verification. For example, a risk assessment created before the system went live says nothing about whether the risk management system was still active six months later. Similarly, a policy describing human oversight says nothing about whether it was ever applied. A data protection impact assessment says nothing about whether the data controls it describes are actually in effect in the live system.

Quality professionals will be familiar with this issue. In production environments, you can't just document that a security mechanism works. You test it under real-world conditions and record the results. A governance document is a statement of intent. A test result is a statement of fact. Quality teams are precisely trained to navigate the space between these two aspects.

Article 14 of the EU AI Act requires evidence, not role assignments.

According to Article 14 of the EU AI Act, effective human oversight of high-risk AI systems must be ensured. The word “effective” carries the entire weight here. It is not about assigning roles or signing a procedure manual. Rather, it must be demonstrated that the oversight mechanism works when it is needed.

Translating this into a test question is straightforward: Can the person in charge see what the system is doing? Do they understand what they are seeing? Can they intervene, override or stop the system if something goes wrong? Is there a log proving that this actually happens in practice, and not just that the capability exists on paper?

Imagine a high-risk system with human-in-the-loop control. On paper, oversight is in place. There is a specifically named reviewer, a documented escalation path and a procedure that requires a human to approve flagged decisions. Then you look for evidence. However, the reviewer was never trained to recognise what a problem scenario looks like. The override has never been used. The oversight log is empty. From a documentation standpoint, the requirement is met. From a testing standpoint, though, nothing was actually verified. The control existed on paper, but failed in practice.

This is because the scope of testing is usually defined in terms of functional and non-functional requirements, so governance obligations end up being someone else’s problem. No one maps Article 14 to a test case. Establishing this connection is not difficult. It just needs to be done intentionally.

Governance obligations are unwritten acceptance criteria.

Once you accept that governance can be tested, the approach becomes familiar. First, the obligations are identified and defined, and then it is verified that they function. This is just like any other system requirement.

Article 9 requires a risk management system that remains active throughout the entire lifecycle of the system, not just at the time of going live. A test can verify whether the risk register has been reviewed since the last model update and whether measures for identified risks have been documented. It can also verify whether what is currently in production still aligns with the scope of the risk assessment.

Article 12 requires automatic logging of events during operation. A test can verify the presence of logs, whether they contain sufficient detail for a regulatory authority to understand a decision and whether they can be altered retroactively. An editable log is not reliable evidence, regardless of how it is labelled.

The fairness obligations under Article 10 require training and test data to be representative, and outputs to not treat protected groups differently. Bias and fairness testing are already integrated into most reputable AI testing programmes. However, explicitly linking this to regulatory obligations changes how results are documented and what constitutes a 'pass'.

Nevertheless, none of this turns testing teams into compliance lawyers. It simply means having a discussion at the start of the project. What obligations apply? What does 'passing' entail in detail? Where will the evidence be stored?

The deadlines are approaching. So are the opportunities.

Under the EU AI Act, a conformity assessment must be carried out before a high-risk AI system can be put into operation. This assessment requires real operational evidence, rather than just guidelines and plans. Technical documentation is not something you create once and then file away. It is a dynamic record that must be kept up to date throughout the system’s entire lifecycle.

The deadlines have recently been pushed back. Consequently, the high-risk obligations will now apply from December 2027 for standalone systems and from August 2028 for systems embedded in regulated products. The reasons for postponing the deadlines are worth noting: The parts of the law dealing with testing, documentation and evaluation were the most difficult to implement within the original timeframe. The requirements have not been watered down. The preparation period has simply been extended.

This additional time offers opportunities, but it is not a reason to delay action. Now is the time to build the evidence base while the system is still under development, rather than having to reconstruct it later if the necessary records were never created. Teams that develop these habits early on will spend significantly less time on reactive work and will be in a much stronger position when the deadline arrives.

New Responsibilities, Proven Discipline.

Testing the governance layer does not require a fresh start as it is not a standalone discipline. Rather, it involves applying structured quality thinking to a new class of obligations. When AI governance is incorporated into the testing scope from the outset — from the 'Definition of Done' through to the test strategy and monitoring plan — the evidence emerges naturally alongside development. In this way, a governance statement becomes a fact rather than a mere declaration of intent.

Would you like to integrate AI governance into your testing process and into your test scope?

Get in touch. We can help you systematically incorporate the governance obligations from the EU AI Act into your test strategy.