The Commission published its draft Guidelines on the classification of the high-risk AI systems. (Here you can read our long analysis.) 

The EU AI Act is considered the world’s first comprehensive law regulating artificial intelligence. One of its most important tenets is simple: the more harm an AI system could do to people and their rights, the stricter the rules should be.

EU decision-makers supported the risk-based approach, but this solution has problems: AI companies have significant influence over how their own systems are classified, high-risk or not.

Why ‘high-risk’ matters?

Under the AI Act, areas such as employment, education, essential public services, law enforcement and the justice system are considered sensitive and classified as high-risk.

High-risk systems face additional requirements, including documentation, risk management and oversight.

​In many cases, the company that provides the AI system decides whether their system meets higher-level obligations. The classification relies on the system’s “intended purpose”, what it says the system is designed to do.

This creates an obvious problem. Imagine two AI tools used in the workplace. One is described as an ‘employee evaluation system’. Another is called a ‘workflow optimisation tool’. The first sounds severe, and intuitively, most people would expect human involvement. But the second? Although both tools can analyse workers’ behaviour and influence decisions about their careers, only the first is likely to be deemed ‘high-risk’. If we rely too heavily on AI system providers’ self-assessment, similar technologies could face very different oversight and consequences for both companies and people.

Another way out of the high-risk category

There is a second issue. Even when an AI system initially falls into a high-risk category, the AI Act allows some systems to be excluded from that category.

For example, a company might argue that its AI performs only a narrow administrative task, improves work already completed by a human, detects patterns without influencing a decision, or merely prepares information for somebody else.

These exceptions can be reasonable. A tool that organises documents should not necessarily be regulated like an AI system deciding whether someone gets a job.

An AI tool described as ‘preparing information’ could still rank applications, highlight certain people or decide which information a human reviewer sees first. Those choices can influence real decisions even when a person technically makes the final call.

Why this matters beyond paperwork?

These classification questions concern situations in which AI can directly impact our fundamental rights. During elections, for example, general-purpose AI chatbots can compare political parties, answer questions about candidates, or provide voting suggestions. Yet they may not have been specifically designed or marketed as election tools.

Similar gaps appear in the justice system. AI may be used to analyse evidence, identify patterns in police information or support legal decision-making. People affected by these systems may have little understanding of how AI shaped the information used against them, making it harder to effectively challenge these decisions. And if it can predict life-altering decisions such as someone's probation period, then we can’t rely on an AI system that is neither transparent nor challengeable.

Therefore, EU guidelines should focus on what an AI system actually does and how it affects people, not simply the self-assessment and self-description its provider gives.

A better approach

The European Commission now has an opportunity to clarify its Guidelines on high-risk AI.

Classification should consider several factors: what the technology can do, how it is likely to be used, and how it operates in the real world. Providers should also be required to present stronger evidence when they claim that a system operating in a sensitive area is not high risk.

The goal is not to classify every AI system as dangerous but to ensure that protections apply where they are genuinely needed.

The AI Act has created an important framework for protecting people as AI becomes part of everyday life. It requires fundamental rights impact assessments, transparency, and challengeability, but only if systems are accurately identified as high-risk.

When the consequences affect jobs, elections, public services or justice, we should not simply rely on what an AI company says its product does. The question should be how the system can actually impact people’s lives.

Further readings

Response to the targeted consultation on the draft Guidelines for the classification of high-risk artificial intelligence systems

Closing the High-Risk Regulatory Gaps: Strengthening Classification and Fundamental Rights Protections under the EU AI Act