AI Bias and Discrimination Risk Under the EU AI Act
AI bias under the EU AI Act: how Articles 9, 10, 14 and 15 address algorithmic discrimination, plus the mitigation lifecycle for high-risk systems.
Algorithmic bias is not a soft ethical concern. Under the EU AI Act it is a hard legal exposure. If your AI system makes or influences decisions about people — who gets a job, who qualifies for credit, who passes an admissions screen — and it performs significantly worse for one demographic group than another, you have a fundamental rights risk that the regulation requires you to identify, measure, and mitigate. Failing to do that is a compliance breach with a fine ceiling of €15 million or 3% of worldwide annual turnover under Article 99(4).
This article explains what algorithmic bias is and where it originates, how the EU AI Act treats it across four interlocking obligations, and what a workable mitigation lifecycle looks like in practice.
What algorithmic bias is — and where it enters the system
Bias means that an AI system produces systematically different outcomes for different demographic groups in ways that cannot be justified by legitimate, relevant differences between those groups. Four entry points account for most real-world cases.
Data bias is the most common source. If the historical records used to train a system over-represent one group, the model learns patterns from that group's experience and generalises poorly to others. A credit model trained on decades of lending decisions made by underwriters who systematically declined applications from certain postcodes will reproduce that pattern at scale.
Proxy variables are a subtler problem. Legally protected characteristics — race, sex, disability status — may not appear in your training data directly. But variables that correlate with them, such as postcode, educational institution, or surname, can act as proxies that produce the same discriminatory effect. The variable is neutral; the outcome is not.
Feedback loops occur when a system's outputs feed back into the data it is later trained on. A recruitment tool that deprioritises profiles from underrepresented groups will, over time, generate training data in which those profiles are associated with rejection, amplifying the original disparity.
Measurement bias arises when the labels or ground truth used to train the system are themselves biased. If "successful hire" is defined by manager ratings, and managers' ratings are influenced by demographic assumptions, the model learns to optimise for a biased definition of success.
Why bias is a high-risk problem under the EU AI Act
The regulation does not contain a dedicated "bias" article. Instead, bias obligations are embedded across four articles that together form the mandatory high-risk compliance stack. This is important: bias is not a supplementary concern you address after the real compliance work is done. It is part of the real compliance work.
Article 9 — risk management must address discrimination
Article 9 requires providers of high-risk AI systems to establish and maintain a risk management system covering the full system lifecycle. The system must identify and analyse known and reasonably foreseeable risks to health, safety, and fundamental rights, including under reasonably foreseeable misuse. Bias resulting in discriminatory outcomes is a fundamental rights risk within scope.
Article 9(2)(c) specifically requires the analysis to cover risks arising from the data used to develop the system. Article 9(5) mandates that providers test high-risk systems at appropriate points during development, and in any event before market placement, to identify risks and implement targeted mitigation measures. That test must cover demographic performance. Residual risks after mitigation must be judged acceptable; if not, they must be further reduced or the system must not be placed on the market.
Article 10 — data governance targets bias at source
Article 10 governs the training, validation, and test datasets used for high-risk AI systems. Paragraph 2 requires that datasets be subject to data governance practices appropriate to the intended purpose, including examination for possible biases that are likely to affect health or safety or lead to discrimination prohibited under EU law.
Article 10(3) requires that datasets be relevant, sufficiently representative, and free of errors as far as possible. For bias mitigation, this translates to concrete dataset obligations: demographic representativeness must be assessed and documented; gaps must be addressed before training.
Article 10(5) creates a narrow but significant exception. Providers and deployers may process special-category personal data — data revealing racial or ethnic origin, health data, biometric data used for identification, and the like — strictly for the purpose of detecting and correcting biases, provided that appropriate safeguards are in place. In practice, this means you can process sensitive demographic data in a bias-audit context even where you would not ordinarily have a basis to do so, but the processing must be limited to that purpose, access must be controlled, and the safeguards must be documented.
Article 15 — accuracy and robustness across demographic groups
Article 15 requires high-risk AI systems to achieve appropriate levels of accuracy, robustness, and cybersecurity. For bias purposes, the relevant obligation is that performance must be consistent. A system that performs well overall but produces significantly higher false positive rates or lower approval rates for a protected demographic group is not sufficiently robust under Article 15.
Providers must define what "appropriate" accuracy looks like across groups, establish testing protocols that measure performance disaggregated by demographic category, and document both pre-market results and the thresholds they have set for acceptable disparity. There is no regulatory prescription for which fairness metric to use — demographic parity, equalised odds, calibration — but the choice must be justified and documented.
Article 14 — human oversight as a bias check
Article 14 requires high-risk AI systems to be designed to allow natural persons to effectively oversee and intervene in system outputs. In a bias context, this means that the humans overseeing a deployment must be able to identify when the system is producing outputs that appear discriminatory and have the authority and practical means to override or suspend the system.
Article 14(4)(c) requires that overseers be able to recognise signs of anomalies, dysfunctions, or unexpected performance. Training overseer staff to recognise demographic disparities in real-time outputs is an obligation that flows directly from this requirement.
Which systems carry the highest bias exposure
Bias risk is not distributed evenly across use cases. The systems with the highest exposure are those that fall within the Annex III high-risk categories where AI directly affects access to economic and social opportunity.
Employment and worker management (Annex III, point 4) — recruitment screening, candidate ranking, promotion decisions, and performance monitoring systems. A 40-person HR-tech company selling a CV-screening tool to employers is a provider and inherits the full high-risk stack. The fact that a human makes the final hiring decision does not reduce the provider's obligations.
Access to essential services (Annex III, point 5(b)) — creditworthiness assessment and credit scoring, excluding fraud detection. A regional lender using an AI credit model to score SME loan applications is a deployer of a high-risk system. If the model produces systematically lower scores for businesses owned by women or in rural areas, the lender is exposed under Articles 26, 9, and potentially 27 (if it is also a public body or operates a public-service credit programme).
Education and vocational training (Annex III, point 3) — AI systems used for student admission decisions, performance evaluation, and assessment scoring. Bias here carries educational exclusion as the harm.
Law enforcement (Annex III, point 6) — risk assessment and predictive tools used by law enforcement authorities. Many such tools are affected by the prohibited-practice boundary: predicting the risk of re-offending based solely on profiling is prohibited under Article 5(1)(d), not merely high-risk.
Note that deployers in the creditworthiness and life/health insurance sub-categories of Annex III point 5 are subject to the Article 27 Fundamental Rights Impact Assessment (FRIA), as are public-body deployers. The FRIA requires a structured assessment of the system's impact on fundamental rights, which must address discriminatory risk directly.
The interplay with non-discrimination law and GDPR
The EU AI Act does not operate in isolation. Discrimination on grounds of sex, racial or ethnic origin, religion, disability, age, and sexual orientation is prohibited by a network of EU equality directives. Algorithmic discrimination is actionable under those frameworks whether or not an AI Act breach also exists.
Under the GDPR, automated processing that produces legal or similarly significant effects on individuals is restricted under Article 22, and data subjects have the right to explanation. Deployers using third-party AI for hiring or credit decisions need both a lawful basis for the processing and a mechanism for providing explanations — and Article 10(5) of the AI Act, discussed above, provides the specific basis for bias-detection processing of special-category data.
These regimes share a key feature: they are effects-based. Intent is not a defence. A system that produces disparate outcomes discriminates, regardless of whether anyone designed it to.
The mitigation lifecycle in practice
Treating bias mitigation as a project with a launch date and an end date is a compliance error. The regulation requires a lifecycle approach.
Before training: assess the intended use case for bias exposure. Identify which demographic groups are relevant to the decision the system will support. Audit training data for demographic representativeness and document the findings. Apply Article 10(5) if processing special-category data to detect existing biases in historical records.
During development: test model candidates for performance disparities across demographic groups. Select fairness metrics appropriate to the use case and document the choice. Apply technical mitigation — rebalancing datasets, adjusting thresholds, applying fairness constraints — and record what was applied and what effect it had.
Before market placement: document the bias risk analysis, the testing methodology and results, and the residual risk assessment in the Annex IV technical file, as required by Article 11. The technical file must demonstrate that bias was addressed, not merely acknowledged. Establish thresholds for acceptable performance disparity and justify them.
Post-deployment: Article 72 requires providers to establish and execute a post-market monitoring plan. Bias drift — where performance disparities widen as real-world data diverges from training data — is a known phenomenon that monitoring must detect. Article 26 requires deployers to keep logs from high-risk AI systems for at least six months. If monitoring detects significant bias increase, Article 73 requires providers to report serious incidents involving harm to fundamental rights to competent authorities without undue delay.
The Annex IV technical file must be kept updated. If you apply corrective measures post-deployment, the documentation must reflect them.
How Confir helps
Managing bias risk requires documentation at multiple points in the development and deployment lifecycle. Confir's AITR module (Data and Technical Robustness, covering Articles 10, 11, and 15) structures the controls that directly govern bias: data governance documentation under Article 10, the performance testing and fairness-metric record under Article 15, and the corresponding Annex IV sections in the generated technical file.
The AIGM module (Governance and Post-Market Monitoring, covering Articles 9, 72, and 73) captures the risk management plan and post-deployment monitoring obligations — the ongoing bias-tracking documentation that demonstrates the system has not drifted into non-compliance after launch. Both modules are driven by deterministic, rule-based logic: the same intake answers produce the same output every time, and the rule behind each finding is human-readable and audit-defensible.
Start at confir.eu — no consultants, no six-month implementation.
FAQ
Does the EU AI Act require a specific fairness metric, such as demographic parity or equalised odds?
No. The Act requires that accuracy and robustness be appropriate and documented, but it does not mandate a particular mathematical definition of fairness. Providers must select a metric suited to their use case, justify the choice, and document both the selection reasoning and the test results. The choice between demographic parity and equalised odds, for example, involves genuine trade-offs that depend on the decision context.
Can we process racial or ethnic origin data to test our model for bias?
Yes, under Article 10(5), but strictly for that purpose and with appropriate safeguards. This provision is a specific derogation from the general prohibition on processing special-category data. The processing must be limited to bias detection and correction, must not feed into the live system's decision logic, and must be documented including the safeguards applied.
What is the deadline for high-risk bias obligations?
Under the Digital Omnibus agreed in May 2026, stand-alone high-risk AI systems listed in Annex III — including recruitment, credit-scoring, education, and law enforcement applications — must comply from 2 December 2027. High-risk AI embedded in regulated products under Annex I has until 2 August 2028. The original 2 August 2026 date has been deferred by this agreement. That said, the documentation takes substantial time to assemble, and 2027 is not distant.
Does a deployer also have bias obligations, or only the provider?
Both. Providers hold the heaviest obligations under Articles 9, 10, 11, and 15. Deployers must follow the provider's instructions for use, maintain human oversight under Article 14 as implemented in the deployed context, and keep logs for at least six months (Article 26). Deployers in the creditworthiness, life/health insurance, and public-service categories must also complete a FRIA under Article 27, which includes assessing discriminatory risk.
What are the penalties for bias-related non-compliance?
Non-compliance with Articles 9, 10, 11, and 15 — risk management, data governance, technical documentation, and accuracy/robustness — falls under the general high-risk obligation tier: fines up to €15 million or 3% of total worldwide annual turnover, whichever is higher (Article 99(4)). For companies with €5 million turnover, 3% is €150,000. For companies with €50 million turnover, 3% is €1.5 million. Under Article 99(6), for SMEs and start-ups the fine is capped at the lower of the percentage or the fixed amount.
Related guides
- Article 9 risk management system
- Article 6 high-risk classification criteria
- Article 8 compliance requirements
- risk classification framework
- Annex III judicial decision systems
- Annex III education AI tools
Manage your EU AI Act compliance in one place
Confir automates risk classification, technical documentation, and audit trails for any company. No consultants. No 6-month projects. 7-day free trial.
Start free trial →