Is Plagiarism Detection AI High-Risk Under the EU AI Act?
AI that detects cheating or AI-written work in exams is high-risk under Annex III point 3. Article 14 oversight, false positives, and the prohibited line.
Yes — for almost every realistic use. An AI system that scans student submissions to detect plagiarism, contract cheating, or AI-generated text, or that flags prohibited behaviour during an exam, is high-risk under Annex III point 3. That heading covers AI used to evaluate learning outcomes, and AI used to monitor and detect prohibited behaviour during tests. A plagiarism or AI-content detector that decides whether a student is penalised falls squarely inside it.
The reason the EU legislature drew the line here is the same reason these tools worry teachers: a false positive can cost a student a grade, a credit, or a place on a course. So the obligations centre on the thing detectors get wrong — accuracy, contestability, and a human who can overrule the machine (Article 14). One adjacent line matters and gets missed: if your proctoring stack also reads a student's facial expressions to flag stress or "suspicious affect", that is emotion recognition in education, which is prohibited under Article 5(1)(f) — a ban, not a high-risk obligation.
This page draws the boundaries: when detection is high-risk, the narrow case where it might not be, where it tips into prohibited territory, and what a provider and a deployer each owe.
Why Plagiarism Detection Sits in Annex III Point 3
Annex III point 3 lists two education use cases that turn an AI system high-risk: systems used to evaluate learning outcomes (including steering a student's learning process), and systems used to monitor and detect prohibited behaviour of students during tests. A plagiarism detector and an AI-writing detector both produce a signal that feeds a grading or disciplinary decision — that is evaluating a learning outcome. Software that watches a candidate during an online exam to flag cheating is the second limb almost word for word.
Article 6(2) is the mechanism: being listed in Annex III is what makes a system high-risk. There is no separate "education-software" carve-out. The classification turns on the use — assessing or policing a student — not on whether the vendor calls it an "integrity tool" or a "writing-feedback assistant".
What counts and what is adjacent
- Plagiarism / source-matching detectors that score similarity against a corpus and feed an academic-integrity decision — high-risk.
- AI-content detectors ("was this written by ChatGPT?") used to penalise students — high-risk, and notoriously error-prone, which sharpens the Article 14 duty.
- Online exam proctoring that flags gaze, second voices, or tab-switching as "prohibited behaviour" — high-risk under the second limb.
- A pure grammar checker that only suggests edits to a student's own draft, with no integrity or grading consequence — typically minimal-risk, because it neither evaluates an outcome nor polices a test.
The False-Positive Problem and Article 14 Human Oversight
AI-writing detectors are the cautionary tale of this category. They misclassify human prose as machine-written, and the error rate is higher for non-native English writers — exactly the cohort least able to contest a penalty. The Act does not ban an imperfect detector; it requires that a human, not the score, makes the call.
Article 14 demands human oversight that is real, not nominal. The deployer's staff must be able to understand the output, interpret it correctly, decide not to act on it, and override it. A detector that auto-fails a student on a confidence score, with no academic reviewing the flag, breaches Article 14. So does an oversight design where the reviewer is told "the tool is 99% accurate" and is, in practice, rubber-stamping.
Three more requirements bite hard here:
- Article 15 — accuracy and robustness. The provider must declare the accuracy metrics and the system's limits. A detector marketed as definitive when it is probabilistic is an Article 13 / Article 15 problem.
- Article 10 — data governance. Training and validation data must be examined for bias — including the non-native-writer skew.
- Article 13 — transparency to deployers. The instructions for use must tell the institution what the tool can and cannot do, so a teacher can oversee it.
The Prohibited Line: Emotion Recognition in Education Is Banned
This is the boundary that catches schools and edtech vendors off guard. Many proctoring products bolt on "affect detection" — claiming to read stress, attention, or deceptive emotion from a webcam. In an educational setting, that is not a high-risk feature you can document your way through.
Article 5(1)(f) prohibits the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions (except where the AI system is intended for medical or safety reasons; detecting a physical state such as fatigue is not emotion recognition at all, under recital 18). The ban has applied since 2 February 2025. So a proctoring tool that scores a student's facial expression for "anxiety indicating cheating" is a prohibited practice — the top penalty tier — regardless of how careful the rest of the system is.
| Capability in an exam-monitoring tool | Tier | Basis |
|---|---|---|
| Detecting AI-written text / plagiarism for grading | High-risk | Annex III point 3 |
| Flagging tab-switching, second voices, gaze-off-screen | High-risk | Annex III point 3 |
| Inferring a student's emotion ("stress", "deception") | Prohibited | Article 5(1)(f) |
| Verifying the candidate's identity 1:1 to log in (face match to their own enrolled template) | Limited / minimal | 1:1 verification is outside Annex III point 1 |
Keep the two apart. Detecting behaviour is high-risk and lawful with the full stack; inferring emotion in education is banned.
The Narrow Article 6(3) Exemption — and Why It Rarely Applies
Article 6(3) lets an Annex III system escape high-risk status if it poses no significant risk of harm to health, safety, or fundamental rights — for instance because it performs a narrow procedural task, or merely improves the result of a completed human activity, or only detects patterns without replacing or influencing the human decision. Only one condition is needed.
A detector that flags potential matches for a teacher to investigate, and demonstrably does not replace or influence the academic-integrity decision, might argue this. But two things usually defeat it. First, any system that profiles natural persons is always high-risk — and scoring students by writing style edges toward profiling. Second, in real deployments the flag does influence the decision; reviewers rarely overturn a confident score. If you rely on Article 6(3), you must document the assessment and you still register under Article 49. Treat the exemption as the exception, not the plan.
Provider vs Deployer: Who Owes What
Most schools and universities are deployers; the edtech vendor is the provider. The duties split accordingly — and the deployer's are lighter than institutions fear, but not trivial.
Provider (the vendor) — Article 16 stack: risk management (Article 9), data governance and bias testing (Article 10), technical documentation (Article 11 / Annex IV), record-keeping (Article 12), transparency and instructions for use (Article 13), human oversight by design (Article 14), accuracy and robustness (Article 15), conformity assessment via the Annex VI internal-control route (education sits outside the biometrics carve-out, so no notified body is generally required), and registration (Article 49).
Deployer (the institution) — Article 26: use the system within its intended purpose and according to the instructions; assign competent human oversight (Article 14 in practice); monitor operation and keep logs for at least six months; inform affected students where required; and — because this is a public body or a public-service provider — complete a fundamental rights impact assessment under Article 27 before first use.
A warning under Article 25: if an institution rebrands the tool, substantially modifies it, or repurposes it to a new high-risk use, it can become the provider and inherit the heavier duties.
Worked Example: Lernbrücke gGmbH, a Mid-Size Online School
Lernbrücke gGmbH, a German non-profit online secondary school, ~140 staff, ~€11M annual turnover, runs remote exams. It licenses a third-party tool that (a) detects AI-written answers and (b) watches the webcam for "prohibited behaviour".
Classification: both functions are high-risk under Annex III point 3. Lernbrücke is the deployer; the vendor is the provider. Lernbrücke's checklist: assign teachers as human reviewers who can override every flag (Article 14), confirm the vendor declares the detector's accuracy and error profile (Articles 13/15), use the tool strictly within its intended purpose and retain logs (Article 26), and — as a public-service educational body — run an Article 27 FRIA before go-live, building on its existing GDPR DPIA.
One feature in the vendor's roadmap forces a stop: an "engagement and stress" module that infers emotion from the webcam. In an education setting that is prohibited under Article 5(1)(f). Lernbrücke must disable it — no documentation makes it lawful — and the procurement contract should bar emotion inference outright. If Lernbrücke ignored this and the module ran, the exposure jumps from the high-risk tier to the prohibited tier: €35 million or 7% of worldwide annual turnover (Article 99(3)) rather than €15 million or 3% (Article 99(4)). The SME cap under Article 99(6) takes the lower of the percentage or the fixed sum — real reassurance for a body Lernbrücke's size, but not a reason to run a banned feature.
Deadlines: What Binds You Now and What Binds You Later
Two clocks run here, and confusing them is the usual planning error.
The Article 5(1)(f) emotion-recognition prohibition is in force now, since 2 February 2025. No deferral touches it. The high-risk Annex III point 3 obligations originally read 2 August 2026 in the Regulation. The Digital Omnibus — Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026 and in force since 27 July 2026 — moved them to 2 December 2027. Plan against 2 December 2027 for the high-risk programme. The ban needs no such hedge — it already binds you.
How Confir Helps
Confir's rule-based engine classifies an exam or integrity tool by its use: it runs the Article 5(1)(f) emotion-recognition test before the Annex III point 3 high-risk test, so a banned affect-detection feature surfaces as a stop-deployment finding rather than slipping into a compliance plan. For the lawful high-risk path it structures the Article 11 / Annex IV documentation and the Article 27 fundamental rights impact assessment, and maps the Article 14 human-oversight design the deployer must staff. The engine is deterministic and rule-based — the same intake produces the same finding, with the rule that fired shown in plain language.
Related guides
- Annex III: education AI systems
- Exam-monitoring (proctoring) AI
- EU AI Act for edtech
- Article 5 prohibited practices
Manage your EU AI Act compliance in one place
Confir automates risk classification, technical documentation, and audit trails for any company. No consultants. No 6-month projects. 14-day free trial.
Start free trial →