Toto-Rex's story
Toto-Rex works in an office at a big company in the EU.
One Monday morning, Toto-Rex's login stops working. Then an email arrives.
'Your employment has ended. This decision was generated by our workforce management system.'
No meeting. No name. Nobody to call.
Toto-Rex has one question: can a machine really make that decision on its own?
Toto-Rex's story is made up. The rules and cases in this video are real.
The short answer
Generally, not on its own.
The details
AI can help with dismissal decisions.
But the GDPR, the EU's data protection law, generally bans dismissals based solely on automated processing.
A few exceptions exist. Q&A 002 explains them.
The EU AI Act adds more rules. It generally treats AI intended for dismissal decisions as 'high-risk'.
High-risk does not mean banned. It means extra duties.
These duties apply from 2 December 2027.
Once these duties apply, the provider must manage risks and document the system. The provider must also build in logging and design the system for human oversight.
The employer must monitor the system and keep the logs it controls. The employer must also assign oversight to people with the necessary competence, training, and authority, and give them the support they need.
Useful to know: once these duties apply, the employer must inform affected workers and workers' representatives before using the system at work.
For systems already on the market or in service before 2 December 2027, these duties generally apply only after significant design changes.
Older systems intended for public authorities must comply by 2 August 2030 in any case.
Real-life examples
Real life: a delivery platform in Italy.
In November 2024, Italy's data protection authority, the Garante, fined Foodinho 5 million euros. Foodinho runs the Glovo app in Italy.
The authority opened its investigation after press reports about the deactivation of a rider's account.
The authority looked at how the company processed personal data about more than 35,000 riders.
It found automated systems at work, including a score that decided which riders could book shifts first, and a system that assigned orders.
It found that the company had not put in place the GDPR safeguards for automated decisions, such as the right to human intervention, to give a view, and to contest the decision.
It ordered the company to reword the messages riders receive when their account is deactivated or blocked.
It also ordered that decisions made by the algorithm be checked by suitably trained staff.
The authority had already fined the company 2.6 million euros in 2021.
In its 2024 decision, the authority also banned further processing of riders' biometric data, such as facial recognition used for identity checks.
Real life: when a score counts as the decision.
In December 2023, the EU's highest court ruled in a case about SCHUFA, a German credit agency.
It held that an automatically produced credit score can itself count as an automated decision, where a lender draws strongly on it to decide on credit.
The case was about credit, not jobs. But it shows that a score can count as the decision when someone else relies on it heavily.
Real life: outside the EU.
In 2021, Bloomberg reported on Amazon Flex, a US service where contract drivers deliver parcels in their own cars.
Algorithms rated drivers in four bands: Fantastic, Great, Fair, or At Risk.
Bloomberg interviewed 15 drivers, including four who said they had been wrongly terminated. Some said they were fired by automated email.
Amazon called the drivers' claims anecdotal and said they did not represent the experience of most Flex drivers.
This report was about drivers working in the US. It does not show a breach of EU law.
Real life: the Netherlands.
In 2023, a Dutch court examined Uber's deactivation of drivers'
accounts. Q&A 002 looks at that case.
Back to Toto-Rex
If a machine really made the decision alone, the GDPR generally bans it, unless an exception applies.
The wording of the email does not prove, on its own, whether a human was involved.
Toto-Rex can ask the company two questions: was a human involved, and which exception does it rely on?
If a tool's score effectively decided the outcome, the SCHUFA ruling suggests that a human signature at the end may not change that.
Toto-Rex can also ask for a copy of the personal data used. Q&A 010 explains how.
And once the AI Act's high-risk duties apply to the company's system, the company must tell affected workers and their representatives before using it at work.
Summary
In short: AI can assist, but in the EU, a dismissal decided solely by a machine is generally banned, with few exceptions.
Dismissal tools generally count as high-risk AI, with extra duties from 2 December 2027.
Three key terms
High-risk AI
The EU AI Act sets extra duties for some uses of AI. AI intended for dismissal decisions is generally one of them.
High-risk does not mean banned. It means extra duties.
Provider
A provider is a person or organisation that develops the AI, or has it developed.
It then offers the AI, paid or free, or starts using it itself, under its own name or trademark.
For example: a software company that sells a dismissal tool.
Deployer
A deployer uses the AI under its authority in professional work.
For example: an employer that uses that dismissal tool.
One organisation can be both. An employer that builds its own tool and uses it is the provider and the deployer.
Further reading
Start with Article 26 of the EU AI Act: what employers using high-risk AI must do.
Then GDPR Article 22: the general rule on decisions based solely on automated processing.
For how the EU Court of Justice reads Article 22, see the SCHUFA judgment.
For the Italian case, see the Garante's announcement of 22 November 2024.
Sources
EU AI Act (Regulation (EU) 2024/1689), Articles 3, 6(2) and (3), 8 to 16, 26, 111(2), and 113, and Annex III, point 4(b), as amended by Regulation (EU) 2026/1744.
GDPR (Regulation (EU) 2016/679), Article 22. CJEU, Case C-634/21, SCHUFA Holding (Scoring), 7 December 2023.
Garante per la protezione dei dati personali, decision against Foodinho S.r.l. of 13 November 2024 (docweb 10074601), press release of 22 November 2024 (docweb 10074840).
Bloomberg, 'Fired by Bot at Amazon: It's You Against the Machine', 28 June 2021.