From Legal Text to AI-specific Risk Sources: A Systematic Analysis of the EU AI Act's High-Risk Requirements
Ronald Schnitzer, Mike Auer, Rumpa Choudhury, Andreas Hapfelmeier, Maximilian Hoeving, Isabelle Painter, Josiane Xavier Parreira, Sonja Zillner
cs.AI
Quick verdict
Analyzes 87 EU AI Act high-risk requirements: 31% target AI-specific risks, yielding a 40-item risk source list; transparent method, but thin validation.
Summary
This paper systematically decomposes Section 2 of the EU AI Act (Articles 8-15), the mandatory requirements for high-risk AI systems, to determine how many directly address AI-specific risk sources and to make the implicit risk sources explicit. Using a requirements-engineering decomposition grounded in ISO/IEC/IEEE 29148:2018, the authors extract 111 candidate requirements and reduce them to 87 via four stated exclusion criteria (removing Article 8, 'may' permissions, regulator-facing obligations, and scoping statements). Four raters with AI engineering/research backgrounds then classified each requirement as AI risk-related (AIR), process compliance (PCR), or documentation compliance (DCR) after codebook-based pilot rounds reaching Fleiss' kappa >= 0.61; final labels came from a confidence-weighted majority vote. Results: 27 AIR (31.03%), 40 PCR (45.98%), 20 DCR (22.99%), concentrated in Articles 10, 13, 14 and 15. From free-text labels on 44 requirements (27 AIR plus 17 borderline), two researchers synthesized 40 distinct AI-specific risk sources across three clusters (data governance, human-AI interaction, technical/cybersecurity), each mapped to source articles. The authors frame the list as a version-0 minimum set and defer coverage comparison against taxonomies like the MIT AI Risk Repository to future work. Key limitations: AIR agreement sits exactly at threshold (kappa = 0.61, pairwise minimum 0.46) among institutionally homogeneous raters; the ad hoc confidence weighting is unvalidated; neither the 87 requirements, the codebook, nor the raw labels are published.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Only 27 of 87 extracted high-risk requirements (31.03%) in AIA Articles 9-15 directly address AI-specific risk sources, while 45.98% impose process obligations and 22.99% documentation obligations.
- Forty distinct AI-specific risk sources can be systematically derived from the AI-risk-related requirements and consolidated into three clusters: data governance, human-AI interaction, and technical/cybersecurity.
- Article 9 functions as a meta-requirement delegating risk-source identification to providers, so the 27 AIR requirements represent only the minimum set of AI-specific risk sources explicitly mandated by the AIA.
- The extraction and classification methodology is rule-based, re-derivable, and transferable to other regulatory instruments such as the Cyber Resilience Act.
Strengths
Rule-based, auditable extraction pipeline
Section 3.1 grounds decomposition in ISO/IEC/IEEE 29148:2018 sentence components with modal verbs as delimiters; 111 candidates are reduced by four countable exclusion criteria (EXC1 Article 8, N=4; EXC2 'may'-permissions, N=7; EXC3 regulator-subject statements, N=6; EXC4 scoping sentences, N=7) to 87 HRRs - each step is stated so a reader can re-trace it.
Multi-rater design with quantified reliability
Four independent raters, a Krippendorff-grounded codebook, and a 21-requirement pilot (24.14% of data) iterated until Fleiss' kappa >= 0.61; Table 1 reports per-category Fleiss' kappa (0.61 AIR, 0.66 PCR, 0.73 DCR) and all pairwise Cohen's kappa (0.46-0.83), including the unflattering Rater1-Rater2 AIR value.
Internally consistent results
Legally traceable deliverable with honest scoping
Candid limitations section
Weaknesses
Load-bearing category has threshold-level reliability
Unvalidated, under-reported aggregation
No reliability metric for the core deliverable's synthesis
Re-derivability claim unsupported by released artifacts
Utility claim untested; minor citation slip
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: From Legal Text to AI-specific Risk Sources (arXiv 2609.13535)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How many of the EU AI Act's high-risk requirements actually target AI-specific risks?
According to this paper, 27 of 87 extracted requirements (31.03%) directly address AI-specific risk sources; 45.98% mandate governance or management processes and 22.99% require documentation artifacts. The AI-risk-related ones concentrate in Articles 10 (data), 13-14 (transparency, human oversight) and 15 (accuracy, robustness, cybersecurity).
What is the EU AI Act Risk Source List?
A list of 40 distinct AI-specific risk sources (e.g., harmful data bias, automation bias, data poisoning, lack of robustness) derived from Articles 9-15 of the EU AI Act, grouped into three clusters and each mapped to the specific legal provision that implies it. The authors call it a version-0 minimum set, since Article 9 obliges providers to identify risks beyond those explicitly named.
Who performed the classification and how reliable was it?
Four raters with AI engineering and AI research backgrounds from the same institution (TU Munich / Siemens). Fleiss' kappa was 0.61 for AI-risk-related, 0.66 for process and 0.73 for documentation categories; pairwise Cohen's kappa ranged from 0.46 to 0.83, so the weakest category sat exactly at the conventional 'substantial agreement' threshold.
Does the paper compare the list against the MIT AI Risk Repository or other taxonomies?
No. The coverage comparison against the MIT AI Risk Repository, the AI Hazard List and the AI Risk Atlas is explicitly deferred to future work; this paper positions its list as the AIA-side reference point that such a comparison presupposes.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table 3's two-column print layout is interleaved in the extraction; cluster membership was reconstructed by ID order and column headers (IDs 1-17 data, 18-30 human-AI interaction, 31-40 technical/cybersecurity), consistent with the stated total of 40 and the three named clusters.
- Footnote markers 3 and 4 are visible in the extraction but a possible footnote 5 near the Section 5.4 citation is not; the Article 16 vs Article 5 issue is flagged as a reviewer question rather than asserted as an error.
- Figure 1 is referenced but its graphical content is not part of the extracted text; the methodology description in Section 3 was used instead.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.