Causal Attribution for Agentic Decisions: Estimators, Coupling, and a Traceability Specification
cs.AI
Quick verdict
Distinguishes marginal vs CRN total effects for agent attribution; clean planted-chain proofs, honest limits, but all-synthetic; key empirical claim untested.
Summary
The paper asks what records an agentic AI system must keep for post-hoc causal attribution of a decision to be possible. It separates two total-effect estimands for intervening on an agent step: a marginal effect where downstream steps re-decide under fresh noise, and a common-random-numbers (CRN) effect holding downstream exogenous noise at factual values, plus a natural direct effect with downstream actions pinned. On a planted four-step Bernoulli chain, Proposition 1 proves the marginal estimand assigns causally inert and decisive steps identical effects on every run; Proposition 2 proves the CRN effect is exactly zero for the decisive step on the ~1−q of runs where direct and mediated paths cancel (measured rate 0.0974 over 20,000 draws; |DE|=0.254±0.014 there). Against Shah's locus rule the two attributions name different components on 52 of 60 draws. Section 5 gives a closed form for shared-u quantile coupling, showing it falls short of the maximal-coupling bound 1−TV (0.412 vs 0.821 at TV=0.179, V=32), and derives batch-invariance replay-floor requirements. Section 6 restates the mediation literature's suppression result: mediated shares can exceed one and rank suppressors above pure mediators, handled by exclusion with reported rates. The discrepancy experiment testing whether observability tracks causation is pre-registered in full but not run; all results are on synthetic chains, with no language model or live pipeline evaluated. Appendix A offers a twelve-requirement traceability specification mapped onto the EU AI Act.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 1
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The marginal total effect used by prior work assigns a causally inert step and a decisive step identical effects on every run of the planted chain (Proposition 1), so no magnitude threshold can separate them.
- The common-random-numbers total effect returns exactly zero for a decisive step on the ~1−q of runs where equal-weight direct and mediated paths oppose (Proposition 2), so an exact zero does not certify inertness.
- Shared-u quantile coupling falls short of the maximal-coupling bound 1−TV with a closed-form gap; maximal coupling's 1−TV agreement is the only model-independent quantity, so it (or a shared branch-independent token order) is a binding design requirement.
- The mediated share |ME|/|TEcrn| is not a valid ranking quantity under suppression — it exceeds one and ranks suppressors above pure mediators — so shares must be undefined-and-flagged at suppressed steps with reported exclusion rates (restating MacKinnon et al.).
- No binding provision of the EU AI Act currently prescribes the per-instance record attribution requires, despite Article 86's right to explanation being in force since 2 August 2026; Appendix A supplies a twelve-requirement specification (R1–R12) with conformance levels L0–L3.
Strengths
Provable separation results with closed forms, validated against hand derivations
Proposition 1 proves the marginal estimand assigns an inert step and the decisive retrieval identical effects on every run of chain (1); Proposition 2 proves the CRN effect of the decisive step is exactly zero when u_fact3 ≥ q. Each of TEmarg, TEcrn, DE matches its closed form at every step within Monte Carlo error; the earlier meaningless 5.55e−17 IEEE-754 identity check was withdrawn.
Exceptional evidentiary candor, put in the abstract not buried
The paper leads with its own estimand's failure ('an exact zero does not certify that a step did nothing'), reports the 18/300 seed anomaly (two-sided binomial p=0.02) rather than reseeding, withdraws a wrong sorting requirement with the reversal data (0.774 sorted vs 0.631 fixed at peaked V=32), and documents a transcription error (0.0446 vs 0.1655 in Table 2) found by scripts/audit_paper_numbers.py.
Genuine methodological import: mediation literature brought into agent attribution
Coupling analysis with practical design consequences
Specific, current regulatory analysis with disclosed limits
Weaknesses
The central empirical question is specified, not answered
All validation on one four-step Bernoulli chain family; nothing touches a real model
Modest mathematical novelty
The 52/60 locus-rule disagreement is characterized, not adjudicated
No public code link despite pervasive references to committed artifacts
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks what an agentic AI system's records must contain for post-hoc causal attribution of a single decision to be possible. It separates two total-effect estimands for an intervention on one step — marginal (downstream steps re-decide under fresh noise, Definition 1) and common-random-numbers (CRN; downstream noise held at factual values, Definition 2) — adds a natural direct effect with the downstream pinned to factual actions (Definition 3), and proves the failure modes of both total effects on a planted four-step chain (Propositions 1–2). It then gives a closed form for shared-u quantile coupling and its gap to maximal coupling (Section 5), restates the mediation-literature suppression pathology for the mediated share (Proposition 3), publishes a pre-registered discrepancy experiment unexecuted (Section 7), and derives a twelve-requirement traceability specification mapped onto the EU AI Act (Section 8, Appendix A). The authors are explicit that the empirical question — whether deployed observability tracks causal effect — is specified and not answered.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the difference between the marginal and common-random-numbers total effect in agent attribution?
The marginal total effect re-rolls every downstream step's randomness after intervening on one step, measuring average system behaviour with the step changed. The common-random-numbers (CRN) effect holds downstream exogenous noise at the values actually observed, making it a same-unit counterfactual. The paper shows the marginal version cannot distinguish a step with no causal path from a decisive one (identical effects on every run), while the CRN version separates them but can return exact zeros for steps that did act.
Why can an exact zero causal effect be misleading?
Because a step can act through two paths that cancel exactly. In the paper's example, when the direct path and the mediated path carry equal weight (w=1/2) and the observed noise inverts the mediator, the CRN total effect is exactly zero even though the step's direct effect is 0.25 and it demonstrably changed the outcome. A zero must therefore be read alongside the step's change rate and the recorded factual noise before concluding the step did nothing.
What does the EU AI Act require for agentic AI logging under Article 12?
Per this paper's reading, Article 12 requires logging capability tied to system-level purposes (risk identification, post-market monitoring, deployer monitoring) and its only enumerated minimum log content applies to remote biometric identification systems; creditworthiness (Annex III point 5(b)) and most other high-risk classes have no prescribed minimum. The paper also notes Article 86's explanation right has applied since 2 August 2026 while the Article 11/12/Annex IV duties were deferred to 2 December 2027 by Regulation (EU) 2026/1744.
Was the paper's discrepancy experiment actually run?
No. The experiment comparing observability-derived rankings against causally measured rankings is fully pre-registered and published — with hypotheses H1–H4, a Plackett-Luce primary model, separation guards and exclusion rules — but not executed, because the live regulated pipeline it requires was unavailable. All quantitative results in the paper come from synthetic planted chains and synthetic token distributions.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–2 could only be assessed via their captions and in-text descriptions; the rendered graphics were not available in the extracted text.
- Referenced repository artifacts (RESEARCH_LOG.md, results/validation_coupling.json, the validation scripts) are cited by name but their contents are not part of the provided text, so claims about them could not be independently checked.
- Regulatory instruments cited (Regulation (EU) 2026/1744, M/613, prEN 18229-1) could not be verified against primary sources within this review; their descriptions are taken from the manuscript.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.