Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty
Qing Zhang, Yifei Huang, Juyoung Lee, Thad Starner, Jun Rekimoto
cs.AI
Quick verdict
Evidence-density UI beats binary AI labels in an oracle user study (N=81, d=1.82), but audit AUC≈0.54 overall and a topic confound limits deployment claims.
Summary
The paper argues that binary 'Made with AI' labels discount AI-generated content regardless of its accuracy — a 'Transparency Penalty' — and proposes visualizing 'Provenance Density' instead: D(T), a tanh-saturated score summing verified claims weighted by source reputation and cubic semantic relevance, gated by an NLI-based 'Consistency Veto' computed over K=5 stochastic samples. A technical audit (N=200; 150 TruthfulQA, 50 FreshQA) shows retrieval density alone detects hallucinations near chance (AUC 0.47 overall; 0.28 on FreshQA), while the Consistency Veto carries most signal (0.60 overall; 0.92 on FreshQA, which holds only three hallucinated items). A within-subjects 3×3 Latin-square user study (N=81, 243 ratings, Prolific) used an Oracle protocol displaying idealized, known-correct indicators: with no signal, participants could not distinguish fluent fabrications from truth (6.28 vs 5.78, p=.43); binary labels yielded a +1.66 gap (d=0.68); Provenance Density yielded +4.15 (d=1.82), with a significant LMM interaction (χ²(2)=31.32, p<.001). Main limitations, mostly acknowledged by the authors: the user study tests an idealized signal rather than the live pipeline (whose overall AUC is ~0.54), the PDI-hallucination cell covers a single topic, FreshQA hallucination counts are tiny, the gpt-4o judge is unvalidated, and 17–26s latency precludes synchronous chat.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Without external signals, users rated fluent fabricated summaries (M=6.28) statistically indistinguishable from ground truth (M=5.78; p=.43), confirming the proposed 'Fluency Trap'.
- Binary 'Made with AI' labels act as blunt warnings that reduce trust in hallucinations (M=5.04) without restoring confidence in true content (M=6.70) — a 'Transparency Penalty'.
- Under an Oracle protocol with known-correct indicators, the Provenance Density interface creates a +4.15-point discernment gap (d=1.82), significantly larger than binary labels (+1.66, d=0.68).
- In the technical audit (N=200), retrieval density alone is near chance for hallucination detection (AUC 0.47 overall), and the NLI-based Consistency Veto carries most of the discriminative signal (0.60 overall; 0.92 on the 3-hallucination FreshQA split).
- D(T) tracks epistemic maturity — static TruthfulQA items average 0.79 versus 0.64 for dynamic FreshQA — making it an intentionally conservative, 'time-aware' signal.
Strengths
Honest dual-stream design
The Oracle protocol (Section 3.3) explicitly separates the interaction question from retriever noise, and Section 7 states the user-study result 'should be interpreted as an upper bound' — candor that keeps the claims auditable.
The audit reports evidence against its own metric
Table 2 shows density-only detection near chance (AUC 0.470 overall; 0.284 anti-discriminative on FreshQA) and attributes most signal to the Consistency Veto (0.601 overall), an unflattering ablation the paper could have hidden.
Above-average statistical transparency
Full open-science package
Weaknesses
Headline effect measured on an idealized signal
Topic confound in the Latin square
Statistically fragile audit
Tiny, author-authored stimulus set
Missing intermediate baseline and marginal key contrast
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This IJCAI/ECAI-format paper (arXiv 2609.03460) argues that binary 'Made with AI' disclosure answers the wrong question — who wrote a text rather than what supports it — and produces a 'Transparency Penalty': accurate AI-generated content is discounted along with fabrications (Section 1, citing Altay & Gilardi 2024; Nakano et al. 2026). The proposed alternative is Provenance Density (PDI), a visualization of D(T), the density of verified claims in a text. Three contributions are claimed: a theory of the 'Fluency Trap' grounded in costly-signaling theory and RLHF sycophancy (Sections 1–2); a formal metric imposing a 'computational handicap' (Section 3, Eq. 1); and dual evidence from a technical audit (N=200, Section 4) and a within-subjects user study (N=81, Section 5). The central empirical claim: an idealized PDI creates a +4.15-point discernment gap (d=1.82) between true and fabricated summaries, versus +1.66 (d=0.68) for binary labels and no detectable discrimination without signal (6.28 vs 5.78, p=.43).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the 'Transparency Penalty' in this paper?
It is the finding that labeling content as AI-generated lowers readers' trust even when the content is accurate. In the study, a binary 'Made with AI' banner cut hallucination ratings from 6.28 to 5.04 but also left true-content ratings (6.70) well below what the proposed provenance-density interface achieved (8.08).
What is Provenance Density and how is D(T) computed?
Provenance Density is a proposed interface metric that visualizes how much verified external evidence supports the claims in a text. D(T) segments text into atomic claims, retrieves web evidence per claim (Serper, top-3 results), weights it by domain reputation and cubed keyword-match relevance, saturates the sum with a tanh (β=5.0, λ=1.2), and multiplies by an NLI-based 'Consistency Veto' computed across 5 stochastic model samples.
Did the user study test the real detection system?
No. The 81-participant study used an 'oracle' protocol: grounded summaries always showed high-density indicators and fabricated ones null indicators, isolating the interface effect under known-correct signaling. The separate 200-item technical audit found the live pipeline's overall hallucination-detection AUC is only about 0.54.
How large was the effect of the provenance-density interface?
Under the oracle protocol, the gap between ratings of true and fabricated summaries was +4.15 points (Cohen's d=1.82), versus +1.66 (d=0.68) for a binary 'Made with AI' label and −0.50 (d=−0.21) with no signal; the mixed-model interaction was significant (χ²(2)=31.32, p<.001).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 were extracted with axis/label text interleaved into the prose; all numeric results cited in this review were taken from Table 2, Table 4, figure captions, and body text, not read off figure pixels.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.