Diversified and Perceptible Counterfactual Examples Leveraging Expert Knowledge
Akram Bensalem, Fahima Djelil, Marie-Jeanne Lesot, Gr{é}gory Smits
cs.AI
Quick verdict
DiCEf embeds fuzzy vocabularies in DiCE to yield perceptible counterfactuals; elegant, validated on one dataset, but lacks baselines and statistical rigor.
Summary
The paper addresses a gap in counterfactual explanation (CFE) methods: numerically small modifications can still be semantically meaningless to the person receiving the explanation. The authors encode the explainee's interpretation of the data as a fuzzy linguistic vocabulary — a strong fuzzy partition whose kernels define zones of indistinguishability — and propose a new scalar embedding ψ that maps each feature value to [0,1] while provably preserving the information of the classical membership-vector embedding (Property 1). Because the resulting space has no structural constraints, gradient-based CFE methods become applicable: DiCEf simply replaces DiCE's distance with a distance in the vocabulary space. On the 2008 US flight delay dataset (XGBoost; four scenarios S1–S4 with roughly 19,000 CFEs each), DiCEf attains 100% fully linguistically perceptible CFEs by design, versus 30.8% for DiCE when the model is trained in the original space (S1) and 49.5% when trained in the vocabulary space (S3). DiCEf also yields higher linguistic diversity (median pairwise L1 of 7.58 in S2 and 7.19 in S4, versus 2.83 and 1.86 for DiCE), similar sparsity (medians 5/5/3/5), and more stable feature choices across training spaces (Jaccard 0.826 versus 0.700). Main limitations: evaluation on a single dataset and model, no comparisons against other knowledge-aware CFE methods, no uncertainty estimates for the CFE-level metrics, and a simplistic treatment of categorical features.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A scalar embedding ψ (Def. 7) maps each feature value to [0,1] while provably (Property 1) preserving exactly the information of the membership-vector embedding of a strong fuzzy partition.
- DiCEf, which evaluates DiCE's cost in the vocabulary-induced space Ψ, generates 100% fully linguistically perceptible CFEs on the flight dataset, versus 30.8% (S1) and 49.5% (S3) for DiCE.
- Training the predictive model in the vocabulary space alone already raises full perceptibility from 30.8% to 49.5% (S1→S3).
- DiCEf preserves sparsity (median changed features 5/5/3/5 across S1–S4) while increasing linguistic diversity (medians 7.58/7.19 vs 2.83/1.86).
- DiCEf's changed-feature sets are more stable across training spaces (mean Jaccard 0.826) than DiCE's (0.700).
Strengths
Elegant, provably sound embedding
The scalar ψ = P/(f−1) construction (Def. 7, Sec 3.3) with the Property 1 equivalence proof (ψ(x)=ψ(y) ⟺ φ(x)=φ(y)) cleanly solves the constrained-membership-vector problem; it generalizes beyond DiCE to any real-valued-vector ML method in the vocabulary space.
Well-isolated experimental design
S1–S4 cross the model's training space (U vs UV) with the generation method (DiCE vs DiCEf), letting Sec 4.2 attribute perceptibility gains separately: +18.7 points from training in UV (S1→S3, 30.8%→49.5% fully perceptible) versus 100% from generating in Ψ (S2/S4).
Operationalized perceptibility
Reproducibility commitment
Weaknesses
No external or naive baselines
Single dataset, model, and task
Underspecified cost back-mapping
Thin uncertainty reporting
Hand-crafted vocabulary unvalidated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This method paper targets a specific, well-motivated gap in counterfactual explanation (CFE): perturbations that are numerically minimal can still be invisible to the explainee, making explanations partially "deceptive" (Sec 3.2 states this explicitly). The central claims are: (1) a scalar embedding ψ (Def. 7) maps each feature value into [0,1] while provably preserving the information of the membership-vector embedding of a strong fuzzy partition (Property 1, Sec 3.3); (2) DiCEf, obtained by evaluating DiCE's cost (Def. 5, Eq. 1) in the vocabulary-induced space Ψ, produces 100% fully linguistically perceptible CFEs while preserving cost minimality, sparsity, and diversity; (3) training the model in the vocabulary space alone already improves perceptibility (S1→S3: 30.8%→49.5% fully perceptible). The claims are appropriately scoped to one dataset; the abstract does not overclaim beyond it.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is DiCEf and how does it differ from DiCE?
DiCEf is a variant of the DiCE counterfactual explanation method. It first maps each numerical feature value into a single scalar in [0,1] derived from a fuzzy linguistic vocabulary (a partition of the feature domain into labeled zones), then generates counterfactuals by minimizing DiCE's usual objective but measuring distances in this vocabulary space instead of the raw data space. Changes that stay inside one labeled zone cost nothing, so the generated counterfactuals only modify features in ways the explainee can perceive linguistically.
What data and model were used to evaluate DiCEf?
The 2008 US domestic flights dataset (about 7 million flights; a 100,000-instance class-balanced sample was used for training with 5-fold cross-validation). An XGBoost classifier predicts whether arrival delay exceeds 15 minutes from 13 features. Four scenarios crossed the training space (original vs vocabulary-embedded) with the generation method (DiCE vs DiCEf), producing roughly 19,000 valid counterfactuals per scenario.
Does DiCEf actually outperform DiCE?
On the flight dataset, DiCEf produced 100% fully linguistically perceptible counterfactuals (by design) versus 30.8–49.5% for DiCE depending on where the model was trained, with similar sparsity (median 5 changed features) and higher linguistic diversity (median pairwise distance 7.58/7.19 vs 2.83/1.86). This is one dataset and one model; the paper reports no comparisons against other knowledge-aware counterfactual methods.
Is the code or data available?
The paper states that the vocabulary details and all materials to reproduce the experiments are available in an institutional GitLab repository (gitlab.imt-atlantique.fr/research/DiCE_f), and the underlying dataset is public (US Bureau of Transportation Statistics, 2008). Access conditions for the repository are not specified in the paper.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–7 are available only as textual descriptions in the extraction; exact distributions, confusion-matrix counts, and per-feature cost bars could not be verified visually.
- The fuzzy vocabulary definitions and implementation details live in an external GitLab repository not included in the text; the 'common-sense' vocabulary's content could not be assessed.
- The ± values in Table 1 are assumed to be standard deviations over the 5 CV folds; the paper does not state this explicitly.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.