How to make effective use of domain experts for image classification?
Dieu-Donné Fangnon, Diane Lingrand, Aurélie Liard, Marco Corneli, Antoine Pasqualini, Frédéric Precioso
cs.CV
Quick verdict
Typed losses let classifiers consume heterogeneous expert attributes; modest gains on 3 datasets, but gains are small, untested, and corrections largely simulated.
Summary
The paper extends Concept Bottleneck Models beyond binary concepts by accepting the heterogeneous attribute formats domain experts actually use: binary values, binary values that vary within a class (masked from the loss), ordered size intervals, fixed values or ranges, and categorical attributes. A fine-tuned EfficientNetV2-L extractor predicts attributes via type-specific losses (masked BCE normalized per class; a differentiable interval-censoring penalty built from softsign steps; cross-entropy for categories), combined with learnable homoscedastic uncertainty weights; a classifier head then maps predicted attributes to classes. On a 12-species Kaggle fish set, accuracy rises from 98.96% to 99.22%; on a 13-class AWA2 subset, from 95.6% to 97.07%; on a new hardwood-charcoal dataset (393 specimens, 16 species, expert IAWA annotations), from 80.94% to 82.05% (83.19% with a logistic-regression head), with 96.18% family-level accuracy. A minimum per-class combined-loss criterion flags likely misclassifications: the 25 worst-scoring charcoal specimens contain 14 of 66 errors, and simulated attribute corrections raise accuracy to 98.7-99.6%. A real anthracologist audit corrected 7 of 8 flagged specimens. Main limitations: gains are small and untested for significance, no CBM/CEM baselines, corrections are largely simulated with oracle access, and the AWA2 subset choice is unexplained.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Typed loss functions (binary, masked-variable, interval, fixed-value, categorical) enable supervised prediction of heterogeneous expert attributes, extending CBMs beyond binary concepts.
- Classifying from predicted expert attributes improves accuracy over direct classification on all three datasets: fish 98.96 to 99.22%, AWA2 13-class subset 95.6 to 97.07%, charcoal 80.94 to 82.05% (83.19% with a LogReg head).
- The minimum per-class combined loss is an effective criterion for selecting samples for expert review, capturing 14 of 66 charcoal errors within the 25 worst-scoring specimens.
- Few attribute corrections suffice to fix most errors: simulated corrections reach 98.7-99.6% accuracy with at most ~6 attribute changes, and a real anthracologist audit corrected 7 of 8 flagged specimens with 4-8 readings each.
Strengths
Principled loss design for heterogeneous attribute types
Section 3 introduces per-type losses: BCE for binary, masked BCE normalized by each class's non-variable attribute count (Sec 3.2), and a differentiable interval penalty f=1-sigma(u-l)+sigma(u-r) using softsign steps (lambda=100 charcoal, lambda=10 fish, eps=0.01 for degenerate single-value intervals, Secs 3.3-3.4). The interval loss is flat inside the admissible band and framed as an interval-censored likelihood, correctly refusing to regress toward bin centers that the annotation does not contain.
New challenging dataset with genuine expert annotations
Section 2.3 and supplementary Sec 8 present a hardwood-charcoal benchmark (393 specimens, 16 species, 3,313 micrographs over three anatomical sections) with IAWA criteria re-scored on carbonised material by anthracologists, plus a stratified specimen-level group 5-fold protocol adopted unchanged from prior work so added supervision is the only changed variable.
Real, not only simulated, expert audit
Useful misclassification triage with quantified separation
Weaknesses
Marginal and inconsistent accuracy gains
Missing baselines despite CBM/CEM positioning
Headline correction numbers are oracle simulations
Unexplained AWA2 13-class subset
Part of the supervision is nearly uninformative and unablated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How is this approach different from Concept Bottleneck Models (CBMs)?
CBMs represent every concept as a binary yes/no value, and Concept Embedding Models (CEMs) replace each concept with two embedding vectors. This paper instead accepts the heterogeneous attribute formats domain experts actually provide - binary values, binary values that vary within a class (masked out of the loss), ordered size intervals, fixed values or ranges, and categorical attributes - designs a differentiable loss per type, then predicts the class from the predicted attributes rather than from raw features.
How much does using expert attributes improve classification accuracy?
Modestly. On a 12-species Kaggle fish dataset accuracy rises from 98.96% to 99.22%; on a 13-class subset of AWA2 from 95.6% to 97.07%; on a new hardwood-charcoal microscope dataset (393 specimens, 16 species) from 80.94% to 82.05% with a linear head and 83.19% with a logistic-regression head. No statistical significance tests are reported, and one classifier head (SVM) actually performs worse with the attribute features.
What is the hardwood charcoal dataset introduced in the paper?
A new dataset of 393 charcoal specimens from 16 hardwood species (3 families, 10 genera), documented by 3,313 high-resolution micrographs across transverse, tangential and radial sections. Archaeologists (anthracologists) annotated each species with 73 binary plus 3 ordinal IAWA anatomical criteria re-scored on carbonised wood, since charcoal deformation invalidates fresh-wood size classes. It is available on request for research purposes.
How does the expert-in-the-loop correction step work?
After training, each sample's minimum per-class combined loss ranks it as potentially misclassified. Experts inspect the highest-ranked samples and correct a few predicted attributes. The paper reports that simulated corrections of at most about 6 attributes raise accuracy to 98.7-99.6%, and in a real audit an anthracologist corrected 7 of 8 flagged charcoal specimens by reading 4-8 attributes per fragment.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (box plots, confusion matrices, interval-function plots) are available only as extracted text and captions; visual claims rely on the numbers stated in the text.
- Equations and some tables (e.g., Eq 2, Table 6) were partially garbled by text extraction and were interpreted from surrounding prose.
- This is arXiv v1 (September 2026); the venue and peer-review status are not stated in the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.