Structuring occupational accident narratives for cross-sector safety analysis: Transferability of accident-process role classification
Aho Yapi, Pierre Latouche, Arnaud Guillin, Yan Bailly
cs.CL
Quick verdict
Claims unit-level accident-role classifiers transfer across sectors (~85.7% OOD balanced accuracy); rigorous, honest, but data/code unavailable.
Summary
This study examines whether classifiers that label the functional role of each fact inside French occupational-accident narratives — work situation (A0), explicitly reported unfavourable condition (A1), accident event or deviation (B), and reported consequence (C) — transfer to unseen sectors without retraining. Narratives from INRS's EPICEA database were segmented into factual units and dual-annotated by two experts (Cohen's κ 0.83–0.91) with consensus adjudication, yielding over 70,000 labelled units from more than 10,000 narratives. Models were developed solely on 42,244 construction-sector units (6,040 narratives) and evaluated untouched on metallurgy, chemistry–plastics, and a private company corpus. Comparing TF–IDF classifiers, frozen Qwen3-Embedding-0.6B representations, and four fine-tuning objectives, average out-of-distribution balanced accuracy rose from 75.0% (TF–IDF logistic regression) and 76.9% (frozen embeddings) to roughly 85.7% after task adaptation; across ten seeds the three leading objectives (cross-entropy, supervised contrastive, SoftTriple) were statistically indistinguishable on average, with no consistent per-corpus ranking. The company corpus was hardest (~81%) and most variable. Full-encoder fine-tuning performed best, and projector benefits depended on the objective. Limitations include unquantified segmentation reliability, a single encoder backbone, missing per-class target results, and no public code or data.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Accident-process roles (work situation, explicitly reported unfavourable condition, accident event/deviation, reported consequence) learned from construction-sector factual units remain identifiable in two unseen EPICEA sectors and an independent company corpus without retraining, at ~85.7% average OOD balanced accuracy for the leading adapted models.
- Task-specific encoder adaptation, not the choice among cross-entropy, supervised contrastive, and SoftTriple objectives, drives the gain from 76.9% (frozen embeddings) to ~85.7% average OOD balanced accuracy.
- No fine-tuning objective consistently dominates across target corpora: 10-seed averages span only 85.6–85.8% and paired-bootstrap rankings flip between metallurgy, chemistry–plastics, and the company corpus (Table 6).
- Within each objective, full-encoder adaptation yielded the highest OOD performance among tested scopes, while projector inclusion helped supervised contrastive learning and SoftTriple, was neutral for cross-entropy, and hurt batch-hard Triplet (Sec 5.4).
- The independently collected company corpus, which differs in reporting environment, narrative length (~2 vs ~7 units), and role prevalence, is the hardest and most variable target (~76–82% balanced accuracy, cross-entropy SD ±2.5 pp over seeds).
Strengths
Large expert-annotated corpus with rigorous agreement protocol
Over 70,000 factual units from >10,000 EPICEA narratives, dual-annotated by two experts with pre-adjudication Cohen's κ 0.83–0.91 and observed agreement 89.3–93.6% (Table 1), narrative-level bootstrap CIs, published decision rules (Appendix B), and confusion matrices pinpointing disagreement structure (Fig 4, Appendix C: A0/A1 dominates in EPICEA, B/C accounts for 411/609 = 67.5% of company-corpus disagreements).
Disciplined source-only transfer evaluation
All hyperparameter selection, early stopping, and configuration choice used three-fold grouped cross-validation (grouped by accident ID) on construction only; metallurgy, chemistry–plastics, and the company corpus were never touched during development (Sec 4.4, Appendix D). Worst-corpus OOD balanced accuracy is reported alongside averages, exposing the company-corpus bottleneck.
Honest statistical treatment of small differences
Strong simple baselines retained and interpreted
Complete hyperparameter disclosure
Weaknesses
Segmentation reliability is never quantified
Single encoder backbone
No per-class performance on target corpora
No code or data availability
Assisted-coding value asserted but not demonstrated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper reframes occupational accident-narrative analysis as **factual-unit-level classification of accident-process roles**: work situation (A0), explicitly reported unfavourable condition (A1), accident event or deviation (B), and reported consequence (C). The core question is transferability: can models trained exclusively on French construction narratives from INRS's EPICEA database (42,244 units from 6,040 narratives) assign the same roles in metallurgy (22,456 units), chemistry–plastics (6,157 units), and an independently collected company corpus (9,539 units from 5,669 industrial-cleaning narratives) **without any target-domain adaptation**? The headline claims are that (1) roles remain identifiable out of domain, with average OOD balanced accuracy rising from 75.0% (TF–IDF) and 76.9% (frozen embeddings) to ~85.7% after task adaptation; (2) the gain comes from adaptation itself rather than the choice among leading fine-tuning objectives; and (3) the independent company corpus is the hardest and most variable target. The claims are explicitly hedged and, on the evidence presented, justified.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does this accident-narrative role classifier achieve?
On sectors and corpora never seen during training, average out-of-distribution balanced accuracy was 75.0% for a TF–IDF logistic-regression model, 76.9% with frozen Qwen3 embeddings, and about 85.7% after fine-tuning the encoder. The hardest target was an independent company corpus at roughly 76–82%, where results were also most variable across training runs.
What are the four accident-process roles used for annotation?
Each segmented factual unit of an accident narrative receives exactly one role: A0 work situation (who, what, where, equipment in normal use), A1 explicitly reported unfavourable condition (defects or hazards stated in the text, not inferred), B accident event or deviation (falls, collisions, releases, loss of control), and C reported consequence (injury, hospitalisation, fatality). When several apply to one unit, a fixed priority C > B > A1 > A0 decides the label.
Is the dataset or code for this paper publicly available?
No. The annotations come from EPICEA, a French national accident database maintained by INRS with controlled access, plus a private corpus from one company; the paper publishes no code and no public data release, although Appendices B and D document the annotation rules and all hyperparameters in detail.
Which fine-tuning method worked best in this study?
None consistently. Cross-entropy, supervised contrastive learning, and SoftTriple achieved nearly identical average out-of-distribution balanced accuracy over ten random seeds (85.6–85.8%), and paired bootstrap tests showed their rankings change depending on the target corpus. The paper's actual conclusion is that task adaptation itself — fine-tuning versus using frozen or lexical features — provides the main gain.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3, 4, 6, 7 and C.8 (role distributions, annotator confusion matrices, depth/projector curves) are referenced but their plotted values are not recoverable from the extracted text; table values were used instead.
- The abstract in the metadata is truncated mid-sentence, but the full text including conclusion and appendices is present, so confidence is unaffected on substance.
- Per-class model results are absent from the paper, so judgments about which roles degrade under shift rest on aggregate balanced accuracy and the annotator confusion analyses.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.