ARID: A Deployable Edge AI System for Structured Information Extraction from Industrial Maintenance Work Orders
cs.CL · cs.AI · cs.LG
Quick verdict
Edge 4-bit LLM extracts 4-field JSON from work orders on Jetson (82.9% F1, 5.3s P50, 12.5W); candid scoping, thin baselines, no variance data.
Summary
ARID is an edge-deployment pipeline that extracts four fields (component, failure mode, symptom, maintenance action) from industrial maintenance work orders into schema-valid JSON on an 8 GB Jetson Orin NX. Offline, two teachers (Gemini-2.5-Pro, DeepSeek-Chat) pseudo-label 2,326 unlabeled OMIn records; exact-agreement filtering at θ=0.75 retains 716 pairs (κ=0.744), augmented by 99 topology-constrained synthetic records (NASD). Online, a one-shot semantic router (τ=0.25; 97.3% expert activation on OMIn) selects a QLoRA adapter over Qwen2.5-3B, and grammar-constrained decoding guarantees JSON validity. On 300 human-labeled OMIn records, token-F1 is 84.8% on an RTX reference stack and 82.9% after llama.cpp Q4_K_M conversion on Jetson; resident serving measures 5,310/5,656 ms P50/P99 at 12.5 W. Zero-shot on MaintNet, semantic F1 drops to 46.4% (symptom field 12.6%) while parser success stays at least 99.8%, isolating vocabulary shift as the bottleneck. Limitations: no fine-tuned external baseline (the cloud comparison is zero-shot GPT-5 Nano at 67.3%), no seed variance for headline numbers, the router is never ablated, and roughly 5.3 s per order limits throughput; the authors themselves bound most of these claims in Section IV-F.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 4-bit Qwen2.5-3B pipeline trained on 716 dual-teacher pseudo-labels plus 99 NASD synthetic records reaches 84.8% token-F1 on 300 human-labeled OMIn records and 82.9% when deployed via llama.cpp Q4_K_M on a Jetson Orin NX.
- Grammar-constrained decoding yields 100% grammar-valid four-field JSON at ~1 ms median overhead, with end-to-end parser success of at least 99.8% even zero-shot on MaintNet.
- Zero-shot MaintNet transfer collapses semantic F1 to 46.4% (symptom field 12.6%), isolating target-domain vocabulary — not JSON formation — as the transfer bottleneck.
- The 99 NASD records produce a statistically significant +1.56 pp action-field gain in the N=100 low-resource setting (McNemar p<0.001), while the full-data gain is only 0.1 pp.
- Resident serving on the Jetson sustains 5,310/5,656 ms P50/P99 at 12.5 W without thermal throttling.
Strengths
Exemplary claim scoping
Section IV-F explicitly bounds all three contributions: the router provides specialization control not compute savings (97.3% expert activation), NASD's full-data gain is only 0.1 pp, and MaintNet 46.4% is not production-ready — candor that materially reduces overclaim risk.
Backend-explicit accuracy chain
Table I labels every token-F1 value with its backend (bitsandbytes 4-bit + outlines on RTX 3060 Ti vs llama.cpp Q4_K_M on Jetson) and isolates the 1.9 pp deployment-conversion cost (84.8% → 82.9%); no reference/deployment numbers are mixed within a cell.
Disciplined ablation statistics
Real-device tail latency and power data
Fully specified training recipe
Weaknesses
No fine-tuned external baseline
Single-run headline numbers without variance
Router contribution empirically untested
Human test-set annotation provenance unreported
Throughput ceiling and thin GCD sampling
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ARID (copyright notice indicates IEEE IECON 2026 acceptance) is a systems paper targeting a narrow, well-posed operational problem: converting short, noisy maintenance work-order narratives into a fixed four-field JSON — component, failure mode, symptom, action — entirely offline on an 8 GB Jetson Orin NX. Its three headline claims, each bounded by the authors themselves in Section IV-F: (1) a complete measured pipeline reaching 84.8% token-F1 on the RTX reference stack and 82.9% on the deployed Jetson (300 human-labeled OMIn records, Table I); (2) 99 targeted NASD synthetic records yield a +1.56 pp action-field gain in the N=100 setting while the full-data effect is only +0.1 pp (Table II); (3) grammar-constrained decoding preserves the machine interface under domain shift — parser success ≥99.8% on MaintNet — even as semantic F1 collapses to 46.4% (Fig. 6). This self-limiting style is the paper's most distinctive feature and should be credited.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does ARID achieve extracting structured data from maintenance work orders?
On 300 human-labeled OMIn records it reaches 84.8% token-F1 on an RTX 3060 Ti reference stack and 82.9% on a deployed 8 GB Jetson Orin NX using llama.cpp Q4_K_M, with 100% grammar-valid four-field JSON covering component, failure mode, symptom, and maintenance action.
What hardware and power does ARID need at the edge?
It runs on an 8 GB NVIDIA Jetson Orin NX (Aetina AIB-SN31, 20 W mode) drawing about 12.5 W during inference without thermal throttling. Resident serving takes about 5.3 seconds per work order at the median; loading the adapter from disk instead of keeping it in memory adds about 1.5 seconds.
Does ARID generalize to other maintenance domains?
Not out of the box. Applied zero-shot to MaintNet, semantic F1 drops to 46.4% (the symptom field falls to about 12.6%) even though parser success stays at 99.8% or higher. The paper's conclusion: the JSON output format transfers under domain shift, but field semantics require target-domain adaptation.
Is ARID's code or data publicly available?
The paper does not announce a code or data release. It builds on public resources (OMIn and MaintNet datasets, Qwen2.5-3B backbone), but training-label construction used commercial teacher APIs (Gemini-2.5-Pro and DeepSeek-Chat), and the latency and power numbers require Jetson hardware to reproduce.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are assessable only via captions and embedded axis labels; curve shapes (e.g., the agreement-retention trade-off in Fig. 2) cannot be independently verified.
- Extraction may omit layout-dependent details (footnotes, supplementary material); the test-set annotation protocol could be described beyond the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.