Measurement-Driven Sub-Network Selection for On-Premise Retrieval-Augmented Factory Agents
Vasileios Rizeakos, Georgios Paisios, Alexandros Machairas, Michael Birbas, Athanasios Bachoumis
cs.AI
Quick verdict
Edge LLM selection on measured quality+throughput beats size-based picks in a solid case study, but single LLM judge and unmeasured retrieval weaken it.
Summary
The paper formulates edge deployment of factory-floor RAG assistants as a post-adaptation selection problem. Weight-shared supernetworks are built from Llama-3.2-3B/1B-Instruct via sandwich-style in-place distillation, yielding 11-candidate grids (2.20–3.21B and 0.90–1.24B parameters). Each candidate is adapted to a 187-page mill operator manual by RAFT-style retrieval-grounded LoRA distillation from the unpruned teacher; deployment then commits one sub-network per device by blending judged RAG quality with measured on-device throughput under an ARC-Easy capability floor (φ=0.8) and memory budget, with a three-anchor throughput predictor cutting per-device benchmarking from 28.8 to 6.0 hours. On 633 held-out questions (3B, rank 6), pruning costs 13.7% judged quality and Stage-2 distillation recovers to −4.6% (bootstrap 95% CI −5.9% to −3.3%). Judged quality decouples from size (R²=0.76) while ARC-Easy tracks it (R²=0.93), and the floor disqualifies the unconstrained blend's picks. The assistant runs on Jetson Orin Nano, RevPi Connect 5, and Arduino UNO Q, with full queries of 20–139 s and 1.3–5.0 W standby. Key limitations: quality rests on a single LLM judge, retriever recall is unmeasured, the 1B core is evaluated on only 38 questions, the manual is proprietary, no code is released, and no external pruning/distillation baselines are compared.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- After structural compression and retrieval-grounded adaptation, judged RAG answer quality no longer tracks parameter count (R²=0.76) even though ARC-Easy general capability does (R²=0.93).
- Deployment-time selection must run after adaptation on measured quality and device throughput under a general-capability floor; size-, speed-, or quality-only rules each forfeit capability or throughput (Table III).
- Retrieval-grounded distillation recovers two thirds of the judged-quality loss from pruning (−13.7% → −4.6%, 95% CI −5.9% to −3.3%; 3B rank 6, n=633).
- A weight-shared supernetwork plus a three-anchor throughput predictor makes per-device selection cheap (6.0 h vs 28.8 h per new device) with 2.5–3.1% prediction error.
- The identical assistant runs across three heterogeneous edge tiers at 1.3–5.0 W standby, retaining ~95% of unpruned judged quality on the 3B tiers.
Strengths
The central post-adaptation-selection claim is genuinely supported by the paper's own measurements
Sec. III-D shows the natural quality proxy is perfectly rank-correlated with parameter count (Spearman ρ=−1.0), and Table III shows size-/speed-driven picks retain only 69–74% of quality/capability while the constrained blend keeps all three; the decoupling itself is quantified (judged RAG R²=0.76 vs ARC-Easy R²=0.93, Fig. 3).
Honest, well-scoped self-critique
Sec. VII-F concedes the single judge, unmeasured retrieval recall, the arbitrary φ=0.8 floor, and separately-judged grids; Table I's headline delta carries a bootstrap 95% CI (−5.9% to −3.3%); judged scores are explicitly declared comparable only within a session (Sec. VI).
End-to-end engineering realism
Cheap commissioning is quantified, not asserted
Ablations locate where the wins come from
Weaknesses
The primary outcome is defined by one LLM judge, with a generator circularity risk
The retrieval half of RAG is never evaluated
Thin, asymmetric evidence at the 1B scale
No external baselines or cost accounting
All evidence derives from one proprietary document
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a real industrial constraint: shop-floor hardware offers 2–8 GB of memory while capable LLMs carry billions of parameters, and cloud offloading is blocked by latency, cost, and data sovereignty. Its reframing: deployment is a *post-adaptation selection* problem, not a single compression step. Three research questions structure the work (Sec. I): whether one weight-shared supernetwork yields sub-networks whose RAG quality stays competitive after structural compression (RQ1); whether hardware-measured selection under a general-capability floor beats parameter-count or proxy-based selection (RQ2); and whether the pipeline supports practical deployment across heterogeneous edge hardware (RQ3). Headline numbers: pruning Llama-3.2-3B to rank 6 (2.77B params) costs 13.7% of judged RAG quality on 633 held-out manual questions; retrieval-grounded distillation returns it to −4.6% (bootstrap 95% CI −5.9% to −3.3%); the same assistant spans a Jetson Orin Nano, a RevPi Connect 5, and a 2-GB Arduino UNO Q at 1.3–5.0 W standby.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What hardware does this on-premise factory LLM assistant actually run on?
The authors deploy the same document-grounded assistant on three edge tiers: an NVIDIA Jetson Orin Nano (8 GB, INT8 ONNX runtime, 3B-parameter model), a RevPi Connect 5 industrial Raspberry Pi CM5 computer (4 GB, 4-bit llama.cpp, 3B model), and an Arduino UNO Q (2 GB, 4-bit, 1B model). A full query (tool routing plus retrieval plus answer synthesis) takes about 20 s on the Jetson and up to roughly 139 s on the UNO Q, with standby power of 1.3–5.0 W.
How much answer quality is lost by compressing the model for edge devices?
On 633 held-out questions about a 187-page mill operator manual, structurally pruning Llama-3.2-3B to 2.77B parameters dropped LLM-judged RAG answer quality by 13.7% relative to the unpruned model under the identical retrieval stack. A second-stage retrieval-grounded distillation brought the compressed model back to within 4.6% of the original (bootstrap 95% CI −5.9% to −3.3%), recovering about two thirds of the loss.
Why not simply pick the smallest model that fits the device's memory?
Because after task adaptation, judged answer quality stops tracking parameter count (R²=0.76 in the paper), so the smallest feasible candidate can be the worst: on the RevPi tier it scored 0.537 judged quality versus 0.726 for the selected rank, and only 69% of the original ARC-Easy general capability. The paper instead selects on measured quality and throughput under a capability floor and memory budget.
Is code or data available for this paper?
No code or dataset release is mentioned. A supplementary webpage (enakronic.github.io/llm-assistant-supplement) hosts qualitative example answers and a vision demo, but the 187-page mill operator manual used for training and evaluation is a proprietary industrial document, and the judged metrics depend on a specific commercial LLM judge (Claude Opus 4.8), both of which limit independent reproduction.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures appear only as embedded caption/axis-label text in the extraction (e.g., Fig. 3's scattered axis numbers); the plotted curves themselves were not visually assessable, though all numeric claims cited here come from tables and body text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.