Instruction Duplication as an Inference-Time Control Primitive
cs.AI · cs.CL
Quick verdict
Repeating the procedural instruction raises machine-visible compliance (+2.95pp All-8) at unchanged accuracy; solid factorial, near-ceiling, audit-hedged.
Summary
The paper proposes instruction duplication — repeating only the procedural instruction, not the question — as a minimal black-box, inference-time control requiring no retraining or decoding changes. In a complete 2×2×2 placement factorial over seven instruction-tuned models, 300 medical multiple-choice questions (MedQA, MedXpertQA, AfriMed-QA), and 16,800 scheduled generations at temperature 0, moving from one to two instruction copies raises a deterministic All-8 protocol-completion diagnostic from 90.22% to 93.17% and pre-provisional TF-IDF lexical recall from 73.44% to 74.81%, while final-answer accuracy remains exactly 60.21%; premature commitment worsens from 1.52% to 2.30%. A blinded audit by one non-clinician author judged 20 of 30 machine-detected improvements perceptual ties, so the prespecified 28/30 confirmation criterion failed. A downstream experiment with the Answer Engineering trajectory editor shows operational relevance: on 1,000 SSNHL cases the reproduced endpoint rose from 84.2% to 97.1% under substring-'steroid' scoring, while conductive-branch preservation fell from 78.6% to 73.8%. Limitations: protocol metrics sit near ceiling after one copy, the pooled lexical gain weakens after length adjustment, the AE endpoint is a narrow string match without a mediation test, and the study is exploratory and confined to medical multiple-choice.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Repeating the procedural instruction once (one→two copies) raises the deterministic All-8 compliance diagnostic from 90.22% to 93.17% while final-answer accuracy stays exactly 60.21%, showing protocol state and task correctness are separately controllable.
- The effect is placement-sensitive: a trailing duplicate is the strongest tested intervention (+2.19 TF-IDF points; All-8 +3.31), and the factorial shows saturation rather than super-additivity.
- Human readers judge 20 of 30 machine-detected improvements as perceptual ties, so the prespecified 28/30 audit confirmation criterion was not met.
- The operational value emerges only through a downstream deterministic controller: AE reached 97.1% on SSNHL with duplication vs 84.2% without, while conductive-branch preservation fell from 78.6% to 73.8%.
- Duplication carries a measurable cost: premature commitment rises from 1.52% to 2.30% (3.12% at three copies).
Strengths
Complete, well-powered factorial design
16,800 scheduled cells (7 models × 300 MedQA/MedXpertQA/AfriMed-QA questions × 8 placements) at temperature 0 with cell-specific seeds and pinned routes; the 2×2×2 cleanly separates copy count from location, showing the trailing-copy contrast (S→SA, B→BA: +2.19 TF-IDF points, +3.31 All-8) beats after-only placement (+0.31, p=.239), which rules out both a 'more is better' and a pure recency account.
Unusually honest reporting of negative and adverse results
The abstract itself carries the failed prespecified audit criterion (10/30 confirmations vs 28/30), the exactly unchanged accuracy (60.21% both arms), and the adverse premature-commitment increase (1.52%→2.30%, 3.12% at three copies); the additivity analysis reports negative residuals (p<.001), i.e., saturation, against the authors' own effect.
Disciplined statistics
Concrete reproducibility commitments
Clean demonstration of the control/accuracy dissociation
Weaknesses
Near-ceiling outcome battery makes the headline relative figure overweighted
Human audit failed its prespecified criterion with a weak rater setup
Pooled lexical gain is partly length-confounded
Downstream AE evidence is narrow, self-cited, and unmediated
Exploratory scope and self-referential validation chain
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a narrow control question: if the substantive query is unchanged, can repeating *only the procedural instruction* (not the question) change the explicit, machine-addressable trajectory a model exposes before committing to an answer? It claims four things: (1) instruction duplication is a minimal black-box inference-time control; (2) a complete 2×2×2 placement factorial (system/before/after presence) separates copy count from location; (3) there is a control-relevant dissociation — explicit protocol state changes while aggregate accuracy does not; (4) a blinded audit plus a downstream Answer Engineering (AE) experiment show why the dissociation can matter when a deterministic controller consumes the trajectory. Headline numbers: All-8 diagnostic 90.22%→93.17% (+2.95 points) on one→two copies; TF-IDF pre-commitment recall 73.44%→74.81%; accuracy exactly 60.21% (3793/6300) in both arms; premature commitment 1.52%→2.30%; audit 10/30 confirmations with 20/30 perceptual ties against a prespecified 28/30 criterion; AE SSNHL endpoint 84.2%→97.1% under duplication, conductive branch 78.6%→73.8%.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is instruction duplication?
It means adding a second copy of the procedural instruction — not the question — to the prompt (for example, once in the system message and once after the question). The paper studies it as a black-box, inference-time control that requires no retraining and no decoding changes.
Does duplicating instructions improve model accuracy?
No. Across 16,800 scheduled generations on 300 medical multiple-choice questions and seven models, final-answer accuracy stayed exactly 60.21% with one or two copies. What changed was machine-visible protocol compliance (All-8 diagnostic 90.22%→93.17%) and lexical exposure before the provisional answer (TF-IDF recall 73.44%→74.81%).
Did human evaluators confirm the improvement?
Only partially. In a blinded audit of 30 machine-detected improvements, 20 were perceptual ties to a human reader and all 10 non-tied cases favored the duplicated response. The prespecified 28/30 confirmation criterion was not met, so the improvement is mainly machine-observable rather than human-perceivable.
Where does the effect matter in practice?
When a downstream program consumes the trajectory. In the Answer Engineering experiment, adding a trailing duplicate raised the SSNHL steroid-treatment endpoint from 84.2% to 97.1% with a trajectory editor, although conductive-branch preservation dropped from 78.6% to 73.8%.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Tables 1–2 arrived as flattened inline text from PDF extraction; their values were cross-checked against the in-prose statistics (e.g., TF-IDF 73.44→74.81, All-8 90.22→93.17, accuracy 3793/6300) and are mutually consistent, so no missing data is suspected.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.