Recompilation Is Not Enough: Test-Guided Decompiled-C Repair
Yuhan Huang, Puzhuo Liu, Jianlei Chi
cs.SE · cs.AI
Quick verdict
WIP paper: LLM repair of decompiled Coreutils C with test gate passes 91/104; promising auditable protocol, but no baselines and undisclosed models/decompiler.
Summary
This paper, self-labeled work-in-progress, argues that recompilability alone is an insufficient acceptance criterion for repaired decompiled C: a recompiled command-line binary can still misparse options, print different bytes, or return a different exit status. The authors propose a bounded two-loop workflow per binary. First, deterministic preprocessing (declarations, intrinsic helpers, ABI cleanup, recovered global data, minimal POSIX shims) plus LLM build repair, driven by compiler/linker diagnostics, proceeds until recompilation under a primary three-iteration budget with recorded diagnostic extensions. Second, once the binary rebuilds, smoke checks and 'related official tests' drawn from the GNU Coreutils suite and assigned before repair expose behavioral discrepancies; compact summaries of mismatches drive deterministic recoveries, then LLM semantic repair capped at three iterations. Coreutils source and complete expected outputs are excluded as inputs, and edits are audited against test-specific and answer-key patterns. On 104 Coreutils 9.5 binaries in a single static-enriched track (sanitized symbols, constants, layout hints allowed), 91 (87.5%) recompile and pass the test gate; 9 do not recompile within budget; 4 recompile but still fail. Logged cost is 6.87M tokens. Limitations are substantial: no baselines or ablations, one utility family, an unnamed decompiler and LLM, an unspecified test-selection rule, no repeated runs, and no stated code or data release. The honest framing and explicit failure taxonomy are commendable, but this is a feasibility demonstration, not a validated benchmark.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Recompilation of decompiled C does not establish test-observed behavior; recompiled binaries can still diverge in option parsing, output bytes, and exit status (§1, Figure 1).
- A bounded two-loop workflow — compiler/linker diagnostics for build repair, then smoke checks plus pre-assigned official Coreutils tests for semantic repair — can repair decompiled C at whole-binary granularity (§3, Figure 2).
- On 104 Coreutils 9.5 binaries in the static-enriched track, 91 (87.5%) recompile and pass the test gate, 9 remain build failures, and 4 remain test-gate failures under the bounded budget (§4, Table 1).
- Test-gate feedback can make LLM-assisted repair of decompiled C more auditable than compile-only recovery (abstract, §5).
Strengths
Explicit failure taxonomy preserves auditability
Table 1 (§4.2) reports 91 passes plus 9 build-budget failures and 4 test-gate failures separately; the bounded budget is explicitly 'part of the method' (§4.1), so successes, extensions, and stopping reasons stay visible under one accounting rule.
Concrete motivating evidence for the core thesis
Figure 1's cut case: the recompiled binary passed smoke checks but only 2 of 5 related official tests; §4.2 then reports two semantic-repair iterations closing the gap with no smoke regression — a clean demonstration of the compile-only blind spot the title claims.
Anti-overfitting hygiene is designed in
Well-argued separation of repair loops
Candid threats-to-validity discussion
Weaknesses
No baselines or ablations
Irreproducible as reported: core tooling undisclosed
Test gate severity is unassessable
Single run with tiny failure cells
Narrow corpus and single track
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an explicitly preliminary, work-in-progress method paper on repairing decompiled C at whole-binary granularity. The core claim (§1, §2): recompilation is a necessary but insufficient acceptance criterion, because a recompiled command-line binary can still misparse options, print different bytes, or return a different exit status. The proposed remedy is a bounded two-loop workflow (§3, Figure 2): build repair driven by compiler/linker diagnostics (deterministic preprocessing first, then LLM repair under a primary three-iteration budget with recorded diagnostic extensions), followed by semantic repair driven by a compact discrepancy summary from smoke checks and pre-assigned official Coreutils tests under a three-iteration cap. Claims are carefully hedged — the abstract says the result 'suggests' an auditability benefit, and §5 concedes this is 'a repair discipline rather than a final decompilation benchmark.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'Recompilation Is Not Enough' mean?
It is the paper's central claim: making decompiled C code recompile does not guarantee the rebuilt binary behaves like the original. A recompiled command-line utility can still parse options incorrectly, print different output bytes, or return a different exit status, so the authors add a behavioral gate — smoke checks plus official GNU Coreutils tests — after recompilation before accepting any repair.
What results does the paper report?
On 104 GNU Coreutils 9.5 binaries (single static-enriched setting, where sanitized symbols, constants and data-layout hints are available), 91 binaries (87.5%) recompiled and passed the test gate; 9 did not recompile within the build-repair budget; and 4 recompiled but still failed the test gate. Total logged token use was about 6.87 million.
Does the repair method use Coreutils source code or answer files?
No. The stated repair inputs are the decompiled C plus compiler/linker diagnostics and sanitized binary/static-analysis sidecars; Coreutils source snippets and complete expected-output files are excluded. Generated repairs are additionally audited against test-specific branch and answer-key patterns to limit overfitting.
Is the evaluation thorough enough to be treated as a benchmark?
The authors themselves label it a preliminary, work-in-progress result: one utility family (Coreutils 9.5), one configuration, no baselines or ablations, a single run, and the decompiler/LLM are not identified. It is presented as evidence for a repair discipline rather than a final benchmark, with pseudo-only and component ablations deferred to future work.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 are images; only their captions and in-text descriptions were extractable, so in-figure details (e.g., the exact discrepancy fragments in the cut case) are taken from the surrounding text, not the figure content.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.