Tag-Aware Structured Text Translation: Towards a Systematic Understanding
Zhanglin Wu, Hengchao Shang, Daimeng Wei, Jiaxin Guo, Zongyao Li, Tengfei Song, Ning Xie, Weidong Zhang
cs.CL · cs.AI
Quick verdict
A systematic SFT+GRPO framework for tag-preserving MT with solid ablations across 6 directions, but headline gains are small and untested for significance.
Summary
The paper tackles translation of text containing HTML/XML-style tags, where LLMs must preserve tag structure without sacrificing fluency. The authors propose a three-level framework: (1) Hy-LST, a data-synthesis strategy mixing two LLM-based tag-insertion methods — direct LST and a two-stage decoupled variant (LST-2S) — to balance tag diversity and naturalness; (2) multi-task SFT over four sub-tasks of increasing difficulty; and (3) multi-reward GRPO combining COMET fluency, tag-set fidelity, and tag-scoped COMET rewards. On the Salesforce localization-xml-mt dataset extended with LLM-synthesized tags, across six directions (en2zh/ja/de/fr/ru, de2fr) and two base models (Qwen2.5-7B, LLaMA-3.1-8B), the full system improves over the strongest existing SFT baseline (LST SFT) from 84.37 to 85.62 average score on en2zh, and outperforms Qwen-max, GPT-4o, GLM-4.7 and DeepSeek-V3 on a 2K real-world en2zh HTML test set (81.33 vs 75.36 best closed-source). Ablations attribute +0.18 to Hy-LST, +0.66 to multi-task SFT, +0.41 to GRPO. Limitations: no statistical significance tests or seed variance despite small deltas; the extended test set is LLM-synthesized and filtered (XML-MATCH threshold 100); reference-free KIWI evaluation on the real-world set is en2zh-only; the diversity-naturalness trade-off is measured by an unvalidated LLM judge; no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Synthetic tagged data exhibits a measurable diversity-naturalness trade-off, and Hy-LST (equal mix of LST and two-stage LST-2S) beats both individual methods (84.55 vs 84.37/83.63 avg on en2zh).
- Multi-task SFT over four difficulty-ordered sub-tasks transfers knowledge, lifting the deployable Task4 mode from 84.55 to 85.21 average.
- Joint three-reward GRPO outperforms every single-reward configuration and the no-RL baseline, adding +0.41 average concentrated in XML-Match (+1.25), while direct GRPO without SFT collapses to 62.18.
- The 7B-based full system outperforms Qwen-max, GPT-4o, GLM-4.7 and DeepSeek-V3 on a 2K real-world en2zh HTML test set (81.33 vs 75.36 for the best closed-source model).
Strengths
Genuinely systematic ablation design
Tables 2, 4, 5, 6 and 7 isolate synthesis strategy, task composition, SFT-necessity (direct GRPO without SFT collapses to 62.18) and reward weights — each claimed component is separately tested rather than bundled.
Broad evaluation coverage
Six language directions (Tables 11–15), two base models (Table 10: LLaMA-3.1-8B replicates the ladder 84.70→86.27), plus a 2K real-world HTML test set with four closed-source baselines (Table 3).
Honest result patterns
Actionable metrics and error taxonomy
Weaknesses
No statistical rigor for small deltas
Synthetic, filtered evaluation set
Weak formalization of the core trade-off
Novelty framing overstated
Reference and setup blemishes
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses tag-aware machine translation: translating text containing paired HTML/XML-style tags whose structure must survive into the output. Framing three gaps (data diversity-naturalness trade-off, capability decomposition, multi-objective alignment), the authors propose Hy-LST data synthesis (equal mix of LST and two-stage LST-2S), four-task multi-task SFT (Fig. 4 prompts), and three-reward GRPO (R1 COMET-20 fluency, R2 tag-set F1, R3 tag-scoped COMET; Eq. 8). Central claims: each level contributes measurable gains; the full system beats existing SFT methods and four closed-source LLMs; and this is 'the first work to explicitly characterize' the synthesis trade-off.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is tag-aware machine translation and why is it hard?
It is the translation of text containing markup tags (paired HTML/XML elements like <b> or <ph>). The system must keep every tag in its output with the correct scope while still producing fluent prose. LLMs tend to drop or misplace tags, and forcing tag preservation can make the translation rigid — the fluency-versus-fidelity tension this paper targets.
What is Hy-LST?
A training-data synthesis strategy proposed in this paper. It mixes, in equal proportion, tags inserted directly by an LLM (LST — natural placement but structurally constrained) with a new two-stage method (LST-2S) that first inserts arbitrary tags into the source and only then translates the tagged source into a tagged target — more diverse but less natural. The hybrid balances the two.
How much better is the proposed system than GPT-4o or Qwen-max?
On a 2,000-page real-world en2zh HTML test set, the 7B-based full system averages 81.33 versus 75.36 for Qwen-max (best of four closed models; GPT-4o scored 63.85). The gap is almost entirely tag fidelity: XML-MATCH 92.10 versus 78.15/52.45, while reference-free quality (COMET22-KIWI) is comparable at 79.47 versus 80.13.
Is the code or dataset available?
The underlying corpus is the public Salesforce localization-xml-mt dataset on GitHub. The paper states detailed prompts and hyperparameters but does not announce a code release; the extended test set depends on commercial LLM API outputs and the real-world crawl is not described as released, so exact replication is not currently possible.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 are only partially recoverable from text extraction (numeric labels garbled); all 15 tables extracted intact.
- Review is based on the provided extraction; any supplementary material beyond Appendix D and page-level layout were not verifiable.
- codeAvailable=false reflects the absence of any release statement in the text, not an explicit refusal.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.