Replicating a Disjoint-Set Union Experiment over Various Notions of Micro Units to assess Translation Effort
cs.CL
Quick verdict
Replicates Miljanović 2025 across five segmentation methods on 478 TPR-DB sessions; fails to confirm P0 dominance, but reports no statistical tests.
Summary
The paper replicates an experiment by Miljanović et al. (2025), who segmented keystroke-logged translation data into micro units (MUs) via a translator-specific pause threshold (median within-word inter-keystroke interval + 2 SD) and grouped overlapping MUs into macro units (MX) as a disjoint-set union, classifying each MX into editing strategies P0–P3. Using 478 from-scratch translation sessions (eight language pairs, ~70,302 target words) from the CRITT TPR-DB 3.0, Carl compares five segmentation methods: translator-specific TSP, MUD, PUB; a static 1000 ms threshold (K1000); and a text-based TTid method where one MU is a coherent keystroke sequence on one target word. A Python notebook automatically extracts and labels the DSU. Key results: under all four temporal methods, P0 (~40.7–42.1%) and P1 (~41.0–46.8%) occur at similar rates, failing to confirm the original claim that P0 dominates; P2/P3 remain rare, and a newly observed P4 strategy (first edit during revision, 3.0–5.0%) exceeds P2. TT segmentation concentrates 85.4% under P0, largely by construction, since each MX then equals one word. The author concludes temporal thresholds fragment data in ways that obscure linguistically meaningful revision and favors product-based units. Limitations: no statistical tests or per-language-pair tables, dataset and annotation differences from the original study confound the comparison, and the text contains duplicated sentences and typographical errors.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 3
● low 0
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 14 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Across the four temporal segmentation methods (TSP, MUD, PUB, K1000), P0 (40.71–42.05%) and P1 (40.99–46.82%) occur at similar rates, so the original finding that P0 is overall the most frequent strategy does not replicate.
- P2 and P3 editing strategies remain very infrequent in the TPR-DB data, consistent with Miljanović et al.
- A newly defined P4 strategy (word first edited during end revision) is more frequent than P2 (3.04–4.95% vs 1.34–2.52%, Table 2).
- Text-based TTid segmentation yields a qualitatively different distribution (85.42% P0) because each MX equals one target word by definition, and the author argues temporal thresholds obscure linguistically meaningful revision, making DSU computation unnecessary for linguistically oriented questions.
Strengths
Threshold-sensitivity design isolates the manipulated variable
Five segmentation methods (TSP, MUD, PUB, K1000, TT) are applied to the same 478 sessions, making the effect of threshold choice directly visible: the four temporal methods cluster (P0 40.71–42.05%, Table 2) while TT diverges sharply (85.42% P0).
Automated, publicly shared pipeline replaces manual annotation
The MicroUnits.ipynb script (Critt-Kent/Key-Gaze-Correlation GitHub repo) extracts and labels the DSU automatically over public TPR-DB studies, removing the subjectivity of Miljanović et al.'s manual procedure and enabling exact re-runs.
Extends the editing-strategy taxonomy with P4
Honest negative replication
Weaknesses
No inferential statistics
Confounded comparison with the target study
Definitional circularity of the TT comparison
Missing operational details
Manuscript quality issues
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This short conference paper (Translation in Transition 8, 2026) replicates a keystroke-logging experiment by Miljanović, Alves, Brost & Neumann (2025). The original segmented translation keystroke streams into temporal micro units (MUs) via a translator-specific pause threshold — median within-word inter-keystroke interval plus two SDs (MUD) — then merged overlapping MUs into macro units (MX) as a transitive closure over shared target-word IDs, i.e., a disjoint-set union (DSU), labeled with editing strategies: P0 (one MU, initial production), P1 (extra MUs during drafting), P2 (extra MUs during end revision), P3 (both). Across 152 sessions from 38 participants (3,096 MX), the original found P0 most frequent and P2/P3 very rare.
Carl's replication asks how the segmentation threshold shapes these conclusions. On 478 from-scratch sessions from the CRITT TPR-DB 3.0 (eight language pairs, 70,302 target words), five segmenters are compared: translator-specific TSP, MUD, PUB (Bandaru et al. 2026); static 1000 ms (K1000); and a product-based TTid method where one MU is a coherent keystroke sequence on a single target word, making each MX exactly one word. A public Python notebook (MicroUnits.ipynb) automates DSU extraction and labeling. Central claims: (1) under all four temporal methods P0 ≈ P1, so P0-dominance does not replicate; (2) P2/P3 remain rare; (3) a new P4 class (word first edited in end revision) exceeds P2; (4) temporal thresholds are theoretically misaligned for linguistic research questions, and product-based units render DSU computation unnecessary.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is a disjoint-set union (DSU) in this translation study?
The paper models macro units as a DSU: overlapping micro units (keystroke sequences sharing the same target-text word IDs) are transitively merged so the result is a set of non-overlapping macro units, each covering the set of target words that were produced and revised together in a translation session.
What did this replication find that differs from Miljanović et al. 2025?
Under all four pause-based segmentation methods, P0 (single unrevised unit, ~40.7–42.1%) and P1 (extra units during drafting, ~41.0–46.8%) occurred at similar rates, whereas the original study found P0 clearly most frequent. The replication also observed a P4 strategy (word first edited during end revision, 3.0–5.0%) that was more common than P2, a category the original P0–P3 scheme did not include.
What data and code are available for this paper?
The analysis uses public studies from the CRITT TPR-DB 3.0 (478 from-scratch translation sessions across eight language pairs, about 70,302 target words), and the Python notebook (MicroUnits.ipynb) is available in the Critt-Kent/Key-Gaze-Correlation GitHub repository, so the pipeline can be re-run on the same public data.
Why does the author argue that temporal pause thresholds are problematic for studying revision?
Because a pause inside a single word splits its keystrokes into separate micro units even when no revision happened, which inflates P1-type counts and fragments the data by typing speed rather than by linguistic units. The author prefers product-based units anchored to target-text words, where each macro unit corresponds to exactly one word.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 and Figure 2 are graphics; their per-language-pair data points are not present in the extracted text, so statements about them rely on the surrounding prose only.
- The reference entries for Bandaru et al. (2026) and Carl et al. (2016) appear incomplete in the provided text; whether this reflects the source PDF or the extraction cannot be determined.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.