How Many Pixels Is a Digit Worth? Place-Aware Coordinate Entropy for GUI Agent Confidence Estimation
Yunxiang Li, Xixin Wu, Helen Meng
cs.AI
Quick verdict
PACE weights digit entropy by place value for GUI click confidence; gains real but modest (2/6 bootstrap CIs cross zero), format-scoped; fair, useful study.
Summary
GUI agents emit click coordinates as short digit-token sequences, and standard text-LLM confidence signals rank correct clicks only weakly. The authors attribute part of this to place-value asymmetry: because correctness is bounding-box-defined, uncertainty in high-order digits (e.g., the hundreds place) predicts wrong clicks far better than units-digit uncertainty, which single-pass methods aggregate uniformly. They propose Place-Aware Coordinate Entropy (PACE): weight each digit's Shannon entropy by its place value (exponent α, default 1), computed from logits of the existing greedy decode—no training, sampling, or extra forward pass. On ScreenSpot-Pro and ScreenSpot-v2 across eight open agents, PACE α=1 posts the best AUROC and AUARC in all six primary agent×metric cells (e.g., 0.857/0.790/0.900 AUROC for EvoCUA-8B/MAI-UI-8B/UGround-V1-7B on ScreenSpot-Pro), beating K=10 sampling baselines at one-tenth the cost. Place weighting adds +0.022 mean AUROC over uniform coordinate entropy, though paired bootstrap shows 2 of 6 primary confidence intervals cross zero. Selective accuracy at 5% coverage rises from 63.8% to 73.2% (five-agent mean). Gains are confined to fixed-scale, separately tokenized integer coordinates; PACE loses to uniform entropy for variable-resolution pixel outputs such as Qwen2.5-VL. Main limitations: single-click grounding only (no closed-loop evaluation), ranking rather than calibration, and no code release stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Bounding-box correctness makes higher-place digit entropy more predictive of wrong clicks than lower-place entropy (single-digit AUROC 0.65→0.77→0.82 across units/tens/hundreds, Figure 2a).
- PACE α=1 achieves the best AUROC and AUARC in all six primary agent×metric cells (Table 1) using a single greedy forward pass, outperforming K=10 sampling baselines at one-tenth the inference cost.
- Place-value weighting improves over uniform Coordinate-Entropy for all eight agents on both benchmarks (mean ΔAUROC +0.019 on ScreenSpot-Pro, +0.066 on ScreenSpot-v2, Table 3).
- At 5% coverage, PACE raises five-agent mean selective accuracy from 63.8% to 73.2%, with the largest gains at low coverage (Table 6).
- The gains apply only to fixed-scale, separately tokenized integer coordinates; for variable-resolution, fractional, or merged-token formats, uniform Coordinate-Entropy is the recommended fallback (Appendix H).
Strengths
Correct built-in ablation and clean decomposition
α=0 reduces PACE to the uniform Coordinate-Entropy baseline, so the paper can cleanly split the gain: coordinate-token restriction alone contributes 0.02–0.08 mean AUROC over the best alternative single-pass summaries, place weighting adds +0.022 (§4.2); Table 4 further compares hundreds-only (0.847), ℓ1-learned (0.851) and PACE (0.851) mean AUROC.
Goes beyond the trivial shortcut
Table 5 (EvoCUA-8B, n=619 same-hundreds-cell examples) shows tens becomes the best standalone predictor (0.812 vs hundreds 0.746), strongest in 11 of 14 settings — a genuinely thoughtful defense of keeping all three digit places rather than only the hundreds.
Cost argument is decisive against sampling baselines
Unusually honest negative and boundary reporting
Weaknesses
Headline claim outruns the significance evidence
The branded component is the smaller contribution
Scope excludes a major current model family
Unexplained selective-prediction anomalies
No code or artifact release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets per-click confidence estimation for open-source GUI grounding agents. The claim chain: (1) text-LLM confidence signals (verbalized, full-sequence logits) discriminate weakly on coordinate outputs — Table 1 shows P(True) AUROC 0.435 on EvoCUA-8B/ScreenSpot-Pro, below chance; (2) the cause is place-value asymmetry: bounding-box correctness makes hundreds-digit entropy more predictive than units-digit entropy (Figure 2a: single-digit AUROC 0.65→0.77→0.82 across units/tens/hundreds); (3) therefore weighting each digit's entropy by place value (PACE, Eq. 2, α=1) recovers discrimination at zero extra cost. Headline: best AUROC and AUARC on all 6 primary agent×metric cells (Table 1), single forward pass, beating K=10 sampling baselines (SafeGround) at 1/10 cost. The scope is honestly delimited in §6 and Appendix H (fixed-scale, separately tokenized integer coordinates), but the abstract's "wins on all primary comparisons" needs qualification against Table 11.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PACE (Place-Aware Coordinate Entropy)?
A training-free confidence score for GUI agents' predicted clicks: it takes the Shannon entropy of each coordinate digit's output distribution and weights it by the digit's decimal place value (at the default α=1, the hundreds digit is weighted 100× the units digit). It is computed from the logits of the single greedy forward pass that already produced the click, adding no inference cost.
Does PACE improve the agent's click accuracy?
No. PACE does not change predictions; it only ranks the existing greedy click by confidence. Reported improvements are in discrimination (AUROC up to 0.900 for UGround-V1-7B on ScreenSpot-Pro), selective accuracy when abstaining on low-confidence clicks (63.8%→73.2% at 5% coverage, five-agent mean), and wrong-click prevention.
When does PACE not work?
It requires coordinates emitted as separate digit tokens on a fixed scale (e.g., exactly three digits per axis). The paper reports it loses to uniform coordinate entropy on variable-resolution pixel outputs (Qwen2.5-VL-32B: −0.012/−0.025 AUROC), short emissions (Magma-8B), and cannot apply to merged digit tokens (CogAgent); a uniform fallback is recommended there.
How much better is PACE than the simpler uniform coordinate-entropy baseline?
Place weighting adds +0.010 to +0.031 AUROC over uniform Coordinate-Entropy on ScreenSpot-Pro and +0.015 to +0.065 on ScreenSpot-v2 across the three primary agents; 4 of these 6 paired-bootstrap confidence intervals exclude zero, so gains are consistent in direction but not always individually significant.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 are available only as captions/axis labels in the extracted text; underlying distributions and CI bands could not be inspected.
- ScreenSpot-v2 primary results are stated to be in an Appendix not included in the extracted text; v2 numbers cited here come from Tables 3, 7, 11, and 13.
- Several cited works are dated 2025–2026 (SafeGround, HyperClick, EvoCUA, MAI-UI) and marked concurrent; those comparisons could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.