RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection
Liyang Fan, Xinping Bi, Yitai Li, Shuaimin Li, Hui Li, Min Yang
cs.CV · cs.CL · cs.HC
Quick verdict
GUI grounding via lightweight reranker crop selection with a single VLM call; strong, honestly analyzed results, though the +5.5% headline needs scrutiny.
Summary
The paper proposes RankGround, a two-stage pipeline for high-resolution GUI grounding that decouples region selection from coordinate prediction. Overlap tiling (default 2x2, nine crops) generates candidates; a fine-tuned multimodal reranker, GroundRanker (Qwen3-VL-Reranker-2B with LoRA), scores all crops in one batched pass (~145 ms); the GUI-specialist VLM then predicts coordinates on the single top-ranked crop. Ranking supervision is derived automatically from ViSurf grounding annotations using a strict containment criterion plus boundary-aware positive augmentation, trained with a pointwise (BCE) to listwise (softmax) curriculum. On ScreenSpot-Pro across Qwen3-VL, MAI-UI, and UI-Venus-1.5 at 2B and 8B scales, RankGround reports absolute gains of 12.08-19.36 points over direct inference and beats ZoomIn/MVP while using one VLM call, claiming a 5.5% average improvement and 1.4x speedup over the second-best method. It generalizes to UI-Vision (49.0% vs 18.88% Direct) and five additional benchmarks. Limitations: reranker Hit@1 caps at 88.2% against 99.75% oracle coverage, icon selection errors reach 19.9%, wide targets lack containment guarantees, inference is slower than direct inference, no code release is stated, curriculum gains (at most 0.35 points) fall within seed variance, and the headline 5.5% aggregation is not directly verifiable from Table 2.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Decoupling crop selection from coordinate prediction lets RankGround achieve higher GUI grounding accuracy than multi-crop baselines (ZoomIn, MVP) with a single GUI-specialist VLM call per query.
- Strict containment labels, boundary-aware augmentation, and a pointwise-to-listwise curriculum on a LoRA-tuned Qwen3-VL-Reranker-2B reach 88.2% Hit@1 crop-selection accuracy on ScreenSpot-Pro.
- Across 3 backbones at 2B and 8B scales on ScreenSpot-Pro, RankGround improves absolute accuracy by 12.08-19.36 points over Direct inference and claims a 5.5% average gain with 1.4x speedup over the second-best method.
- The reranker generalizes out-of-distribution without benchmark-specific tuning: 78.70% Hit@1 and 49.00% end-to-end accuracy on UI-Vision versus 18.88% for Direct.
- This is the first application of a multimodal reranker to region selection in GUI grounding.
Strengths
Quantified problem decomposition with coverage guarantees
Section 3.2.1 formalizes the coverage condition (Eq. 3), argues 50%-overlap tiling satisfies it for sub-crop targets via the interval covering lemma, and measures 99.75% Oracle coverage with target-size statistics (Table 1: P90 relative area 0.156% for text, 0.034% for icons). The >18-point Direct-vs-Oracle gap (Fig. 1) precisely motivates the selection stage.
Controlled, broad empirical evaluation
Table 2 covers 3 backbones x 2 scales with matched Direct/ZoomIn/MVP comparisons and per-type accuracies; Table 3 ablates nine partitioning layouts with Oracle, random-corrected Gain@1, and latency; Tables 5-8 test OOD transfer (UI-Vision 49.0% vs 18.88% Direct) and five extra benchmarks, honestly reporting the ScreenSpot-V2 regression vs Direct (95.36 vs 96.31).
Transparent failure analysis including negative results
Practical deployment cost accounting
Principled supervision construction
Weaknesses
Reranker remains the accuracy bottleneck with no recovery path
Curriculum contribution is within noise and untested for significance
Evidence of test-benchmark tuning
Headline numbers are not directly verifiable from Table 2
Reproducibility and citation hygiene gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RankGround and how does it work?
RankGround is a two-stage GUI grounding framework. It first splits a high-resolution screenshot into overlapping crops (default nine), uses a small fine-tuned multimodal reranker called GroundRanker to score which crop contains the requested UI element, and then runs a GUI-specialist vision-language model once on only the best crop to predict the exact click coordinate. This avoids the multiple expensive VLM calls of prior multi-crop methods.
How much more accurate and faster is RankGround than previous methods?
On the ScreenSpot-Pro benchmark the paper reports absolute accuracy gains of 12.08-19.36 points over direct single-pass inference across three backbone models at 2B and 8B scales, plus a claimed 5.5% average improvement and 1.4x faster inference over the second-best multi-crop method. It is slower than plain direct inference; its efficiency advantage is against multi-crop pipelines like ZoomIn and MVP.
What is GroundRanker trained on?
GroundRanker is a 2B-parameter Qwen3-VL reranker fine-tuned with LoRA. Since no crop-ranking dataset existed, the authors automatically generated supervision from the ViSurf GUI grounding dataset (6,789 instances): a crop is positive only if it fully contains the target element, with boundary-aware augmentation. Training uses a two-stage objective - binary cross-entropy first, then a listwise ranking loss.
Does RankGround work outside ScreenSpot-Pro?
The paper reports 78.70% top-1 hit rate and 49.00% end-to-end accuracy on the out-of-distribution UI-Vision benchmark (versus 18.88% for direct inference), and it beats ZoomIn on five additional benchmarks (ScreenSpot-V2, MMBench-GUI-L2, OSWorld-G, OSWorld-G-Refined, VenusBench-GD), though on ScreenSpot-V2 direct inference remains slightly better (96.31% versus 95.36%).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract is truncated mid-sentence in the extracted text ('...across a'), so a final scope qualifier may be missing.
- Figures 1-6 cannot be analyzed from text; judgments rely on captions and in-text numbers only.
- Tables were extracted as inline text; minor digit-transcription risk exists, though internal cross-checks (delta column versus accuracy differences) were broadly consistent.
- Any supplementary material or appendix beyond the 10-page main text was not available.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.