Semantic-ITC: A Frame-wise Indoor Mobile Laser Scanning Dataset and Benchmark for Semantic Segmentation
Haiyang Wu, Muhammad Affan, George Vosselman, Ville Lehtola
cs.CV
Quick verdict
First frame-wise indoor MLS segmentation benchmark; well-documented, but single building, VFM-derived labels, and single-seed evaluation limit generality.
Summary
The paper introduces Semantic-ITC, claimed as the first public dataset and benchmark for frame-wise indoor mobile laser scanning (MLS) semantic segmentation. It comprises 52 sequences, 79,108 LiDAR frames, and 1.23 billion points labeled with 16 classes across five indoor scene types in one building (>11,000 m2). Labels come from a hybrid pipeline: DINOv3 (ADE20K) predictions on synchronized RGB images are projected onto LiDAR points and accumulated multi-view with confidence-weighted voting (55%/60% acceptance thresholds), corrected by BIM-derived structural labels via ICP plus kNN voting, manually refined in CloudCompare (9.2% of points changed), and back-projected exactly to the original frames. Released data include frame-wise XYZI+label files, RGB, calibration, and poses; BIM is withheld. A sequence-level split (test = sequences 1-10 from a separate area) prevents temporal leakage. Five baselines trained on single-frame XYZI reach 69.29-79.27% mIoU (WaffleIron best; PTv3 best OA 93.26%), with errors concentrated at boundaries and confusable classes (door-to-wall confusion 26.8%). Main limitations: one building and one scanner; ground truth partially inherits visual-foundation-model biases; single-seed training with no variance estimates; ignore points excluded from the release; baseline code release not stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Semantic-ITC is the first public dataset and benchmark that defines semantic segmentation on single measured indoor MLS frames, with labels attached directly to LiDAR points.
- A hybrid annotation pipeline combining DINOv3 image cues, BIM-derived structural transfer, and manual refinement can produce 1.23 billion frame-wise point labels, with manual refinement changing 9.2% of checked points.
- Single-frame indoor MLS segmentation is genuinely difficult: the best of five baselines reaches 79.27% mIoU, with errors concentrated at object boundaries and on long-tailed, geometrically ambiguous classes such as door versus wall (26.8% confusion).
- A sequence-level split with a spatially separate test area is necessary to prevent temporal leakage between adjacent MLS frames.
Strengths
Fills a genuine benchmark gap
Table 1 shows existing indoor datasets (S3DIS, ScanNet++, LiDAR-Net, 3DSES) lack frame-wise labels; Semantic-ITC is the first to combine labels-on-measured-points with per-frame labels, matching the online-perception use case argued in Sec. 1.
Multi-source annotation with quantified quality control
Sec. 3.7 reports closed-path consistency (0.001-0.012 m, mean 0.004 m), ICP RMS residuals (0.007-0.127 m, mean 0.039 m), an independent annotator re-labeling ~25M points in ambiguous regions, and a transparent 9.2% manual-change rate (113M of 1.2B points, Sec. 3.5).
Leakage-aware benchmark design
Transparent long-tail and error reporting
Sensor-realistic benchmark input definition
Weaknesses
Generalization confined to one building and one scanner
Ground truth partially inherits VFM/ADE20K biases
Single-seed evaluation without variance
Ignore points silently dropped from the release
Baseline code and tuning procedure unstated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Semantic-ITC dataset?
Semantic-ITC is a public dataset and benchmark for semantic segmentation on individual indoor mobile laser scanning (MLS) frames. It contains 52 indoor sequences, 79,108 LiDAR frames, and about 1.23 billion points labeled with 16 classes (wall, floor, ceiling, table, chair, etc.), collected in a university building across classrooms, corridors, meeting rooms, offices, and study areas.
How were the Semantic-ITC labels created?
Through a hybrid pipeline: a DINOv3 vision model segmented synchronized RGB images; predictions were projected onto LiDAR points and accumulated across multiple views with confidence weighting; BIM-derived structural labels corrected building elements; humans manually refined the result (changing about 9.2% of points); finally, labels were transformed back to the original LiDAR frames.
How does Semantic-ITC differ from ScanNet or S3DIS?
ScanNet and S3DIS provide labels on reconstructed rooms, fused RGB-D geometry, or meshes. Semantic-ITC labels the raw, sparse points of each moving LiDAR frame (median about 15,900 points per frame), preserving real MLS artifacts such as occlusion, scanning holes, and non-uniform density - the setting relevant to online robot perception rather than offline scene reconstruction.
What is the best result on the Semantic-ITC benchmark?
In the paper's single-frame benchmark, WaffleIron achieved the best mean IoU of 79.27%, while Point Transformer V3 (PTv3) achieved the best overall accuracy (93.26%) and mean class accuracy (88.10%). Remaining errors concentrate at object boundaries and on confusable pairs such as door versus wall.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (workflow diagram, point-cloud renders, qualitative comparisons) could not be inspected; assessments rest on the extracted text and tables.
- Table 9 confusions are reported only for PTv3, so cross-method error-structure analysis was not possible.
- Appendix C settings are compact; undocumented per-baseline tuning may exist but is not visible in the extracted text.
- The dataset DOI and project page were not independently verified; availability claims are taken from the paper's text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.