A Systematic Evaluation of the COTQ Provincial Land Cover Product: Structural Consistency, Spectral Separability, and Relative Positioning Against ESA, ESRI, and Google Products
\'Etienne Clabaut, Samuel Foucher, Yacine Bouroubi
cs.AI
Quick verdict
COTQ, Québec's 10-m land-cover map, matches ESA WorldCover on structural, spectral, and expert checks; credible but partly circular and statistically untested.
Summary
The paper evaluates COTQ, a 10-m annual land-cover map of Québec produced for the MRNF via a semi-supervised deep-learning pipeline (superpixel-based sample selection, synthetic mosaics, cloud/shadow augmentation, DeepResUNet, temporal majority voting), against ESA WorldCover, ESRI Land Cover, and Google Dynamic World under a harmonized legend across eight Sentinel-2 tiles spanning temperate to tundra domains. Structural metrics (object-size distributions, shape complexity, ARI, IoU), four-band spectral separability metrics (Silhouette, Calinski-Harabasz, SAM), and a targeted photo-interpretation of COTQ-WorldCover disagreements converge on the same ordering: COTQ and WorldCover form the top pair (ARI 0.75, mIoU 0.82; CH 4351 vs 3479). In 320 sampled disagreement points, COTQ was supported 162 times versus 75 for WorldCover, though WorldCover wins for forest and riparian wetlands. Built-up IoU against a 50-cm Jilin-1 reference was 67.5% for COTQ versus 52.5-57.4% for the global products. Limitations: no province-wide accuracy estimate, single-area built-up validation using a reference produced by the authors' own pipeline, single-analyst interpretation, no statistical testing, and a comparator (WorldCover) selected partly for its similarity to COTQ itself.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Across structural (OSD, complexity, ARI 0.75, mIoU 0.82) and spectral (Silhouette, SAM, CH 4351 vs 3479/3171/2871) analyses, COTQ and ESA WorldCover consistently form the strongest-performing pair, with ESRI Land Cover and Dynamic World more spatially generalized.
- Against a localized 50-cm Jilin-1 reference, COTQ achieves the highest built-up IoU (67.5%) versus WorldCover (52.5%), Dynamic World (57.4%), and ESRI Land Cover (55.2%).
- In 320 photo-interpreted COTQ-WorldCover disagreement points, COTQ is supported 162 times versus 75 for WorldCover for most classes, while WorldCover is favored for forest and riparian/shoreline wetlands.
- Many COTQ-WorldCover disagreements stem from legitimate class-definition differences (e.g., quarries as Built-Up vs Bare/Rock, peatland continuity) rather than errors.
- COTQ's annual update capability, absent from WorldCover since its 2021 release, gives it a decisive operational advantage for monitoring soil artificialisation in Québec, though no province-wide accuracy estimate is provided.
Strengths
Multi-angle evaluation design with convergent evidence
Structural indicators (OSD log-log R² 0.79-0.98, compactness, ARI 0.75, mIoU 0.82), spectral metrics (Silhouette, SAM, CH in Table 4), and expert disagreement analysis (Table 5) independently yield the same product ordering; the framework is explicitly framed as transferable to regions lacking authoritative ground truth (Sections 3.1, 5, Table 6).
Transparent legend harmonization
Complete cross-product mapping rules are provided (Table 2), and the comparison is honestly framed as 'end-use suitability under a shared legend' rather than a comparison of original taxonomies (Section 3.3), which pre-empts a common misuse of such harmonized comparisons.
Disciplined photo-interpretation protocol
Candid limitation statements and external anchoring
Methodological recap is complete enough to understand the product being evaluated
Weaknesses
Partially circular validation design
Spectral metrics structurally favor the sensor-native product
Built-up validation rests on a single site and a self-produced reference
Single-analyst interpretation without reliability measures
No inferential statistics anywhere
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the COTQ land cover product?
COTQ (Cartographie de l'Occupation du Territoire Québécois) is a 10-m resolution annual land-cover map of Québec, Canada, produced for the provincial natural-resources ministry (MRNF) from Sentinel-2 satellite imagery using deep learning. It separates three urban classes (residential, industrial/commercial, roads) and distinguishes bare earth from exposed rock, and is designed for yearly monitoring of land occupation and soil artificialisation.
Is COTQ more accurate than ESA WorldCover or other global land-cover maps?
The study provides no province-wide accuracy estimate. Its relative evidence favors COTQ over ESA WorldCover for most evaluated classes: in 320 photo-interpreted disagreement points COTQ was supported 162 times versus 75, and for built-up areas COTQ reached 67.5% IoU against a 50-cm reference versus 52.5% for WorldCover (Dynamic World 57.4%, ESRI 55.2%). WorldCover performed better for forest and shoreline wetlands. The claim is therefore relative and setting-specific, not a universal accuracy ranking.
Where can I access the COTQ map and the data used in this study?
The COTQ map and the three global reference products (ESA WorldCover, ESRI Land Cover, Google Dynamic World) can be inspected visually through a public Google Earth Engine app (ee-dga-couverture.projects.earthengine.app/view/cotq). Sentinel-2 imagery is freely distributed by ESA. The evaluation scripts are not publicly released; the paper states derived metrics and scripts are available from the authors upon reasonable request.
Why does Québec need its own land-cover map instead of using global products?
Global 10-m products are standardized for worldwide consistency, not local fidelity. Québec's ministry needs an annually updated map with locally defined classes (separating roads, residential, and industrial surfaces to track soil artificialisation) and full control of the production chain for reproducibility. The closest global analogue, ESA WorldCover, has had no public update since its 2021 release, so it cannot support annual change detection.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- All figures (OSD log-log plots, ARI/mIoU matrices, Silhouette/SAM charts, disagreement maps, photo-interpretation examples) could not be visually inspected from the text extraction; class-wise photo-interpretation counts (Fig. 18) and per-site variability behind the eight-area means are therefore unverifiable.
- The text was extracted as a single line; the section 4.2.2 heading appears missing between the Silhouette (4.2.1) and SAM (4.2.3) subsections, though the content reads continuously.
- No per-site result tables are included, so regional variation across the eight tiles cannot be assessed from the text.
- The 50-cm Jilin-1 reference classification is not publicly documented in the paper, so the Table 3 IoU values cannot be externally verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.