A Sentinel-2 benchmark dataset for deep-learning active-fire segmentation across 25 California wildfires
Shreyan Mitra, Mohammadreza Narimani, Parastoo Farajpoor
eess.IV · cs.CV · cs.LG · eess.SP · physics.geo-ph
Quick verdict
A well-documented, openly licensed Sentinel-2 active-fire segmentation benchmark of 2,148 chips across 25 California fires, whose labels are algorithmic and therefore measure rule reproduction rather than fire-detection truth.
Summary
This data paper releases an open benchmark for active-fire segmentation in Sentinel-2 imagery. The archive contains 2,148 paired 512x512 image-mask chips drawn from 25 large California wildfire incidents, with acquisitions from 23 July 2020 to 21 August 2026. Each image is a fixed-rendering three-channel composite of Level-2A bands B12, B11 and B8A at 20 m sampling; masks encode 0 for background, 1 for shortwave-infrared-rule active fire and 255 for invalid observations. Labels are produced by a HOTMAP-motivated hot-target rule: seed pixels require converted B12 of at least 0.35 and a B12-to-B8A ratio of at least 4.0, a more permissive set uses 0.25 and 2.0, and growth is limited to two four-neighbour dilation steps followed by an eight-connected sieve. Positive labels are extremely sparse, occupying 0.0766 percent of all grid cells, with 841 of 2,148 chips fire-bearing and a median of 79 fire pixels per positive chip. Partitioning is incident-disjoint: 18 training, 3 validation and 4 test fires. A reference ResNet-34 U-Net trained for 40 epochs reaches validation IoU 0.876 at epoch 35 and, at a validation-selected probability threshold of 0.99, pooled test IoU 0.837, precision 0.897 and recall 0.926 against the rule masks. A mask-blind single-analyst review of 233 test chips gives 89.7 percent chip agreement and 88.5 percent rule-component confirmation. Data are on Zenodo under CC BY 4.0 with MIT-licensed preparation and training code.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The release provides 2,148 paired 512x512 image-mask chips from 25 California wildfires, with 841 fire-bearing chips and active-fire labels occupying 0.0766 percent of grid cells.
- An incident-disjoint partition (18 training, 3 validation, 4 test fires) supports evaluation of transfer to fires excluded from training.
- A reference ResNet-34 U-Net reaches pooled test IoU 0.837, precision 0.897 and recall 0.926 against the SWIR-rule masks at a validation-selected threshold of 0.99.
- A mask-blind analyst review of 233 test chips agrees with the rule labels on 89.7 percent of chips and confirms 88.5 percent of rule components.
Strengths
Documentation is unusually complete and honest about what the labels are
Sections 3.3, 4.2 and 4.3 give the exact radiometric conversion (Eq. 1), the rendering limits (0.60 for B12, 0.50 for B11, 0.45 for B8A) and the full label rule (Eqs. 2-4) including the numerical cutoff of 1e-6 in the ratio. The Limitations section states plainly that agreement with the masks measures reproduction of the reference rule, not independent fire detection.
Incident-disjoint partitioning with an explicit inventory
Table 2 gives chips, fire-bearing chips, fire pixels and prevalence per partition (training 1470/548/284948/0.0739, validation 246/137/75028/0.1163, test 432/156/71451/0.0631). Partition membership is derivable from the file basename, so leakage checks require no metadata lookup.
Sparse-positive structure is quantified, not just asserted
Open licences, versioned archive and pinned software commit
Mask-blind analyst review is a genuine, separately distributed asset
Weaknesses
Supervision is circular: the labels are computed from the same three bands given as input
Only four test incidents, reported as pooled pixel counts with no per-incident variance
Analyst review is a single analyst with no reliability estimate
Cloud and smoke are deliberately not masked, and their effect is unquantified
Sampling frame is narrow by construction
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a data paper, not a methods paper, and it is refreshingly clear about that. It releases the California Sentinel-2 Active-Fire Segmentation Dataset, version 1.0.0: 2,148 paired 512x512 image-mask chips drawn from 25 large California wildfire incidents, with acquisitions from 23 July 2020 to 21 August 2026. Images are fixed-rendering three-channel composites of Level-2A bands B12, B11 and B8A at 20 m sampling; masks encode 0 for background, 1 for SWIR-rule active fire and 255 for invalid observations. The authors' own framing is precise: an active-fire mask describes where a satellite observation contains evidence of ongoing combustion, not the area eventually burned. That distinction, stated in Section 2 and repeated in Section 3.4, saves downstream users from a common and consequential misreading.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is in the California Sentinel-2 active-fire segmentation dataset?
It contains 2,148 pairs of 512x512 images and masks from 25 large California wildfires acquired between July 2020 and August 2026. Images are fixed-rendering three-channel composites of Sentinel-2 Level-2A bands B12, B11 and B8A at 20 m sampling. Masks use three values: 0 for background, 1 for active fire identified by a shortwave-infrared rule, and 255 for invalid observations. The data are on Zenodo under CC BY 4.0 at DOI 10.5281/zenodo.22713948, with MIT-licensed code on GitHub.
How reliable are the fire labels in this dataset?
They are algorithmic, not field-verified. The rule flags pixels with converted B12 of at least 0.35 and a B12-to-B8A ratio of at least 4.0, then grows by two four-neighbour steps into a looser set with thresholds 0.25 and 2.0. A mask-blind review of 233 test chips by one analyst found 89.7 percent chip-level agreement with the rule and confirmed 88.5 percent of rule components, but there was only one analyst and no second-annotator or independent thermal check.
Why is the dataset split by fire incident rather than randomly?
Chips from the same fire share the same landscape, weather and acquisition conditions, so a random split would let a model memorise incident-specific appearance. The authors assign whole incidents to partitions: 18 for training, 3 for validation and 4 for testing. This measures transfer to unseen fires, though the paper notes it does not guarantee a geographic buffer between separate incidents.
Can I use this dataset to benchmark my fire-detection model against published results?
Yes, but with a caveat about what the numbers mean. The published reference model is a ResNet-34 U-Net that reaches pooled test IoU 0.837, precision 0.897 and recall 0.926 at a probability threshold of 0.99. Because the masks were generated from the same B12 and B8A bands that are fed to the network, those scores measure how well a model reproduces the labelling rule, not independent fire-detection accuracy.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 through 9 could not be inspected; all figure-derived statements come from the surrounding text.
- The Zenodo archive and GitHub repository were not downloaded, so file-level counts are as reported in Tables 2 and 3 rather than independently verified.
- The four test incidents are named but no per-incident metric table is included, so the stability of the pooled test IoU across fires cannot be assessed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.