Tabular Deep Learning vs Classical Machine Learning for Urban Land Cover Classification
Muntasir Tabasum, Tanpia Tasnim, Md. Ekramul Islam, Al Zadid Sultan Bin Habib
cs.LG · cs.AI · cs.CV
Quick verdict
Benchmarks 16 classical vs tabular-deep models on UCI ULC; CatBoost/RF win decisively; honest about limits but the abstract overclaims TDL parity.
Summary
The paper benchmarks 16 models on the UCI Urban Land Cover dataset (147 features, 9 classes, 168 train / 507 test samples): classical ML (logistic regression, decision tree, kNN, naive Bayes, SVM, random forest, GBM, AdaBoost, XGBoost, CatBoost, MLP) versus tabular deep learning (TabNet, TabTransformer, FT-Transformer, a 1D CNN, and TabSeq), under shared preprocessing and metrics (accuracy, macro precision/recall/F1, one-vs-rest macro AUC-ROC, confusion matrices). TDL models were trained with class-weighted cross-entropy and Optuna tuning; classical baselines used library defaults. CatBoost (82.25% accuracy) and random forest (81.66%) lead; naive Bayes reaches 77.51%; the best TDL models, TabTransformer (74.75%) and TabSeq (73.96%), trail by about 7.5 points. The authors report that weighting improves recall/F1, though comparing Tables 1 and 2 shows accuracy or F1 declining for four of five TDL models. They conclude tree ensembles remain strong defaults and TDL is complementary, with post-hoc binomial confidence intervals acknowledging wide overlap on the single split. Main limitations: one small dataset with a single split and single run, untuned classical baselines versus tuned TDL, an unpaired approximate significance test, and an abstract claim that TDL can match or exceed ensembles that no reported result demonstrates.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- CatBoost (82.25%) and Random Forest (81.66%) achieve the highest test accuracy on UCI ULC, reaffirming tree ensembles as strong defaults for heterogeneous, imbalanced tabular features.
- Attention-based TDL models (TabTransformer 74.75%, TabSeq 73.96%) are competitive with the best non-ensemble classical baselines on this dataset.
- Class-weighted cross-entropy for TDL generally improves minority-class recall/F1 without large AUC degradation.
- Accuracy alone obscures model differences; macro recall/F1 and AUC-ROC are essential for imbalanced ULC evaluation.
- TDL can match or exceed tree ensembles when non-linear feature interactions are significant and imbalance handling is effective (conditional claim).
Strengths
Broad, unified model coverage
16 models spanning linear, kernel, tree-ensemble, and neural paradigms are run in one pipeline with identical preprocessing, validation, and metrics (Sec. 3, Table 2); Appendix A.3 provides ROC curves and confusion matrices for every model.
Statistical hedging unusual for benchmark papers
Fig. 2 adds 95% binomial CIs to single-split accuracies, and the text explicitly concedes that overlapping intervals limit evidence for large gaps and that paired tests (McNemar, Friedman) could not be applied.
Reproducibility provisions are concrete
Imbalance-aware, multi-metric evaluation
Weaknesses
Abstract overclaims relative to results
Asymmetric tuning undermines the 'strong baselines' framing
Weak inferential basis
Weighting narrative conflicts with the tables
Tiny training regime unexamined
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Which model performs best for urban land cover classification on the UCI dataset?
In this paper, CatBoost leads with 82.25% test accuracy, followed by Random Forest at 81.66% and Naive Bayes at 77.51%. The best deep learning models, TabTransformer (74.75%) and TabSeq (73.96%), trail the top ensembles by roughly 7.5 accuracy points.
Does deep learning beat tree ensembles on tabular data according to this study?
No. On the UCI Urban Land Cover dataset, no tabular deep learning model matched CatBoost or Random Forest in any reported experiment, even with class-weighted loss. The abstract's suggestion that TDL can match or exceed ensembles is a conditional claim the reported results do not demonstrate; the authors position TDL as complementary rather than superior.
What data does the paper use and how large is it?
The UCI Machine Learning Repository's Urban Land Cover dataset: 147 tabular features extracted from a single high-resolution aerial image, covering 9 classes (asphalt, building, car, concrete, grass, pool, shadow, soil, tree), with only 168 training samples and 507 test samples, and noticeable class imbalance.
Did class-weighting improve the deep models' results?
The evidence is mixed. TabNet improved from 64.10% to 66.67% accuracy, but accuracy fell for the other four deep models. The paper emphasizes recall and F1 gains from weighted cross-entropy, yet its own tables show F1 declining for three of five models under weighting, so the benefit claim is only partially supported by the reported numbers.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are present only as captions in the extracted text; visual claims (ROC steepness, confusion-matrix structure) could not be verified against the actual plots.
- Minor extraction artifacts (stray footnote numerals, a duplicated appendix header) suggest slight layout noise, but all tables and key numbers appear intact.
- Review is based on the arXiv v1 workshop version; any subsequent peer review the work undergoes may differ in scope.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.