Isolated Sign Language Recognition for Icelandic Sign Language: Experiments in a Low-resource Setting
Finnur Ágúst Ingimundarson, Guðný Björk Þorvaldsdóttir, Mathias Müller, Sarah Ebling
cs.CL
Quick verdict
First ISLR study for Icelandic Sign Language: credible low-resource experiments with large transfer gains, but single-run results limit firm conclusions.
Summary
This paper reports the first isolated sign language recognition (ISLR) experiments for Icelandic Sign Language (ÍTM), using ÍTM SignWiki, a dictionary-derived dataset of 1,845 videos covering 849 classes, 86% of which have only two examples, making the full task effectively one-shot recognition across unseen signers. The authors compare two open-source frameworks, OpenHands and SPOTER, on three tasks of increasing vocabulary size (22, 117, and 849 classes), evaluate three pose estimators (MediaPipe, AlphaPose, SDPose), and test two cross-lingual transfer strategies. With ÍTM data alone, SPOTER beats OpenHands on all three tasks (e.g., 50.00 vs. 13.64% on Minimal; 8.36 vs. 1.41% on Full), and MediaPipe poses outperform the alternatives. Transfer brings the largest gains: SPOTER pretrained on ASL Citizen reaches 72.7%, 47.9%, and 22.6% test accuracy (+14–24 points, frozen-encoder finetuning), while multilingual OpenHands training over eight datasets in six additional sign languages lifts the Full task from 1.41% to 28.86%. Limitations are candidly stated: one run per configuration with no seeds or significance testing, a validation set covering only 79 classes and often the same signer as training, a confounded framework comparison, and a dataset not yet publicly released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- SPOTER outperforms OpenHands on all three ÍTM tasks when trained on ÍTM data alone (test accuracy 50.00/23.93/8.36 vs 13.64/9.40/1.41).
- Cross-lingual transfer yields the largest gains: ASL Citizen pretraining lifts SPOTER to 72.7/47.9/22.6% test accuracy (+14-24 pp, frozen-encoder finetuning), and OpenHands multilingual training lifts the Full task from 1.41% to 28.86%.
- MediaPipe poses beat AlphaPose and SDPose for ISLR in the reduced SPOTER format — the first such comparison for SLR.
- SPOTER learns effectively from very small training sets, corroborating Boháček and Hrúz (2022), and the results remain far from practical deployment.
Strengths
First ISLR work on ÍTM with honest resource accounting
Tables 1-2 document the full corpus (1,845 videos, 849 classes, 33 signers) and its central pathology: 732 classes (86.22%) have exactly two samples. Section 3.1 further discloses demographic skew (89.7% female signers, all white, one left-handed signer in 267/1,845 samples) that most dataset papers omit.
Signer-disjoint test design measures real generalization
Section 3.2 guarantees the test sample for each class comes from a signer unseen for that class, and the authors explicitly warn validation accuracy is misleading (79 of 849 classes, mostly same-signer as training) — e.g., SPOTER Minimal validation 72.72 vs test 50.00.
Restrained, hypothesis-generating framework comparison
Clearly attributed transfer results with ablation
First MediaPipe vs AlphaPose vs SDPose comparison for SLR
Weaknesses
No variance or significance reporting despite 'significantly' wording
Framework confounds named but not resolved
Weak model-selection alignment and a small internal inconsistency
Dataset not yet released; unified-vocabulary glosses unverified
Aging tooling and untested headline on smaller tasks
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper reports the first isolated sign language recognition (ISLR) experiments for Icelandic Sign Language (ÍTM), building on the first author's master's thesis. Using ÍTM SignWiki — a dictionary-derived corpus of 1,845 videos, 849 classes, and 33 signers (Tables 1-2), where 86.22% of classes have exactly two examples — the authors frame the full 849-class task as effectively one-shot recognition across unseen signers. Central claims: (1) SPOTER beats OpenHands on all three tasks trained on ÍTM alone (test accuracy 50.00/23.93/8.36 vs 13.64/9.40/1.41); (2) cross-lingual transfer dominates — ASL Citizen pretraining lifts SPOTER to 72.73/47.86/22.61 (+14-24 pp, frozen-encoder variant, Table 6), and OpenHands multilingual training over six additional sign languages lifts the Full task from 1.41 to 28.86 (Tables 7-8); (3) MediaPipe beats AlphaPose and SDPose for ISLR, the first such SLR comparison; (4) results are explicitly far from practical use.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is isolated sign language recognition (ISLR)?
It is the task of recognizing a single sign from a short video clip, treated as a classification problem — distinct from continuous sign language recognition of full sentences and from sign language translation. This paper applies ISLR to Icelandic Sign Language (ÍTM) for the first time.
How well does sign language recognition work for a very low-resource sign language?
On the ÍTM SignWiki data used here (1,845 videos, 849 sign classes, 86% of classes with only two examples), training only on ÍTM data tops out at 50% accuracy on a 22-class task and 8.36% on the full 849-class task. Transferring from better-resourced sign languages helps most: ASL pretraining reaches 72.7%, 47.9%, and 22.6% on the three tasks, and multilingual training over seven sign languages reaches 28.86% on the full task.
Is the Icelandic Sign Language (ÍTM) dataset from this paper publicly available?
Not at the time of writing — the paper states the dataset 'will soon be published'. The adapted OpenHands and SPOTER code is already released on GitHub (ZurichNLP/openhands-itm and ZurichNLP/spoter-pose), so the ÍTM results become reproducible once the data lands.
Which pose estimator works best for sign language recognition?
In this paper's reduced SPOTER input format, MediaPipe beat AlphaPose and SDPose on all three ÍTM tasks (e.g., 50.0 vs 45.5 vs 31.8% accuracy on the 22-class task). This agrees with earlier SLR-focused comparisons but differs from sign language translation studies, where alternatives to MediaPipe scored higher.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (left-/right-handed contrast example) cannot be assessed from the extracted text; only its caption context is available.
- Extraction otherwise appears complete (abstract through Appendix B), supporting high confidence; Table 4 packs ±augmentation pairs onto single lines, resolved via the legend.
- The underlying master's thesis (Ingimundarson, 2026) was not reviewed and may contain additional dataset detail.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.