Real-Time Video Anomaly Detection Using YOLO Pose Estimation and CLIP-Based Semantic Scoring
Vanodhya G. Warnasooriya, Amir Hajian, Watchara Ruangsang, Supavadee Aramvith
cs.CV · cs.AI · eess.IV
Quick verdict
Two-stage YOLO-pose + CLIP-prompt VAD hits 51 FPS (3.36× speedup); throughput claim solid, but identical baseline AUROCs in Table I undermine accuracy claims.
Summary
The paper presents a two-stage real-time video anomaly detection pipeline: YOLO v11n-pose detects persons and regresses 17 COCO keypoints, then each padded person crop is encoded by a frozen CLIP ViT-B/32 and scored by the maximum cosine similarity against four fixed text prompts (lying, falling, fighting, sitting on the floor). Frame scores are Gaussian-smoothed and thresholded at 0.7. On CUHK Avenue, ShanghaiTech Campus, and a custom 40-video indoor dataset from Chulalongkorn University, reported frame-level AUROC is 89.26%, 70.26%, and 84.13%, with end-to-end throughput of 51.39 FPS on a Titan XP — a 3.36× speedup over a reimplemented multi-feature baseline (15.32 FPS) — and a five-camera live deployment at ~10 FPS per feed. The main stated limitation is prompt coverage: anomalies outside the four categories (cycling, vehicle intrusion) depress ShanghaiTech performance. Critically, the baseline and proposed methods report byte-identical AUROC on Avenue and ShanghaiTech, which the authors attribute to a shared detector — an explanation that does not hold, since the scoring stages differ. The threshold is tuned 'to maximize AUROC,' a threshold-independent metric, and neither code nor the custom dataset is released.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Direct CLIP text-image cosine similarity can replace optical flow, standalone pose estimation, and GMM/kNN density scoring while maintaining accuracy.
- The two-stage pipeline sustains 51.39 FPS end-to-end on a Titan XP, a 3.36× speedup over a reimplemented multi-feature baseline at 15.32 FPS.
- Frame-level AUROC reaches 89.26% on CUHK Avenue, 70.26% on ShanghaiTech Campus, and 84.13% on the custom CU Indoor dataset (+2.01 over the baseline).
- Four fixed natural-language prompts enable person-level anomaly categorization without labeled anomaly data for scoring, and the system ran stably on five live CCTV feeds at ~10 FPS each.
Strengths
Genuine, honestly decomposed speedup
Table III breaks throughput per component (YOLO 87.76 FPS, CLIP 124.04 FPS vs baseline Mask R-CNN 23.41 FPS) and explicitly attributes the gap between component rates and the 51.39 FPS end-to-end figure to stream decode, CPU-GPU transfer, variable CLIP batch sizes, and overlay rendering — overheads usually hidden in such comparisons.
Simple, deployable design with real field evidence
Two checkpoints and an editable prompt set (Sec. VI) make adaptation retraining-free; deployment on five live CCTV feeds at ~10 FPS each corroborates the real-time claim beyond isolated benchmarks.
Honest scoping of 'zero-shot' and limitations
New labeled indoor anomaly resource with detection metrics
Weaknesses
Internally inconsistent headline comparison (Table I)
Statistically incoherent threshold protocol
Comparison set omits the closest competitors
No ablations and no trivial baseline
Reproducibility gaps and an unexplained operating point
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv:2608.31074, IEEE copyright notice) presents a two-stage, person-centric video anomaly detection (VAD) system for fixed indoor CCTV. Stage 1 runs YOLO v11n-pose for person detection plus 17-keypoint COCO-topology regression; Stage 2 encodes each person crop (padded δ=10 px, resized 224×224) with a frozen CLIP ViT-B/32 and scores it by the maximum cosine similarity against four fixed prompts — lying, falling, fighting, sitting on the floor (Sec. III, Eqs. 2–3). Frame scores are max-aggregated over persons, Gaussian-smoothed (σ=5, w=15), and thresholded at θ=0.7. Central claims: (1) direct CLIP scoring replaces optical flow + standalone pose estimation + density scoring at comparable accuracy; (2) 51.39 FPS end-to-end on a Titan XP, a 3.36× speedup over a reimplemented multi-feature baseline (15.32 FPS, Table III); (3) frame-level AUROC of 89.26% (Avenue), 70.26% (ShanghaiTech), 84.13% (custom CU Indoor, +2.01 over baseline); (4) stable live deployment on five CCTV feeds at ~10 FPS/feed (Sec. VI).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy and speed does this YOLO + CLIP anomaly detection method achieve?
The paper reports frame-level AUROC of 89.26% on CUHK Avenue, 70.26% on ShanghaiTech Campus, and 84.13% on the authors' custom CU Indoor Anomaly dataset, with end-to-end throughput of 51.39 FPS (about 3.36× faster than their reimplemented multi-feature baseline at 15.32 FPS) on an NVIDIA Titan XP GPU. A live deployment on five CCTV feeds ran at roughly 10 FPS per feed.
Is the method really zero-shot?
Only partially. The CLIP ViT-B/32 scoring stage is zero-shot — it scores person crops against four fixed text prompts (lying, falling, fighting, sitting on the floor) without labeled anomaly examples. However, the YOLO v11n-pose detector is supervised and was fine-tuned on 1,154 manually labeled frames from the authors' custom indoor dataset, so the full system is not label-free.
Which anomalies can the system detect, and what are its limits?
It detects four prompt-defined person-level categories: lying on the floor, falling down, fighting, and sitting on the floor. Anomalies outside these categories — such as cycling, skateboarding, or vehicle intrusion — fall outside prompt coverage; this is the authors' stated reason for the lower 70.26% AUROC on ShanghaiTech Campus.
Is the code or the CU Indoor Anomaly dataset publicly available?
The paper does not provide any code or dataset release link. The prompts, hyperparameters, and hardware (Titan XP) are described in the text, but CU Indoor Anomaly's availability is not stated, so the results on that dataset are not externally verifiable.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure contents (Fig. 1 pipeline diagram, Fig. 2 GUI/close-up outputs) were assessed from captions and in-text descriptions only; rendered images are not inspectable in the text extraction.
- The extracted text is complete from title through reference [14]; no truncation was observed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.