Spectra: A Rules-Driven LLM Pipeline for Automated KYC Document Processing
Miray Wahib, Ethan Tran, Rea Mourad, Mira Muti, Nikita Dvornik
cs.AI
Quick verdict
Staged LLM pipeline + rules DB for KYC: 100% classification (n=21), 89.4% extraction on synthetic docs; good engineering, tiny eval, no baselines.
Summary
The paper presents Spectra, an AI-assisted KYC onboarding platform developed at Royal Bank of Canada. The authors restructure the traditional four-party onboarding process into a two-team workflow (Gathering and Approval) and automate document handling through a six-stage pipeline: file sanitization, prompt-injection detection, classification, deterministic schema lookup, extraction, and validation. Compliance policy is encoded in a PostgreSQL rules engine, enabling checklist generation without LLM calls and focused context injection per document. Classification and extraction use LlamaCloud; validation and injection detection use OpenAI o3-mini and GPT-4.1-mini. On 21 real KYC documents across 9 types, classification reached 100% accuracy (average confidence 99.3%), though the authors concede the set is small and clean. Extraction, tested on 15 synthetic documents with 141 annotated fields across 27 field types, achieved 89.4% exact-match accuracy (precision 86.6%, recall 94.7%, F1 90.5%); corporate resolutions scored only 50% and the powers_to_bind field 0%. One controlled case reportedly cut review items from 206 to 8 (96%). Main limitations: small and partly synthetic evaluation data, no baselines or ablations, unevaluated validation and injection-detection stages, a single-case efficiency claim, unreleased code and data, and unexamined tension between the 100%-confidence auto-approval threshold and 43.3% average extraction confidence.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Encoding KYC compliance policy as a queryable PostgreSQL rules engine enables deterministic checklist generation and focused per-document context injection without LLM calls.
- Decomposing document processing into six isolated stages achieves 100% classification accuracy on 21 real KYC documents and 89.4% exact-match extraction accuracy on 15 synthetic documents with 141 annotated fields.
- Auto-approving only 100%-confidence data points and flagging the rest reduced human review items from 206 to 8, a 96% reduction, in a controlled test case.
- Consolidating a four-party onboarding process into a two-team workflow eliminates intermediate handoffs that compound delays.
Strengths
Deterministic policy layer with per-requirement schemas
The PostgreSQL rules engine (Section 2.3, Appendix A) generates checklists without LLM calls, and join tables (must_contain, doc_data_requirements) let the same document type carry different extraction schemas depending on which requirement it fulfills — a genuinely thoughtful encoding for regulated workflows.
Principled pipeline decomposition
Section 2.2 motivates the six-stage split with concrete failure modes of monolithic prompting (context limits, error localization, retry granularity, traceability), and Stage 4 is deliberately LLM-free, keeping the audit trail traceable to policy clauses.
Credible, specific failure analysis
Security-aware ingestion
Self-acknowledged evaluation limits
Weaknesses
Headline numbers outrun the evidence
No baselines or ablations
Core safety stages unevaluated
Efficiency claim rests on a single case
Dataset description inconsistency and no release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Spectra is an AI-assisted KYC onboarding platform built at Royal Bank of Canada under the RBC Amplify program (arXiv 2609.25474, cs.AI). The paper advances four claims: (1) encoding compliance policy as a queryable PostgreSQL rules engine enables deterministic checklist generation and focused context injection without LLM calls; (2) a six-stage pipeline — file sanitization, injection detection, classification, schema lookup, extraction, validation — achieves 100% classification accuracy on real documents and 89.4% extraction accuracy; (3) confidence-gated review cut human review items from 206 to 8 (96%) in one controlled case; (4) consolidating a four-party process into a two-team workflow (Gathering/Approval) eliminates compounding handoff delays. Claims 1 and 2 are partially supported by the presented evidence; claim 3 rests on a single unnamed case; claim 4 is asserted without timing data. The case lifecycle (draft, pending_approval, in_review, approved/returned) and the four-eye principle are described in Sections 2.1 and 2.5.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does the Spectra KYC system achieve?
The paper reports 100% classification accuracy on 21 real KYC documents across 9 types (average confidence 99.3%), and 89.4% exact-match extraction accuracy on 15 synthetic documents containing 141 annotated fields across 27 field types (precision 86.6%, recall 94.7%, F1 90.5%). Both test sets are small, and the authors note the classification documents were clean PDFs and the extraction set was synthetic, so these figures may not transfer to noisier real-world documents.
Which AI models power Spectra?
Document classification and extraction run through LlamaCloud, LlamaIndex's managed document-processing service, while validation and prompt-injection detection use OpenAI's o3-mini and GPT-4.1-mini via an internal gateway. The schema-lookup stage is a deterministic PostgreSQL query with no LLM involved.
Does Spectra eliminate human reviewers in KYC onboarding?
No. Final approval stays with a human Approval Team, as banking regulations require a four-eye check. The system auto-approves only data points extracted with 100% confidence, flags everything else for review with clickable source evidence, and in one controlled test case reduced review items from 206 to 8 — a 96% reduction.
Is Spectra's code or data publicly available?
No. The paper describes an internal Royal Bank of Canada system built on the proprietary LlamaCloud service; neither the code, the rules-engine policy contents, the 21 real documents, nor the 15 synthetic evaluation documents are released, so the reported results cannot currently be independently reproduced.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 (workflow, pipeline, extraction comparison, ER diagram) appear only as captions in the extracted text and could not be inspected
- Neither evaluation corpus (21 real documents, 15 synthetic documents) is released, so per-type numbers cannot be verified
- The semantics of extraction 'confidence' (43.3% average) versus the 100%-confidence auto-approval threshold could not be reconciled from the text alone
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.