ContraVis: Evidence-Grounded Visual Analytics for Contradiction Review in Legal Contracts
Luis Sante, Paula Lima, Mariana Rocha, Jorge Poco
cs.HC · cs.CL
Quick verdict
ContraVis: typed paragraph graph conditions LLM and anchors visual inspection; suggestive gains over standalone LLM but no stats on 4 contracts and 3 lawyers.
Summary
ContraVis is a visual analytics system for human-in-the-loop contradiction review in legal contracts. It models contracts as typed paragraph graphs whose edges are either explicit cross-references (regex-matched legal identifiers) or semantic links (all-MiniLM-L6-v2, cosine >= 0.80). The same graph conditions GPT-4.1-based contradiction detection, via 1-hop neighborhoods serialized into prompts, and structures the interface's four coordinated views (document, candidate list, neighborhood explorer, grounded chat). Synthetic contradictions (six LegalWiz types) were generated with GPT-4.1 and injected into four CUAD contracts (15-55 pages, 208-517 paragraphs; five per contract), filtered by NLI score >= 0.70, perplexity shift < 0.01, and manual review. In the controlled comparison, graph-conditioned detection recovered 4, 4, 3, and 4 of 5 injected contradictions versus 3, 3, 1, 1 for standalone LLM analysis, while surfacing more false positives (2-4 vs 1-2). A formative study with three Brazilian contract lawyers (30-minute think-aloud sessions) found positive usability and usefulness ratings, though adoption intent diverged, and rule-exception relations were sometimes misclassified as contradictions. Main limitations: only 20 injected contradictions across four documents with no statistical testing, a single baseline (no full-document long-context or RAG baseline), potential circularity between generation and detection (same model, graph-conditioned generation prompts), and a small, single-firm expert study. Code, prompts, and evaluation material are reported as publicly available.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A typed paragraph graph can serve simultaneously as LLM conditioning context and as the analyst's interactive representation, aligning model context with human inspection (Sec. I, IV-A).
- Graph-conditioned GPT-4.1 detection recovers more injected contradictions than standalone LLM analysis as contract length grows: 4/4/3/4 vs 3/3/1/1 of 5 across four contracts (Table II).
- Graph conditioning surfaces additional non-injected candidates (FPs 2-4 vs 1-2 per contract), which is acceptable because missed contradictions cost more than extra review in legal practice (Sec. VI-A).
- A formative study with three contract lawyers found that in-context evidence comparison supported contradiction validation, while rule-exception relations were sometimes misclassified as contradictions (Sec. VII).
- Contradiction review in contracts is a structure-aware, human-in-the-loop analytical task rather than clause-pair classification (Sec. I).
Strengths
Dual-role graph design
The typed paragraph graph conditions LLM context (1-hop neighborhoods serialized into GPT-4.1 prompts, App. D) and structures analyst inspection (four coordinated views, Sec. V) — a concrete answer to opaque RAG context; views map cleanly onto tasks T1-T5 (Table I).
Disciplined synthetic-data pipeline
Contradiction injection is gated by NLI score >= 0.70 (nli-deberta-v3-base), per-word perplexity increase < 0.01 (GPT-2), and manual review; the detector is blind to insertion positions (Sec. IV-B, App. C) — above-average care for a synthetic benchmark.
Consistent ablation result with cost anchors
Candid limitations section
Actionable formative finding
Weaknesses
No inferential statistics on a tiny sample
Single, weakest baseline
Generator-detector circularity
Matching and precision cost under-specified
Thin expert study
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ContraVis (arXiv 2609.27014, cs.HC/cs.CL) is a visual analytics system for human-in-the-loop contradiction review in legal contracts, framed by the authors as an early-stage design study (Sec. I, citing Sedlmair et al.). Stated contributions: (1) problem characterization and task abstraction — requirements R1-R4 elicited from two contract lawyers, abstracted into tasks T1-T5 (Table I); (2) a typed paragraph graph with a dual role: the same structure — referential edges from regex-matched legal identifiers (e.g., 'Section 3.2') plus semantic edges from all-MiniLM-L6-v2 embeddings at cosine >= 0.80 — conditions GPT-4.1 detection via serialized 1-hop neighborhoods and structures the analyst-facing interface; (3) the system with four coordinated views (Document, Contradiction Analysis, Related Paragraph Explorer, Grounded Legal Chat), a controlled comparison, and a formative expert study. The central empirical claim is that graph-conditioned reasoning recovers more injected contradictions than standalone LLM analysis as contract length grows, while surfacing more candidates for validation.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ContraVis?
ContraVis is a visual analytics system that helps lawyers find and validate contradictions in legal contracts. It represents a contract as a graph of paragraphs linked by explicit cross-references and semantic similarity, uses that graph to steer GPT-4.1-based contradiction detection, and shows the same structure to the analyst through coordinated views: document, candidate list, neighborhood explorer, and a grounded chat.
Did graph conditioning actually improve contradiction detection?
In the paper's controlled comparison on four CUAD contracts (15-55 pages, five injected synthetic contradictions each), graph-conditioned GPT-4.1 recovered 4, 4, 3 and 4 of 5 contradictions while the same model without the graph recovered 3, 3, 1 and 1. The graph version also produced more false positives (2-4 vs 1-2 per contract). With only 20 instances and no statistical tests, this is suggestive rather than conclusive.
Were the contradictions in the study real?
No. They were synthetic: GPT-4.1 generated them across six categories (temporal, numerical, authority, process, policy reversal, specificity) and injected them into real CUAD contracts, with filtering by an NLI model (score >= 0.70), a perplexity-based fluency check, and manual review. The authors note that no public benchmark of naturally occurring document-level contract contradictions exists.
Was the system tested with real lawyers?
Yes, in a small formative study: three Brazilian contract lawyers completed roughly 30-minute think-aloud sessions. Usability and usefulness ratings were positive, but only one of the three said they would use the tool in their own workflow, and rule-exception relationships were sometimes misclassified as contradictions.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 are not analyzable from the extracted text; interface and workflow claims rest on captions and body text only.
- Table II values were parsed from a flattened text stream; column alignment (LLM TP/FP vs ContraVis TP/FP) is consistent with the narrative but could not be visually verified.
- Appendices B-F are partially embedded; some construction and procedure details may be truncated.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.