Viveka-Insight: a cross-lingual concept graph and citation-grounded retrieval resource over the complete works of Swami Vivekananda in English and Bengali
cs.CL · cs.DL · cs.IR
Quick verdict
Resource paper links non-parallel EN/BN Vivekananda corpora via an LLM concept graph; admirably honest evaluation but core-layer precision is just 0.60.
Summary
The paper presents Viveka-Insight, a bilingual resource and open-source pipeline covering the complete works of Swami Vivekananda: the nine-volume English Complete Works and the ten-volume Bengali Vani o Rachana (about 15M characters), two related but non-parallel corpora. Four layers are released: a structure-preserving parse (32,694 paragraphs, 168,842 sentences) with per-paragraph anchors deep-linking to the published editions; an LLM-built concept graph of 8,362 language-agnostic concepts whose canonical English labels act as string-equality cross-lingual keys (87,518 paragraph–concept and 55,872 concept–concept edges); a 60,850-form bilingual alias inventory; and 200 edges judged by three annotators with all individual judgments released. Evaluation comprises known-item cross-lingual retrieval over 194 LLM-verified rendered lecture pairs (Recall@10 0.86 in both directions after fusion and reranking), a 30-question citation-integrity audit (0 fabricated markers among 136), and a human study placing strict-consensus concept precision at 0.60 (κ=0.61), rising to 0.71 at extractor weight ≥0.8 but dropping to 0.54 on Bengali versus 0.68 on English. Main limitations: the concept layer's precision is only moderate, lowest in the Bengali half that performs the cross-lingual bridging; answer faithfulness is unmeasured beyond structural citation validity; the retrieval benchmark shares its embedder with pair discovery; and the QA audit is small.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Canonical English, lowercase, hyphenated concept labels act as string-equality keys that link Bengali and English passages across two non-parallel corpora with no parallel data.
- Fused multi-granular retrieval with cross-encoder reranking achieves Recall@10 of 0.86 in both directions over 194 verified rendered-lecture pairs.
- Strict two-annotator consensus places concept-extraction precision at 0.60 (κ=0.61), rising to 0.71 at extractor weight ≥0.8 while retaining 98.1% of concept-bearing paragraphs.
- Extraction is markedly weaker in Bengali (0.54 vs 0.68 English), precisely the half of the graph that cross-lingual access depends on.
- Citation-grounded generation with post-hoc marker validation produced zero fabricated citations across 136 markers in a 30-question audit.
Strengths
Honest calibration of the central artifact
Headline concept precision is reported as strict-consensus 0.60 with the graph framed as a broad, imperfect expansion layer (§4.1, §9.3); the Bengali deficit (0.54 vs 0.68) is located exactly where cross-lingual access depends, and all per-annotator judgments are released (§5.2, §9.3).
Statistical reporting above resource-paper norms
Wilson 95% CIs on all precision figures; Cohen's κ=0.607 [0.482–0.720] plus Gwet's AC1=0.701; McNemar exact p=0.23 for annotator symmetry; Fisher exact p=0.023 (threshold) and p=0.043 (language gap); Fleiss κ=0.49 on the Bengali three-rater panel (Tables 8–9).
Negative results reported and claims scoped accordingly
Strong reproducibility posture
Verifiable internal numeric consistency
Weaknesses
Moderate precision in the load-bearing half
Answer faithfulness unmeasured
Known-item benchmark circularity
'Calibrated' overstates a weak association
Thin and partially dependent annotation panel
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a resource-and-system paper: it releases bilingual, non-parallel corpus infrastructure (nine English volumes, ten Bengali volumes, ~15M characters of Vivekananda's works) plus the pipeline that builds it. Four artifacts ship: a structure-preserving parse with per-paragraph deep-link anchors (32,694 paragraphs, 168,842 sentences; Tables 1/4); an LLM-built concept graph of 8,362 concepts with 87,518 relation-typed paragraph–concept edges and 55,872 concept–concept edges (Tables 2–3); a 60,850-form bilingual alias inventory; and a 200-edge human-annotated set with all per-annotator judgments. The central claim is architectural: canonical English, lowercase, hyphenated concept labels act as a string-equality key linking Bengali and English passages with no parallel data. Supporting claims are quantified: known-item cross-lingual retrieval at Recall@10 0.86 both directions (Table 6), zero fabricated citations among 136 markers (Table 7), and concept-extraction precision of 0.60 strict-consensus, 0.71 at weight ≥0.8, 0.54 Bengali vs 0.68 English (Tables 8–9).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Viveka-Insight?
An open bilingual resource and pipeline that indexes the complete works of Swami Vivekananda — the nine-volume English Complete Works and the ten-volume Bengali Vani o Rachana, about 15 million characters — into a structured, sentence-segmented edition with a cross-lingual concept graph, a bilingual alias lexicon, and citation-grounded question answering that deep-links every claim back to the published source text.
How accurate is the concept graph?
A human evaluation of 200 sampled edges put strict-consensus precision at 0.60 (Cohen's κ=0.61). Filtering to high extractor confidence (weight ≥0.8) raises precision to 0.71 while keeping 98% of concept-bearing paragraphs. Precision is lower on Bengali (0.54) than English (0.68), and the authors themselves caution against citing the graph as a scholarly index.
Does it work across languages without translations?
Yes — each concept carries one canonical English label shared by both languages, so a Bengali paragraph and an English passage attach to the same node without any parallel data. On 194 verified rendered-lecture pairs, cross-lingual known-item retrieval reached Recall@10 of 0.86 in both directions after fusion and reranking.
Is the code and data available?
Yes. The pipeline is MIT-licensed on GitHub (tamalrkm/viveka-insight); the derived data layers — parse, concept graph, alias inventory, entity layer, and the 200-item annotated set with per-annotator judgments — are CC BY 4.0 on Zenodo (DOI 10.5281/zenodo.21669973). The source editions themselves are not redistributed; the resource uses anchors to link back to the publicly hosted versions.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (retrieval architecture) and the Streamlit interface are available only as caption/description; UI behaviour and figures could not be visually assessed.
- External artifacts (GitHub repository, Zenodo archive) were not inspected; availability and contents of released files are taken from the paper's statements.
- The abstract in the supplied metadata is truncated mid-sentence, but the complete abstract appears in the extracted full text.
- Alias counts differ between §4.1 (60,838) and abstract/Tables 2–4 (60,850); the discrepancy cannot be resolved from the text alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.