Lie to me: Detecting Managerial Evasiveness in Earnings Calls via Conversational Audio Encoders
cs.LG · cs.CE
Quick verdict
Claims frozen Moshi audio predicts SEC late filings (AUROC~0.89) beyond a weak text rubric; ablations solid but text baseline and construct validity need work.
Summary
The paper proposes a two-branch late-fusion framework that predicts adverse SEC outcomes—primarily NT 10-K/NT 10-Q late-filing notices—from earnings-call Q&A. The text branch runs an LLM-as-a-judge (Moonshot kimi-k2.6) over up to eight analyst–manager exchanges with a 45-tag binary rubric, producing a 139-dimensional call vector. The audio branch extracts frozen Moshi hidden states (last layer of the 32-layer temporal transformer, 12.5 Hz, subsampled to 256 frames) from dual-stream Q&A audio and reads them with a lightweight BiLSTM-attention head. On 1,039 calls (212 late-filing positives, 548 firms) under firm-grouped 5-fold cross-validation, the audio branch reaches AUROC 0.902 on the designated seed (0.847±0.023 across seeds), fusion 0.897 (0.893±0.002), versus 0.553 for the text rubric and 0.898 for mean-pooled Moshi frames under the same nonlinear head; reassigning audio across calls drops AUROC to 0.483. The authors candidly report that fusion does not beat audio alone and the text branch adds nothing. Key limitations: the near-chance text baseline means the 'beyond lexical content' claim rests on one weak representation; captions were preferentially retrieved for adverse-event calls, so controls are not randomly sampled; no financial controls exclude a general-distress interpretation; the abstract's duration-alone figure (0.71) never appears in the body or tables; and no code, confidence intervals, or significance tests are reported.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A frozen Moshi conversational encoder read by a lightweight DeepVoice-style head predicts SEC late filings at firm-grouped CV AUROC 0.902 (seed 0), far above the 45-tag LLM text rubric (0.553).
- The audio signal is incremental to the text rubric and call length, and collapses to chance (AUROC 0.483) when frame sequences are reassigned across calls, locating the signal in the specific call's delivery.
- Most of the audio signal resides in the frozen Moshi representation itself (mean pooling 0.898) rather than in sequential modeling (0.902).
- Conversational audio encodes managerial evasiveness beyond lexical content, yielding an early-warning signal of adverse SEC outcomes.
Strengths
Decisive negative controls on the audio input
Table 2's cross-call reassignment drops AUROC to 0.483 (chance), ruling out corpus-wide priors; within-call frame shuffling (0.830) vs mean pooling of original frames (0.898) further separates frame identity from ordering.
Leakage-aware evaluation design
Firm-grouped 5-fold CV by SEC CIK (Sec. 4.1) prevents same-firm calls spanning train/test folds, and supervision is extrinsic: EDGAR labels within 24 months, never seen by the judge (Sec. 3.1, 3.3).
Classifier-family controls isolate the representation
Unusually honest reporting of nulls and variance
Well-motivated encoder choice
Weaknesses
Near-chance text branch undermines the headline framing
Construct 'evasiveness' is never measured
Non-random control sampling
No financial or call-format controls
Abstract–body inconsistency and missing uncertainty quantification
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.13893, four-page conference format) frames managerial evasiveness in earnings-call Q&A as a predictor of extrinsic SEC events, primarily NT 10-K/NT 10-Q late-filing notices under Rule 12b-25, with 8-K Item 4.02 restatements as a secondary outcome. It proposes a two-branch late-fusion model: an LLM-as-a-judge (Moonshot kimi-k2.6) converts up to eight analyst–manager exchanges into a 139-float vector via a 45-tag binary rubric (Sec. 3.3), while a frozen Moshi dialogue encoder supplies per-frame hidden states (RT×4096, 12.5 Hz, subsampled to L=256) read by a DeepVoice-style BiLSTM-attention head (Sec. 3.4). Central claims: (1) audio predicts late filings far better than the text rubric (0.902 vs 0.553, Table 1); (2) the signal is incremental to text and call length; (3) it is call-specific, vanishing to 0.483 under cross-call reassignment (Table 2).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does the model achieve in this paper?
Under firm-grouped 5-fold cross-validation on 1,039 earnings calls (212 followed by SEC late-filing notices), the audio-only model reaches AUROC 0.902 on the designated seed (0.847±0.023 across three seeds) and text+audio fusion reaches 0.897 (0.893±0.002), versus 0.553 for the LLM text rubric alone and 0.898 for simply averaging the same Moshi audio features.
What data does the paper use?
1,039 publicly posted earnings-call videos from AlphaStreet's YouTube channel across 548 firms, linked via SEC CIK numbers to EDGAR records: NT 10-K/NT 10-Q late-filing notices within 24 months after an estimated call date (fiscal-quarter end plus 35 days), with 8-K Item 4.02 restatements as a secondary outcome.
Is code or data available for this paper?
The paper does not mention any code, prompt, or dataset release. The Moshi encoder is publicly available, but the assembled 1,039-call sample, the auto-captions, and the kimi-k2.6 judge outputs are not, so exact replication is currently not possible from the paper alone.
Does the model actually detect lying or evasiveness?
Not directly. The model is trained to predict SEC late-filing notices, and the authors interpret the audio signal as managerial evasiveness. However, the paper provides no human evasiveness labels, no financial-distress controls, and no probing of what the audio features encode, so general corporate distress acoustics remain a plausible alternative explanation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is present only via its caption in the extracted text; the diagram's visual content is not available, though the rubric grouping it depicts is fully described in Sec. 3.3, so no reviewed number depends on the figure.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.