When Agents Disagree: Bayesian Backward Reasoning as a Label-Free Anchor for Multi-Agent Collective Decision-Making

Summary

The paper targets multi-agent LLM collective decision-making when agents disagree. It argues that voting, electoral rules, and LLM judges all aggregate forward evidence-to-label predictions whose errors are correlated. The authors construct a shared reverse posterior R(d|x) ∝ P(e|d)P(d|a) per instance via Bayesian backward reasoning — ordinal likelihood and context-activation maps elicited from the LLM, then replayed exactly on a small Bayesian network over the agents' top-k candidates. Jensen-Shannon divergence between each forward posterior and R yields a cross-path consistency signal driving three label-free, training-free aggregators: hard selection (MinJS), softmax reweighting (FwdJS, τ=5), and log-linear fusion with the anchor (LogLin, wR=0.2). On a 2,000-case DDXPlus diagnostic slice across five backbones (Qwen3-30B, Mistral-Small-3.1-24B, Llama-3.1-8B, GLM-4-32B, DeepSeek-V4-Flash), LogLin is best in every main-table setting, exceeding the strongest electoral baseline by 1.2–4.7 pp on the disagreement subset; MinJS beats random selection on every backbone. Notably, R is the weakest standalone predictor yet the best anchor: swapping in pool-mean or generic forward posteriors degrades both fusion heads on every backbone, and R collides least often with the forward consensus on the same wrong label (π 0.196–0.413). Optional two-stage labeled calibration adds up to 2.83 pp on LogLin. Main limitations: one synthetic closed-set dataset, no significance tests or confidence intervals for small margins, unreported pool sizes after availability filtering, and no released code.