What if the most intuitive idea in multi-agent AI — that having LLMs debate each other improves reasoning — turns out to be wrong? A wave of rigorous empirical studies published in 2025 suggests that multi-agent debate (MAD) may be far less effective than the field has assumed, and in some cases, actively harmful.
The Promise and the Problem
The concept is elegant: instead of relying on a single LLM to reason through a complex problem, have multiple agents independently generate answers, then debate their reasoning over several rounds until they converge on a better solution. Since Du et al. introduced the "Society of Minds" framework in 2023, dozens of variants have appeared — assigning agents distinct personas, structuring adversarial roles, or implementing judge-based arbitration.
But a fundamental question has gone under-examined: do these debates actually produce genuine deliberation, or are they simply a computationally expensive form of ensembling?
The Systematic Reckoning
Zhang, Cui, and Chen et al. (2025) deliver what may be the most sobering assessment to date. Their study evaluates five representative MAD methods across nine benchmarks using four different LLMs, structured around three evaluation dimensions: performance, efficiency, and robustness.
The findings challenge the prevailing narrative. Across 36 experimental scenarios (four models times nine benchmarks), none of the MAD methods achieved a win rate higher than 20% when compared to Chain-of-Thought prompting — a far simpler, single-agent baseline. The underperformance became even more pronounced when compared to Self-Consistency, particularly when controlling for the number of LLM calls. The authors state their position directly: existing MAD approaches are less effective than currently believed, even underperforming simple single-agent methods.
However, the paper offers more than criticism. It identifies a key factor that does consistently improve MAD performance: model heterogeneity. When agents are drawn from different LLM families rather than being copies of the same model, debate produces genuine epistemic diversity. The authors argue that the field must embrace model heterogeneity as a foundational design principle.
When Debate Becomes Harmful
Wynn, Satija, and Hadfield (2025) push the analysis further by examining the failure modes of multi-agent debate. Their experiments reveal an unsettling dynamic: debate can sometimes cause agents to shift from correct answers to incorrect ones.
The mechanism is what they call "answer corruption." When a weaker model participates in a debate alongside a stronger one, the weaker agent's flawed but persuasive reasoning can lead the stronger agent to abandon its correct answer. On CommonSenseQA, debate consistently degraded performance regardless of team composition. Moreover, the longer a debate continued, the worse performance could become — a finding that directly contradicts the intuition that more deliberation leads to better outcomes.
The study identifies three contributing factors: sequential revision bias (agents over-weight the most recent arguments), social conformity (agents tend to agree with the majority regardless of correctness), and sycophancy (agents defer to confident-sounding but incorrect reasoning).
Beyond Accuracy: Process-Level Analysis
Wu, Li, and Li (2025) take a different approach by examining not just whether debates produce correct outcomes, but whether agents engage in genuine reasoning processes. Using the Knight-Knave-Spy logic puzzle — a task with verifiable ground truth that requires nontrivial deductive reasoning — they systematically vary six structural and cognitive factors across 1,800 puzzle instances.
Their results reveal that intrinsic model reasoning strength is the dominant factor governing debate success, while structural parameters such as debate order and confidence visibility offer limited gains. At the process level, they observe that majority pressure suppresses independent correction — a dynamic that mirrors well-documented problems in human group decision-making.
The study proposes three desiderata for effective debate: inclusive deliberation (minority views must not be silenced by majority pressure), rationale over assertion (position changes should correlate with argument validity, not rhetorical confidence), and advancement of understanding (agents should correct initial errors through reasoning exchange, not merely aggregate independent guesses).
Open Questions
These findings raise uncomfortable questions for the multi-agent AI community. If debate does not reliably outperform simpler methods, what justifies its computational overhead? Under what precise conditions does debate add genuine value versus merely adding noise? And how do we design debate protocols that foster truth-seeking rather than social conformity?
The emerging consensus suggests that naive applications of multi-agent debate — particularly those using homogeneous agents from the same model — may function more like an echo chamber than a deliberative assembly. Progress will likely depend on embracing heterogeneity, developing process-level evaluation metrics, and drawing more carefully from the rich literature on human group decision-making.
Looking Forward
The trajectory of this research points toward a more nuanced understanding of when and how multi-agent collaboration helps. Rather than abandoning debate entirely, the field appears to be moving toward conditional frameworks — identifying the specific combinations of model diversity, task structure, and debate protocols that unlock genuine collective reasoning while avoiding the documented failure modes.
References
Zhang, H., Cui, Z., Chen, J., Wang, X., Zhang, Q., Wang, Z., Wu, D., & Hu, S. (2025). Stop overvaluing multi-agent debate — we must rethink evaluation and embrace model heterogeneity. arXiv preprint, arXiv:2502.08788.
Wynn, A., Satija, H., & Hadfield, G. K. (2025). Talk isn't always cheap: Understanding failure modes in multi-agent debate. arXiv preprint, arXiv:2509.05396.
Wu, H., Li, Z., & Li, L. (2025). Can LLM agents really debate? A controlled study of multi-agent debate in logical reasoning. arXiv preprint, arXiv:2511.07784.