Agentic Debate: Enhancing LLM Reasoning through Multi-Agent Consensus

Large Language Models (LLMs) are prone to hallucinations, logical slips, and cognitive biases. While techniques like Chain-of-Thought (CoT) help models break down reasoning steps, the single-model output is still susceptible to individual prompt variance.

The Agentic Debate pattern addresses this by placing multiple LLM agents in a structured debate. By critiquing, verifying, and debating their respective viewpoints over multiple rounds, agents resolve errors and arrive at a more robust consensus.

Agentic Debate Consensus System

The Debate Mechanism

A standard Agentic Debate workflow involves:

1. Problem Input: The user presents a complex question (e.g., a logic puzzle or math problem).
2. Independent Drafts: Multiple agents (with different system instructions or models) generate their initial answers.
3. Debate Rounds: Each agent is shown the answers generated by the other agents and asked to update their own answer in response.
4. Judge / Consensus: A final coordinator or judge LLM evaluates the debate and compiles the final answer.

Let A_i^{(r)} represent the answer of agent i in round r. In round r+1, the update is:

A_i^{(r+1)} = \text{LLM}\left(\text{Prompt}, A_1^{(r)}, A_2^{(r)}, \dots, A_n^{(r)}\right)

This iteration runs for a fixed number of rounds or until A_i^{(r+1)} \approx A_i^{(r)} for all agents.

Python Code: A Simple Agentic Debate System

Here is a Python script implementing a 2-agent debate loop:

import openai

def call_agent(model, system_prompt, user_prompt):
    client = openai.OpenAI()
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt}
        ],
        temperature=0.3
    )
    return response.choices[0].message.content

# Setup debate variables
question = "A bottle of water and a cup cost $1.10. The bottle costs $1.00 more than the cup. How much does the cup cost?"
agent_a_system = "You are Agent A, a logical reasoner. Solve the problem step-by-step."
agent_b_system = "You are Agent B, a critical mathematician. Analyze the problem step-by-step."

# Round 1: independent drafts
ans_a = call_agent("gpt-4o-mini", agent_a_system, question)
ans_b = call_agent("gpt-4o-mini", agent_b_system, question)
print("Round 1 - Agent A:", ans_a)
print("Round 1 - Agent B:", ans_b)

# Round 2: each agent sees the other's answer and may revise
def debate_prompt(question, own_answer, peer_answer):
    return (
        f"Question: {question}\n\n"
        f"Your previous answer:\n{own_answer}\n\n"
        f"Another agent answered:\n{peer_answer}\n\n"
        "Compare the two solutions. If the other agent found an error in your "
        "reasoning, correct it. Otherwise defend your answer. "
        "End with a line reading 'FINAL: <value>'."
    )

ans_a = call_agent("gpt-4o-mini", agent_a_system, debate_prompt(question, ans_a, ans_b))
ans_b = call_agent("gpt-4o-mini", agent_b_system, debate_prompt(question, ans_b, ans_a))
print("Round 2 - Agent A:", ans_a)
print("Round 2 - Agent B:", ans_b)

# Judge: a third call compiles the consensus
judge_system = "You are an impartial judge. Report the correct final answer only."
verdict = call_agent(
    "gpt-4o-mini",
    judge_system,
    f"Question: {question}\n\nAgent A said:\n{ans_a}\n\nAgent B said:\n{ans_b}"
)
print("Consensus:", verdict)

Why It Works

Agentic debate is effective for several reasons:

• Bias Reduction: Different agents highlight alternative interpretations, preventing the system from locking onto an initial incorrect assumption.
• Cross-Verification: Errors in calculation or logic are frequently caught and corrected by peer models.
• Self-Correction: Showing agents alternative answers encourages them to re-evaluate their logical paths.

Conclusion

The catch with debate is that agents converge whether or not they are converging on the truth. LLMs are agreeable by construction — show one model another model's confident answer and it will frequently abandon a correct solution to match. Run enough rounds and you reliably get consensus; you do not reliably get accuracy. The literature on multi-agent debate shows gains concentrated on problems with a verifiable answer (arithmetic, logic puzzles, code that either compiles or does not) and much thinner, sometimes negative, results on open-ended reasoning where there is nothing to anchor the agents against a ground truth.

That points at when not to use this. If you are running three copies of the same model with the same weights and near-identical prompts, you are mostly paying 3-5x the tokens to sample the same distribution repeatedly. The errors correlate, so all agents make the mistake together and then agree about it. Debate earns its cost when the agents differ in a way that matters — different model families, genuinely adversarial role prompts where one agent's job is to attack rather than answer, or one agent equipped with a tool (a calculator, an interpreter, a search index) that the others lack.

Before building a debate loop, run the cheap baseline: sample the same model five times at temperature 0.7 and take the majority answer. Self-consistency captures a large fraction of the benefit at a fraction of the complexity, and if it does not help on your task, debate probably will not either. Cap rounds at two or three regardless — beyond that, agents mostly restate themselves with rising confidence, which is the failure mode this pattern was supposed to prevent.