#36 AI agents and MCP 5.3 min
Multi-Agent Debate: Do Two Agents Beat One? A Test
Two agents argued, and did no better than a plain vote, for more calls.
Test the answer reader
Copy the code folder first. No model is needed for this. In the code folder start Python and run from agents import number. Predict, then print number for these strings: 'ANSWER: 5', 'ANSWER: **5**', 'The answer is 5', 'ANSWER: 5 then ANSWER: 7', and 'ANSWER: 4.5'. I got 5, 5, None, 7 and 4.
A hint
The pattern looks for ANSWER: in capitals, then digits, and takes the last match.
Count the calls by hand
Read the vote and debate functions in agents.py. Without running anything, work out how many model calls a vote uses for one question, and how many a debate uses when the agents agree and when they disagree. Then work out the call total for 6 questions if all six debates ended in disagreement.
A hint
The video's own totals were 18 for the vote and 26 for the debate, with two disagreements.
Add a seventh question
In questions.py add a seventh tuple with a question that has one whole number as its answer, for example 'What is 12 times 12?' with the answer '144'. Predict the new call counts in the three lines of compare.py. Running it needs Ollama with llama3.2 as in the video, so treat the right answer counts as a guess.
A hint
The call counts do not depend on whether the model is right, except for the judge call in a debate.
Spot the unfair comparison
The video says a debate should be compared with the cheapest thing that could work. Read compare.py and name the two baselines it uses, then write two sentences about what you would add to the test before believing a result, using the limits the video states.
A hint
The video mentions more questions, a larger model, and a check in code.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Six questions, one small model (llama3.2), one fixed set of seeds; a difference of one question proves nothing about debate against a vote. Replies are capped at 90 tokens, and a reply with no final number counts as wrong. Only the call counts are firm.
Up next in AI agents and MCP
More in AI agents and MCP
- #01 Local AI Agent in Python: No API Key (Ollama)
- #12 Multi-Agent Patterns in Plain Python: Pipeline, Router, Loop
- #22 OpenAI Agents SDK vs LangGraph vs Plain Python
- #23 Agent Memory: What to Keep and What to Forget
- #24 Approval Gates for Risky Agent Actions
- #25 Tool Errors: Retries, Fallbacks and Made-Up Answers
Related videos
- #64 Chain of Thought: When Step by Step Helps, and Its Cost LLM engineering and evals
- #88 Indirect Prompt Injection: A Note That Gave Orders AI security and governance
- #39 Context Engineering: What to Keep From Chat History AI agents and MCP
- #293 RAG for Beginners: The Idea, Then 37 Lines of Python RAG and vector search
