The AI Panel Template – Mathew's Library

01 Why a panel beats one answer

In a study published in the journal Science on October 22, 2026, Microsoft's AI got 83% of the hardest New England Journal of Medicine cases right. Generalist doctors working alone, with books and the internet, got 36%. It was a simulation, and no real patient was treated.

The AI isn't one chatbot. It works like a panel of doctors, except every doctor on it is an AI role. They go back and forth for several rounds before they commit to an answer. The researchers say a setup like this can help counter a classic mistake: locking onto the first guess.

The AI had to ask before it answered, and argue with itself before it committed.

That move isn't only for medicine. If you use AI for any decision that matters, give it roles, and make one of them disagree.

02 The four roles and their prompts

Paste the case first (a short summary of the decision), then run each role in turn. Replace the bracketed parts.

1. Propose

Here is the situation: [SHORT SUMMARY].
Propose the three most likely answers or options, ranked, with one line each on why.

2. Ask what's missing

Before choosing, what is the single most useful question or check that would separate these options?
Ask me for it. Do not answer yet.

3. Argue against it

Take the current favourite: [OPTION]. Argue against it as hard as you can.
What does it fail to explain, and which other option fits the facts better?

4. Ask what it costs

For each next step or check we are considering, what does it cost in money and time,
and is it worth it compared with deciding now? Recommend what to skip.

03 How to run it

Start from a short summary. In the study, the AI got only a short summary, like a patient walking in the door, and had to ask for everything else. Make it ask before it answers. Answer the questions role 2 asks, one at a time. Go several rounds. Run roles 1 to 4 again with the new answers until the favourite stops changing. Run it twice. Start a fresh chat and run the whole panel again. Where the roles still disagree is where to look harder.

04 What the panel can and can't do

Model On its own With the panel

Google's Gemini Flash (cheaper) 46% About 69%

OpenAI's GPT-5 79% 83%

With GPT-5, the panel added about four points, but it cut about $3,000 of estimated tests per case. The gap between the weak models and the strong ones was still bigger than anything the panel added.

The rule. The model sets the ceiling, and the process sets the bill. On a cheaper model, a panel setup is how you close part of the gap.

Not a doctor. The panel was never tested against specialists or teams of doctors, and its test with real patients at Beth Israel Deaconess in Boston is still to come. For your health, it is a second opinion at most.

05 The decision checklist

I gave it a short summary, not my conclusion. So it has to ask for the rest.

It asked its questions before it answered. Role 2 ran before any pick.

One role argued against the favourite. Role 3 ran on the final pick, not just the first one.

I know what each next step costs. Role 4 named what to skip.

I ran it twice. Where the two runs disagree, I looked harder.

Sources

TechCrunch: Microsoft's Mustafa Suleyman says its AI outperforms doctors on hard cases Unite.AI: Microsoft diagnostic AI outperforms generalist physicians in Science study Gizmodo: Microsoft says its new AI is more than twice as good at diagnosing diseases as doctors Microsoft AI: The path to medical superintelligence

Mathew's LibraryLibrary card

Free access

Keep reading free with your library card

One email opens this template and all 86 resources in Mathew's library. No account, no password.