ChatGPT vs Claude for research: a source-only critique test

GPT, Claude, and Gemini found the same objections to our fictional product plan. The useful difference in EVA's November 2, 2026 test was Claude's extra paragraph: it separated the founder's unproven adoption assumption and warned that each objection rested on a single excerpt.

What did this ChatGPT vs Claude for research test measure?

We tested whether models could criticize a fictional product plan without going beyond supplied excerpts. Nobody was asked to search the web or find research papers.

The November 2 test ran in EVA Multi Chat with GPT-6.1 Sol, Claude Opus 5.5, and Gemini 4 Argon, using the names shown in EVA. The prompt and responses below are copied from the test record without edits.

NoteRelay, the fictional product, would upload meeting recordings to its cloud and keep them indefinitely. Every attendee would need an account. The founder wanted to launch with an annual paid plan, without a trial, and assumed teams would adopt immediately.

We kept the source packet small on purpose. A synthetic buyer objected to cloud storage, a synthetic pilot note described signup friction, and a synthetic procurement note required a pilot before an annual commitment. None of these excerpts came from real customers.

A source-only critique evaluates a plan against the evidence supplied in the prompt. An objection that needs outside evidence should be labeled as an assumption or left out.

This was the complete prompt, unchanged:

Fictional founder research exercise. All source excerpts below are synthetic, not real interviews or market statistics. Plan: Launch NoteRelay, a meeting-summary tool for small consulting teams. It uploads raw recordings to our cloud, keeps them indefinitely, requires every attendee to create an account, and launches as a paid annual plan without a trial. The founder assumes teams will adopt it immediately. S1, synthetic buyer interview: "We cannot send client recordings to a third-party server; deletion after processing is mandatory." S2, synthetic pilot note: "Guests refused another signup; the organizer needs a shareable summary with no attendee account." S3, synthetic procurement note: "We need a hands-on pilot before approving any annual commitment." Find the three strongest objections to this plan, citing the sources. Use only S1-S3, no external facts. Return exactly three numbered objections, each at most 25 words and ending with its source ID. Distinguish supported objections from unproven assumptions.

Did the models find the same supported objections?

Yes. Each model identified the storage objection from S1, the account requirement from S2, and the annual commitment problem from S3.

Counting objections alone, you can't pick a winner.

Source: EVA test record, November 2, 2026.

All objections stayed within the prompt's word limit, according to the record's whitespace-based count. The transcript below preserves the response text; list numbering was visible in the screenshots but isn't present in the captured text.

Where did the answers differ?

The handling of the adoption assumption differed. Read the final objection in each response, then Claude's extra paragraph.

GPT-6.1 Sol

Supported objection: Cloud uploads and indefinite retention conflict with the buyer’s prohibition on third-party recording storage and mandatory deletion after processing. S1

Supported objection: Mandatory attendee accounts conflict with guests’ signup refusal and the organizer’s need for account-free summary sharing. S2

Supported objection: Annual commitment without a hands-on pilot conflicts with procurement requirements; the founder’s immediate-adoption assumption remains unproven. S3

GPT's final sentence does call the assumption unproven. It just leaves that comment attached to the procurement objection. A reader can separate the ideas, but the answer doesn't do that work in a dedicated section.

Claude Opus 5.5

Supported: Sending client recordings to NoteRelay's cloud and keeping them indefinitely breaks the buyer's rules against third-party servers and requiring deletion. S1
Supported: Guests in the pilot refused to make accounts. Organizers need summaries they can share without anyone else signing up. S2
Supported: Launching only as a paid annual plan with no trial clashes with procurement's need for a hands-on pilot before any annual commitment. S3

Unproven assumptions: The founder's belief that teams will adopt it right away has no support in S1–S3, and all three sources point to barriers instead. Each objection also comes from just one synthetic excerpt. That means the objections are backed by the sources, but we don't know how common they are across small consulting teams.

A founder should copy that last sentence into their notes. The excerpt gives you a reason to question the plan, but no basis for saying how common the objection is. For keeping assumptions apart from evidence, Claude's answer was the most useful in this run.

Gemini 4 Argon

Supported objection: Uploading and indefinitely storing recordings on your cloud conflicts with buyers who forbid third-party server transfers and mandate post-processing deletion. S1

Supported objection: Requiring every attendee to register contradicts the unproven immediate-adoption assumption, as guests refused signups and organizers need account-free shareable summaries. S2

Supported objection: Launching as a paid annual plan without a trial contradicts the unproven immediate-adoption assumption, as procurement requires a hands-on pilot first. S3

Gemini identified the account and procurement barriers. But S2 and S3 don't discuss the founder's expectation of immediate adoption. The model inferred that link. It may be reasonable, but the sources don't state it. The storage objection also says "buyers," although S1 is a single synthetic buyer excerpt.

Why does staying inside the sources matter?

In this exercise, "guests refused another signup" supports an account-friction objection; it doesn't tell you how the whole market will behave.

A source label identifies an excerpt to check. The label alone doesn't establish that the excerpt supports every clause of the objection.

Read the sentence before the label. Does its subject still match the source? Has a single interview turned into a claim about buyers as a group? Has the model added a causal explanation that the note never gave?

The guide to AI model disagreement provides a broader way to decide which differences need verification. Here, the disputed point is small enough to check against the prompt itself. You don't need another generated critique to establish what S2 says.

How can you repeat the critique on your own plan?

Give each model the same plan and excerpts, then review every objection against its cited source. Decide what counts as an acceptable answer before reading the responses.

For the NoteRelay plan, start with the attendee account requirement. Put that decision next to the pilot note about guests refusing signup. Keep the source labels when you paste your own notes, and don't shorten an excerpt so much that you lose what the speaker meant.

Replace the plan and excerpts in this adaptation of the recorded prompt:

Plan: [Paste the proposed decision and its assumptions.]
S1: [Paste a relevant excerpt.]
S2: [Paste a relevant excerpt.]
S3: [Paste a relevant excerpt.]

Find the three strongest objections to this plan, citing the sources.
Use only S1-S3, no external facts.
Return exactly three numbered objections, each at most 25 words
and ending with its source ID.
Distinguish supported objections from unproven assumptions.

Keep an untouched copy of each response before asking follow-up questions. Otherwise, you'll end up comparing an original answer with an answer that has already received corrections.

For a source-only critique, prefer the answer whose objections you can trace and whose unsupported assumptions you can identify. Fluent prose isn't an acceptance criterion.

If the answers agree on the blockers, compare the remaining uncertainty rather than asking for a more dramatic critique. A useful next action might simply be another interview about the annual commitment. That would collect missing evidence instead of recycling the same packet.

What can't this test tell you about research quality?

It can't identify a general research winner or predict performance on a different source packet. This was a single recorded prompt using synthetic material, not a study of research reliability.

We didn't compare native ChatGPT, Claude, or Gemini research interfaces. We also didn't measure backend version IDs, per-model speed, or credit consumption. The product names in the record identify what was selected in EVA for that run.

Use the research comparison framework if you also need to evaluate source finding and synthesis.

FAQ

Can I apply the result to academic research?

No. The source packet contained synthetic business notes, not academic papers. Use your own relevant papers and a subject-specific review criterion before applying the method to academic work.

Did you test deep research?

No. We supplied the source packet in the prompt and asked for a critique. Retrieving sources wasn't part of the task.

Can I use the prompt for research writing?

Use it to check whether a proposed argument matches excerpts you've already collected. Read the original sources yourself before citing them. The prompt is a critique aid, not a substitute for that reading.

Try the same exercise with a decision you're working on. In EVA Multi Chat, put the critiques side by side and check what each answer calls supported.