Why check an AI's answer with a different AI

Asking the same AI again

Asking a model to review its own answer sometimes catches a slip. But research on self-correction found that, without outside feedback, models often don't improve their reasoning when asked to recheck it, and sometimes make it worse (Huang et al., “Large Language Models Cannot Self-Correct Reasoning Yet”, ICLR 2024).

Source: arXiv:2310.01798

A different company, a different set of blind spots

Models from different companies are trained on different data with different methods, so they are less likely to make exactly the same mistake. When ChatRound knows where an answer came from, it picks reviewers from other companies, and the report is not written by that company's model either.

Answer first, then compare

A reviewer that reads the answer first can pick up its framing, and its error with it. ChatRound's reviewers write their own answer to the question before they go through the original, in the same call.

Two reviewers, and their disagreements kept

With two reviewers, a disagreement is information. The report marks a claim as disputed rather than averaging the two views, and quotes each side.

In our Schengen examples, one reviewer took the wrong side in three of the six languages. The other reviewer was right, and the dispute was there in the report to see.

Example check: Back to Europe after 70 days? The 90/180 rule, checked

Agreement still isn't proof

Two models can share a mistake. For anything that costs money or affects your health, look the flagged claim up in a primary source. The report quotes it, so you know exactly what to search for.