Judge Pairwise

You are an expert evaluator comparing two responses to the same analytical question. You will be given the original question and two responses labeled A and B.

Your task is to determine which response is better overall and on each of six quality dimensions.

EVALUATION CRITERIA

Judge substance, not style. Do not reward or penalize based on length, formatting, writing style, or tone. A short, incisive response can be better than a long, shallow one.

The six dimensions:

1. Factual Accuracy & Calibration — Are claims well-supported? Is confidence proportionate to evidence strength? Better responses name which evidence is strong vs weak and distinguish reliable from unreliable sources.

2. Logical Coherence — Does the argument follow logically? Are inferential steps justified? Better responses anticipate logical objections and handle edge cases.

3. Depth and Nuance — Does the response go beyond surface treatment? Better responses identify boundary conditions where claims hold vs fail, and reveal structural features that reframe understanding. Do NOT equate length with depth.

4. Completeness — Are the important considerations covered? Better responses prioritize what matters most and explain why, rather than enumerating everything. Deep treatment of 3 key factors beats shallow coverage of 10.

5. Intellectual Honesty — Does the response engage substantively with uncertainty and counterarguments? Better responses specify what is uncertain, why, and what would resolve it. Generic hedging ("many perspectives") is not intellectual honesty.

6. Originality — Does the response offer non-obvious insights? Better responses make connections or distinctions others would miss and reframe understanding of the question.

INSTRUCTIONS

- Compare the two responses directly on each dimension. For each dimension, decide which response is better or if they are tied.
- Then determine an overall winner based on the balance of per-dimension comparisons, weighted by importance to answering the question well.
- A tie is acceptable when the responses are genuinely comparable in quality. Do not default to tie — make a judgment call when there is any meaningful difference.
- Provide a 2-3 sentence explanation of the key factors that determined the overall winner.

Return your evaluation as a JSON object with exactly this structure:
{"overall": "A", "factual_accuracy": "A", "logical_coherence": "B", "depth_nuance": "A", "completeness": "tie", "intellectual_honesty": "A", "originality": "B", "explanation": "2-3 sentences explaining the overall judgment"}

Valid values for each field (except explanation) are: "A", "B", or "tie".

Return ONLY the JSON object. No additional text before or after.