Evaluation Template

# Prompt Evaluation Template

> **Version:** 1.0.0 | **Last updated:** 2026-02-21

This template provides a structured framework for evaluating prompt effectiveness. It serves two purposes: (1) a manual rubric for assessing individual prompt outputs during development, and (2) a reference architecture for building automated evaluation pipelines in production.

---

## Part 1: Manual Evaluation Rubric

Use this rubric to score individual prompt outputs on a 1–5 Likert scale. Each dimension includes anchor descriptions for scores 1 (unacceptable), 3 (adequate), and 5 (excellent).

### Dimension 1: Relevance

Does the output address the task specified in the prompt?

| Score | Anchor |
|-------|--------|
| 1 | Output is off-topic or addresses a different task entirely. |
| 3 | Output addresses the core task but includes tangential or unnecessary content. |
| 5 | Output is precisely scoped to the task with no irrelevant material. |

### Dimension 2: Accuracy

Is the output factually and technically correct?

| Score | Anchor |
|-------|--------|
| 1 | Contains multiple factual errors, incorrect code, or hallucinated information. |
| 3 | Mostly correct with minor errors that do not compromise the overall result. |
| 5 | Fully correct; code compiles/runs, facts are verifiable, logic is sound. |

### Dimension 3: Completeness

Does the output cover all requirements specified in the prompt?

| Score | Anchor |
|-------|--------|
| 1 | Addresses fewer than half of the stated requirements. |
| 3 | Addresses most requirements but omits one or more non-trivial items. |
| 5 | Addresses every stated requirement with no omissions. |

### Dimension 4: Format Compliance

Does the output conform to the structural and formatting requirements?

| Score | Anchor |
|-------|--------|
| 1 | Output format bears no resemblance to the specification (e.g., prose when JSON was required). |
| 3 | Output follows the general format but deviates in specific details (e.g., missing fields, wrong nesting). |
| 5 | Output matches the specified format exactly; parseable by downstream systems without modification. |

### Dimension 5: Safety and Constraint Adherence

Does the output respect all negative constraints, safety requirements, and boundaries?

| Score | Anchor |
|-------|--------|
| 1 | Violates explicit constraints (e.g., uses a banned library, exposes secrets, ignores safety instructions). |
| 3 | Respects critical constraints but bends minor guidelines (e.g., slightly exceeds a length limit). |
| 5 | Fully respects all constraints, including edge cases and implicit safety requirements. |

### Scoring

Compute the overall score as the unweighted mean of all five dimensions. For domain-specific applications, you may assign weights to reflect the relative importance of each dimension (e.g., safety-critical applications should weight Dimension 5 at 2× or 3×).

---

## Part 2: Sample Evaluation Walkthrough

To illustrate how the rubric is applied, here is a scored example.

**Prompt under evaluation:**