PRISMA PHASE 5 PROMPT REGRESSION PLAN V0.1

# Phase 5 Prompt Regression Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task once approved.

**Goal:** Add a deterministic local prompt-regression workflow that fingerprints prompt assets, runs the Phase 4 evaluation harness, compares the result with the committed Phase 4 baseline, and emits an informational regression report.

**Architecture:** Phase 5 extends the existing `evals/` boundary without changing application behavior. The regression runner reads the current prompt asset, computes a stable fingerprint, runs the Phase 4 eval harness through the public API boundary, compares the generated scorecard with the committed baseline, and writes a generated report under `.local/`. The Phase 4 baseline remains immutable.

**Tech Stack:** Python 3.11, Pydantic, standard-library `hashlib`/`json`, existing Phase 4 eval models and runner, pytest, ruff, mypy. No new runtime dependency is expected.

---

> Production LLM Engineering Platform  
> Planning document only. This document defines what Phase 5 should implement. It does not implement code, create directories, or modify application files.

**Status:** Draft v0.1  
**Date:** 2026-06-30  
**Document:** PRISMA_PHASE_5_PROMPT_REGRESSION_PLAN_v0.1.md  
**Audience:** developers, AI engineers, ML engineers, technical reviewers, and maintainers  
**Companions:** [PRISMA_PROJECT_PLAN_v0.1.md](PRISMA_PROJECT_PLAN_v0.1.md), [PRISMA_REPOSITORY_ARCHITECTURE_v0.1.md](PRISMA_REPOSITORY_ARCHITECTURE_v0.1.md), [PRISMA_PHASE_2_BASELINE_RAG_API_PLAN_v0.1.md](PRISMA_PHASE_2_BASELINE_RAG_API_PLAN_v0.1.md), [PRISMA_PHASE_4_EVALUATION_HARNESS_PLAN_v0.1.md](PRISMA_PHASE_4_EVALUATION_HARNESS_PLAN_v0.1.md), [ADR-0001](adr/ADR-0001-code-data-separation.md), [ADR-0002](adr/ADR-0002-provider-adapter-boundary.md), [ADR-0003](adr/ADR-0003-secrets-via-environment.md), [ADR-0004](adr/ADR-0004-evaluation-first-development.md)

---

## 1. Purpose

Phase 5 exists to make prompt changes measurable, reviewable, and reproducible.

Phase 4 made Prisma measurable through a committed golden dataset, deterministic metrics, a generated scorecard, and a committed baseline. Phase 5 builds directly on that foundation by comparing the current prompt-driven behavior against the Phase 4 baseline. It should answer:

- Did a prompt change alter behavior?
- Which evaluation cases changed?
- Which metrics improved?
- Which metrics regressed?

This phase is an engineering control for prompt changes. It is not prompt engineering, prompt optimization, prompt tuning, prompt generation, or a CI gate.

## 2. Scope

### In Scope

- Prompt snapshots as committed metadata records for the baseline prompt state.
- Prompt fingerprinting for the configured prompt asset.
- Prompt metadata in generated regression reports.
- A local regression runner invoked as `python -m evals.regression`.
- Deterministic baseline comparison against `evals/baselines/phase4-baseline.json`.
- Generated regression report written under `.local/prisma/evals/regression.json`.
- Per-case and per-metric comparison of the current scorecard against the committed baseline.
- Informational reporting of changed workflow status, citations, and retrieval attempts where available.
- Correctness tests for fingerprinting, comparison, report generation, and baseline immutability.
- Non-secret prompt-regression configuration in `configs/defaults.toml`.
- README and `docs/DEVELOPMENT.md` updates documenting the local regression smoke path.

### Explicit Non-Goals

- No CI gates.
- No GitHub Actions.
- No PromptFoo.
- No online evaluation.
- No LLM-as-judge.
- No semantic grading.
- No prompt optimization.
- No automatic prompt rewriting.
- No prompt generation.
- No prompt registry.
- No hosted services.
- No dashboards.
- No human review workflow.
- No automatic approval or rejection of prompt changes.
- No provider comparison.
- No Phase 6 observability, traces, cost budgets, or runtime metrics.

## 3. Architectural Alignment

### Project Plan

The project plan defines Phase 5 as Prompt Regression: versioned prompt artifacts, regression runs on prompt changes, baseline comparison, and change reports. This plan implements that slice only. It deliberately excludes CI gates, observability budgets, dashboards, hosted services, and model benchmarking.

### Repository Architecture

Prompt assets remain under `prompts/` as data. Regression logic lives under `evals/` because it measures behavior against committed evaluation assets. Generated reports are runtime artifacts under `.local/` and are not committed. The planned flow is: