Diffprompt v2 Program Design

# diffprompt v2 — Program Design Spec

> Status: DRAFT for review · Author: Rudra (w/ Claude) · Date: 2026-06-19
> Scope: the full v2 program — production hardening + three product modules + platform (service & web UI).

---

## 1. Vision

diffprompt today answers one question: *"I changed my prompt — did it help or hurt, before I ship?"* It does this well as a one-shot CLI.

v2 turns it into the **pre-production reliability layer for LLM apps**: not just a one-time diff, but a continuous, explainable system that tells you *which part* of a prompt drives behavior, *when* a model silently shifts under you, and *why* an agent made a decision. The through-line is **accuracy, speed, and explainability** — the AI-reliability + observability niche, made concrete.

Three product modules, one engine:

| Module | Question it answers | Maturity hook |
|---|---|---|
| **Attribution** (Idea 1) | *Which sentence in my prompt causes this behavior?* | deep algorithmic story (Shapley) |
| **Drift** (Idea 2) | *Did the model change under me?* | continuous monitoring, the observability hero |
| **Provenance** (Idea 3) | *Why did my agent do that — and was it hijacked?* | agentic security/reliability, most unique |

## 2. Goals & non-goals

**Goals** (the user's four words, made measurable):
- **Accurate** — judge agreement with human labels ≥ 0.8 on a labeled eval set; deterministic, reproducible analysis given a fixed seed; calibrated confidence.
- **Faster** — a 20-test diff completes in < 30s on warm cache against a fast provider; streaming results so first insight appears in < 3s.
- **Sleeker** — verdict-first output (shipped), a cohesive web dashboard, one consistent CLI grammar, zero `print()` noise.
- **Prod-ready** — provider abstraction, structured retry, structured logging, self-telemetry (OTel), ≥ 85% test coverage, CI gating lint + types + tests, graceful failure everywhere.

**Non-goals (v2)**:
- Not a training/fine-tuning tool.
- Not a general LLM gateway (we wrap providers for our use, not as a product).
- Not a replacement for prod observability (Langfuse/Phoenix) — we integrate with them, we are the *pre-prod and drift* layer.
- No multi-tenant SaaS auth/billing in v2 (single-tenant/self-host first; flagged as future).

## 3. Current state (post-hardening, as of this spec)

Already shipped this cycle (PRs #3/#5/#7/#9, #11 in flight):
- `--model`/`--top-n` honored; stale tests fixed.
- CI matrix py3.10–3.12 + badge.
- Pipeline parallelized (anchors/generation/judge) + token-bucket Groq rate limiter.
- Verdict-first output redesign + `output/insights.py` (latency/length/sparkline/scorecard/trade-off), terminal + HTML.
- Ontology hardened against non-string tags (#11).

Known debt this spec addresses: `print()` debug in `cascade.py`; ad-hoc retry; Groq/Ollama hardcoded; no streaming; no self-telemetry; thin coverage on `runner`/`clusterer`; judge over-leniency (observed 100/100); ontology wrapper-key schema; Windows cp1252 unicode crash on piped output.

## 4. Target architecture