Promptline Design

# Promptline — Design Spec

**Date:** 2026-07-03
**Status:** Approved design, pre-implementation

## What & Why

Promptline is an open-source, pip-installable pipeline for automatic prompt optimization:

1. **Calibrate** an LLM-as-judge against human labels (with a measurable agreement certificate).
2. **Optimize** system prompts with from-scratch implementations of state-of-the-art algorithms (GEPA flagship).
3. **Gate** deployment on statistically significant improvement over the incumbent.
4. **Serve** the active prompt from a versioned registry via an HTTP endpoint.

**Differentiation (research-validated):** existing tools are either optimizers that trust their metric blindly (DSPy, gepa, promptim, AdalFlow) or eval platforms that inspect judges but don't optimize (promptfoo, Braintrust, Weave). No open-source tool chains *calibrated judge → optimizer → statistical deploy gate*. Promptline is that chain.

**Goals:** portfolio-quality showcase, reusable OSS library, deep learning vehicle (algorithms implemented from scratch, not wrapped).

**Non-goals (YAGNI):** multi-tenant SaaS, billing/credits/markup, accounts. Users bring their own OpenRouter API key. A hosted/billed layer is a possible later sub-project, deliberately out of scope.

## Architecture

Library-core with thin shells. Everything lives in one Python package; CLI, TUI, and FastAPI server are thin layers over the same core. The web dashboard is a static React app served by FastAPI.