Prompt Comparison
# Prompt Comparison Recipe
## Goal
Compare prompt versions objectively and pick the best one for production.
## Metric Ideas
- task success rate
- format validity rate
- refusal correctness rate
- latency and token cost deltas
## Sample Workflow
1. Freeze an evaluation set with representative queries.
2. Run Prompt A and Prompt B on the same set.
3. Score outputs with the same rubric/validator.
4. Compare quality, cost, and latency side by side.
5. Promote only if thresholds improve without regressions.
## Simple Python Examplewhen to use it
Community prompt sourced from the open-source GitHub repo decided-indication109/AI-Engineer-in-90-Days (MIT). A "Prompt Comparison" style prompt — adapt the placeholders and specifics to your task. Imported as-is and not independently retested here, so check the output before relying on it.
tags
productivitycommunitydeveloper
source
decided-indication109/AI-Engineer-in-90-Days · MIT
more in Productivity
Productivity✓ tested
Summarize a doc into decisions & actions
chief of staff who extracts what to DO, not just what was said
Productivity✓ tested
Draft a reply to a hard email
calm, direct communicator who de-escalates without caving
Productivity✓ tested
Turn a brain-dump into a weekly plan
planning coach who protects your focus, not just your calendar