Prompt Evaluation Using Promptfoo

# Prompt evaluation using promptfoo


## What is Promptfoo?

**Promptfoo** is a testing tool for AI prompts — think of it as **JUnit/pytest, but for your LLM prompts**.

## How it works

You write test cases like:

> *"Given this prompt → I expect the output to contain X / not contain Y / match a pattern"*

Promptfoo runs your prompt against the model and checks if the output passes your conditions. You can also run the **same prompt across multiple models** (GPT-4, Claude, Gemini) side by side to compare results.

---

## Why it's used

| Problem | How Promptfoo helps |
|---|---|
| You tweak a prompt and break something | Catches regressions automatically |
| You want to compare GPT vs Claude | Runs both and shows a comparison table |
| You don't know which prompt version is better | A/B tests prompts with real metrics |
| You're building a RAG/chatbot app | Tests edge cases before they hit production |

---

#### One-line summary

> **Promptfoo = automated testing + benchmarking for your AI prompts**, so you don't have to manually copy-paste into ChatGPT and eyeball the output every time you change something.


## How to use Promptfoo?

- For using promptfoo, you must install nodejs in your system.

<img width="1875" height="842" alt="image" src="https://github.com/user-attachments/assets/6a93a30a-c7ef-410a-8761-9acb619a0273" />

- After installation run the following commands in your command prompt: