Extraction System Prompts

# Extraction System Prompts

This directory contains all **parameter extraction prompt versions** used in the LLM Query Engine.  

Each prompt is stored as a separate JSON file and loaded automatically by `prompt_loader.py`.

## Comparison Table

| Version | Style | Strength | Results (P / R / F1 / Perfect) | Best For |
|---|---|---|---|---|
| **v1 – default** | verbose full rules | very explicit schema & parameter rules | P **0.889**, R **0.934**, F1 **0.896**, perfect **53.8%** | When you want the highest overall F1 and strong recall; baseline evaluation and documentation reference |
| **v2 – strict** | short strict | minimal text, strong constraints | P **0.903**, R **0.847**, F1 **0.865**, perfect **50.0%** | When you care more about precision and stricter behavior than maximum recall |
| **v3 – few-shot heavy** | example-driven | relies on patterns from few-shot groups | P **0.807**, R **0.816**, F1 **0.802**, perfect **38.5%** | Testing few-shot/example-heavy strategies and small-model behavior |
| **v4 – hybrid (default)** | rules + few-shot | combines v1-style rules with example-heavy few-shot groups for robustness | P **0.825**, R **0.804**, F1 **0.802**, perfect **38.5%** | Recommended default for the pipeline: robust rule-following + few-shot patterning (long‑ vs short‑term, ranges, operator phrasing) |

> Based on an internal test set of 26 queries (see notebook results). v1 achieved the highest F1 and recall, v2 the highest precision. v4 trades a bit of raw score for more robust handling of tricky real‑world phrasing and is our current **default recommendation**.

## extraction_v1_default

**Purpose:** Baseline, full-rule extraction prompt.  

Contains detailed parameter guidelines, strict wording preservation rules, and example-based behavior descriptions.

**Characteristics:**

- Strict about copying *exact* user wording  
- Covers all parameters defined in the current extraction schema (rooms, price, surface, location, proximity, balcony, short-term fields, etc.)  
- Rejects vague surface area terms  
- Differentiates parks vs. public transport  
- No JSON output — model returns `parameter: phrase` lines  
- Works both with and without few-shot examples

**Recommended Use Cases:**

- Baseline evaluation  
- Testing small models (1–3B) with strong instructional guidance  
- Comparing against stricter/minimal prompts  
- Ensuring consistent extraction rules across models

**Few-Shot Group:**  

`few_shot_group` is set to `basic_v1`.

Few-shot examples are attached by `prompt_loader.py` when running evaluations/experiments.

**File:**  
`extraction_v1_default.json`

**Full Prompt:**