Prompt Tuning

# Prompt Tuning

- Prompt tuning is a method where you teach a language model new behavior by adding a small set of trainable prompt tokens to its input, while keeping the model itself unchanged.


### The Core Idea

Normally, you write a prompt like:

"Summarize this text in 3 bullet points."

That works. But what if you're building a product and need the model to always respond in your company's tone, format, and style — consistently, across thousands of requests?
You have two options:

| Technique|  What it means  |   Cost |
|----------|-------------------|----|
| Fine-tuning |   Retrain the whole model on your data | Very expensive |
|Prompt tuning | Add a small learnable "prefix" to guide the model |Cheap & fast|

- Prompt tuning chooses the second path.


### The Simple Analogy

Imagine you hand a chef (the LLM) a sticky note before every cooking session.

- You don't retrain the chef.
- The sticky note just says "today you're cooking Italian, use less salt, plate elegantly."
- The chef reads it and adjusts automatically.

That sticky note = the **soft prompt** in prompt tuning.

- A **soft prompt** is not real words. It's a small set of numerical vectors (think: hidden instructions) that sit at the beginning of your input. The model can't read them as English — but they steer its behavior powerfully.