PROMPT TESTING SUMMARY

# Contextuals Prompt Variants - Objective Testing Results

## Overview

We conducted comprehensive objective testing of 5 different prompt variants using **Qwen 3 30B** via Ollama to determine which provides the best contextual awareness for LLM applications.

## Test Methodology

### Test Setup
- **Model**: Qwen 3 30B (qwen3:30b-a3b-q4_K_M)
- **Framework**: PydanticAI with Ollama
- **Questions**: 10 diverse contextual awareness scenarios
- **Metrics**: Contextual awareness score, token efficiency, response time, context coverage

### Test Questions
1. Weather-based clothing recommendation
2. Health and safety assessment for outdoor activity  
3. Cultural and linguistic appropriateness
4. System performance diagnosis
5. Time zone and scheduling awareness
6. Environmental health decision
7. Current events awareness
8. Multi-factor activity planning
9. Personalized productivity optimization
10. Transportation decision with multiple factors

## Results Summary

### Final Rankings

| Rank | Variant    | Score | Coverage | Tokens | Efficiency* | Avg Time |
|------|------------|-------|----------|--------|-------------|----------|
| 1    | DEFAULT    | 0.980 | 1.000    | 126    | 0.778       | 14.6s    |
| 2    | STRUCTURED | 0.980 | 1.000    | 53     | 1.849       | 13.5s    |
| 3    | COMPACT    | 0.960 | 1.000    | 28     | 3.429       | 17.3s    |
| 4    | MINIMAL    | 0.930 | 1.000    | 20     | 4.650       | 13.6s    |
| 5    | DETAILED   | 0.910 | 0.929    | 219    | 0.416       | 16.8s    |

*Efficiency = Score per 100 tokens

## Key Findings

### 1. **STRUCTURED** Variant - Best Overall Choice
- **Score**: 0.980 (tied for highest)
- **Token Efficiency**: 1.849 (2nd highest)
- **Tokens**: 53 (moderate)
- **Strengths**: Excellent balance of awareness and efficiency, parseable format
- **Use Case**: **Recommended for most production applications**

### 2. **DEFAULT** Variant - Highest Awareness
- **Score**: 0.980 (tied for highest)
- **Token Efficiency**: 0.778 (moderate)
- **Tokens**: 126 (higher)
- **Strengths**: Best overall contextual awareness, comprehensive coverage
- **Use Case**: Standard applications without strict token limits

### 3. **COMPACT** Variant - Production Balance
- **Score**: 0.960 (very high)
- **Token Efficiency**: 3.429 (high)
- **Tokens**: 28 (low)
- **Strengths**: Good balance of efficiency and awareness
- **Use Case**: Production environments with moderate token constraints

### 4. **MINIMAL** Variant - Maximum Efficiency
- **Score**: 0.930 (good)
- **Token Efficiency**: 4.650 (highest)
- **Tokens**: 20 (lowest)
- **Strengths**: Ultra-efficient, essential context only
- **Use Case**: Extreme token limits, mobile apps, embedded systems

### 5. **DETAILED** Variant - Comprehensive Context
- **Score**: 0.910 (lowest)
- **Token Efficiency**: 0.416 (lowest)
- **Tokens**: 219 (highest)
- **Strengths**: Rich context, detailed guidelines
- **Use Case**: Complex scenarios requiring extensive context

## Surprising Results

1. **DETAILED variant performed worst** despite having the most information
   - Likely due to information overload confusing the model
   - Lower context coverage (0.929 vs 1.000 for others)

2. **STRUCTURED variant tied for best score** with much fewer tokens than DEFAULT
   - JSON-like format appears to help model parse context efficiently
   - Best token efficiency among high-scoring variants

3. **MINIMAL variant achieved 0.930 score** with only 20 tokens
   - Demonstrates that concise, well-structured context can be highly effective
   - Highest token efficiency overall

## Recommendations by Use Case

### Production Applications (General)
**Use: STRUCTURED variant**
- Excellent contextual awareness (0.980)
- High token efficiency (1.849)
- Parseable format for programmatic use

### Token-Constrained Environments
**Use: COMPACT or MINIMAL variants**
- COMPACT: Better awareness (0.960) with 28 tokens
- MINIMAL: Maximum efficiency (4.650) with 20 tokens

### Rich Context Requirements
**Use: DEFAULT variant**
- Highest contextual awareness (0.980)
- Comprehensive coverage (1.000)
- Clear, readable format

### API Integrations
**Use: STRUCTURED variant**
- JSON-like format easy to parse
- Excellent performance
- Moderate token usage

## Prompt Variant Contents

### DEFAULT Prompt (126 tokens)