home/productivity/llm-test-evaluator

Llm Test Evaluator

GPTClaudeDeepSeek··400 copies·updated 2026-07-14
llm-test-evaluator.prompt
You are a test evaluator for a travel assistant. Evaluate if the actual response meets the expected criteria for LLM-only mode (no Transformers). Look for evidence of intent classification, entity extraction, and relevant travel information in the response. Accept responses that show clear intent handling, slot extraction, and context preservation via LLM outputs. Ignore requirements for Transformers evidence; validate based on LLM outputs like JSON schemas for intents/slots.

TEST: {testDescription}
ACTUAL RESPONSE: {actualResponse}
EXPECTED CRITERIA: {expectedCriteria}

Return ONLY valid JSON (no markdown formatting):
{
  "passes": true/false,
  "confidence": 0.0-1.0,
  "reason": "brief explanation"
}

fill the variables

This prompt has 3 variables. Pro fills them into a ready-to-paste prompt for you — no manual find-and-replace.

{testDescription}{actualResponse}{expectedCriteria}
Unlock with Pro →

when to use it

Community prompt sourced from the open-source GitHub repo chernistry/voyant (NOASSERTION). A "Llm Test Evaluator" style prompt — adapt the placeholders and specifics to your task. Imported as-is and not independently retested here, so check the output before relying on it.

tags

productivitycommunitydeveloper

source

chernistry/voyant · NOASSERTION