Custom Prompt Eval

GPTClaudeDeepSeek··▲ 253 copies·updated 2026-07-14

custom-prompt-eval.prompt

---
title: Custom Prompts
description: Allows you to create your own set of evaluations
---

Each LLM application has its unique needs and it is not possible to have a one-size-fits-all evaluation tool. 

A sales assistant bot needs to be evaluated differently as compared to a calendar automation bot. 

Custom prompts help you grade your model the way you want it.


Parameters:
- `prompt`:	Evaluation prompt used to generate the grade
- `choices`:	List of choices/grades to choose from
- `choices_scores`:	Scores associated with each choice
- `eval_type`:	One of ["classify", "cot_classify"], determines if chain-of-thought prompting is to be applied or not
- `prompt_var_to_column_mapping (optional)`: mapping between variables defined in the prompt vs column names in the data

### How to use it?

when to use it

Community prompt sourced from the open-source GitHub repo uptrain-ai/uptrain (Apache-2.0). A "Custom Prompt Eval" style prompt — adapt the placeholders and specifics to your task. Imported as-is and not independently retested here, so check the output before relying on it.