home/productivity/prompt-eval-b-prompts-and-prompt-datasets

Prompt Eval B Prompts And Prompt Datasets

GPTClaudeDeepSeek··▲ 661 copies·updated 2026-07-14

prompt-eval-b-prompts-and-prompt-datasets.prompt

# 子项目 B:Prompt 管理 + Prompt 测试集 Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** 让用户管理带 `{{参数}}` 模板的 Prompt(system+user,不可变版本)并维护 Prompt 测试集(复用数据集表,列即参数),为后续评测(C/D)提供素材。

**Architecture:** app-server 新增 `prompts`/`prompt_versions` 两表 + `/prompts` API;`{{name}}` 抽参/校验抽到 `services/common` 的 `prompt_template`(C 复用);Prompt 测试集复用 `datasets`/`dataset_versions`,新增 `DatasetKind.PROMPT`,`create_version` 按 kind 分支跳过固定列校验(列名照旧写进 `stats.columns`)。前端加 Prompt 页 + 数据集页 prompt 类型。

**Tech Stack:** FastAPI + SQLAlchemy 2.0 + 编号 SQL 迁移;pandas;pytest;React 19 + Vite + TS + Tailwind v4。

**Spec:** [`docs/superpowers/specs/2026-06-15-prompt-eval-B-prompts-and-prompt-datasets-design.md`](../specs/2026-06-15-prompt-eval-B-prompts-and-prompt-datasets-design.md)

**铁律(`CLAUDE.md`)**:改 `app/models/**` 必配【下一个编号】幂等 `db/migrations/NNN_*.sql`;改 RBAC 目录时 `bootstrap.py` 与编号迁移两处一起改。当前最新迁移 = `018_llm_manage_perm.sql`,本计划新增 `019`、`020`。提交信息都以 `Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>` 结尾。

## 文件结构

| 文件 | 职责 | 任务 |
|---|---|---|
| `services/common/modelforge_common/prompt_template.py` | `{{name}}` 抽参 + 语法校验 | T1 |
| `services/common/tests/test_prompt_template.py` | 模板工具单测 | T1 |
| `services/app-server/app/models/prompt.py` | `Prompt`/`PromptVersion` 模型 + latest 属性 | T2 |
| `services/app-server/app/models/__init__.py` | 注册模型 | T2 |
| `services/app-server/db/migrations/019_prompts.sql` | 建两表 | T2 |
| `services/app-server/app/bootstrap.py` | `prompt:read`/`prompt:write` | T3 |
| `services/app-server/db/migrations/020_prompt_perms.sql` | 权限种子 | T3 |
| `services/app-server/tests/test_bootstrap.py`、`tests/test_migrations_apply.py` | 计数断言 +2 表 +2 权限 | T3 |
| `services/common/modelforge_common/enums.py` | `DatasetKind.PROMPT` | T4 |
| `services/app-server/app/services/dataset_service.py` | prompt 分支(跳过固定列校验) | T4 |
| `services/app-server/app/schemas/prompt.py` | Prompt + PromptDataset schema | T5 |
| `services/app-server/app/services/prompt_service.py` | create/add_version/validate | T6 |
| `services/app-server/app/api/prompt.py` | `/prompts` 路由 | T7 |
| `services/app-server/app/api/datasets.py` | `POST /datasets/prompt` | T7 |
| `services/app-server/app/main.py` | 注册 prompt 路由 | T7 |
| `services/app-server/tests/test_prompts.py` | 模型/service/API/数据集分支 测试 | T2,T4,T6,T7 |
| `frontend/src/api/client.ts` | prompt client + createPromptDataset | T8 |
| `frontend/src/pages/PromptsPage.tsx` | Prompt 页 | T9 |
| `frontend/src/App.tsx`、`components/AppShell.tsx` | 路由 + 导航 | T9 |
| `frontend/src/pages/DatasetsPage.tsx` | prompt 类型集成 | T10 |

---

### Task 1: 共享模板工具 `prompt_template`

**Files:**
- Create: `services/common/modelforge_common/prompt_template.py`
- Test: `services/common/tests/test_prompt_template.py`

- [ ] **Step 1: 写失败测试**

`services/common/tests/test_prompt_template.py`:

fill the variables

This prompt has 2 variables. Pro fills them into a ready-to-paste prompt for you — no manual find-and-replace.

{{参数}{{name}

Unlock with Pro →

when to use it

Community prompt sourced from the open-source GitHub repo LingyiChen-AI/ModelForge (Apache-2.0). A "Prompt Eval B Prompts And Prompt Datasets" style prompt — adapt the placeholders and specifics to your task. Imported as-is and not independently retested here, so check the output before relying on it.