prompt-optimizer

Optimize prompts with evals. Keep every instruction, example, and external context reference causal.

Record before editing:

• task type: new, refine, port, or debug
• target model family and snapshot, if known
• prompt surface: system, developer, user, tool descriptions, examples, schemas
• layer owners: platform, deployer/persona, retrieved context, user payload
• objective and non-goals
• inputs, tools, and external files available
• required output shape
• success criteria and failure cases
• hard constraints: latency, verbosity, safety, budget, tool use, style

If success criteria or examples are missing, create a small eval set first. If the bottleneck is model choice, retrieval, tool schema, or missing evals, say so before rewriting.

For repo or agent prompts, list stable context by exact path:

Rules:

• Reference stable files by repo-relative path instead of copying them.
• Paste only excerpts needed for the prompt or eval case.
• Mark whether a file is loaded, referenced, or out of scope.
• Avoid vague context pointers such as "read the docs".

Read references/model-family-notes.md.

• Known family: optimize for that family.
• Unknown family: write a portable base plus short adapter notes.
• Snapshot changes: rerun evals.
• Cross-family divergence: specialize only the failing layer.

Read references/core-patterns.md.

• Put stable policy in system or developer.
• Put task-local facts, retrieved context, and variables in user-facing sections.
• Keep one owner per behavior rule.
• Use headings or tags only to separate content types.
• Put tool policy in prompt text; keep schemas in provider-native tools.
• Keep persona light unless it changes behavior.
• Use the shortest wording that preserves the constraint.
• Cut filler, repeated reminders, dead examples, and rationale that does not affect evals.

Read references/meta-optimization-loop.md for refinements.

1. Baseline the current prompt on the same eval slice.
2. Cluster failures by root cause.
3. Write concrete edit criticisms.
4. Generate two to four candidates:
   • minimal-diff repair
   • structure-first rewrite
   • examples-first or tool-rule variant
   • provider adapter when needed
5. Compare candidates on the same cases.
6. Keep a short optimization log.
7. Validate the winner on holdout cases.
8. Stop on plateau, oscillation, overfit, excessive cost, or non-prompt bottleneck.

Return:

1. Target
2. Success Criteria
3. External Context
4. Optimized Prompt
5. Adapter Notes
6. Eval Set
7. Optimization Log
8. Residual Risks

For existing prompts, include a concise diff-style note of the main behavioral changes.

• editing before defining the eval target
• mixing policy, examples, and raw context without boundaries
• duplicating rules across layers
• putting durable policy in user payloads
• asking for chain-of-thought
• keeping contradictory legacy instructions
• overfitting to one or two examples
• retaining examples that no longer improve evals
• fixing tool-use failures only in prompt text when tool descriptions or schemas are weak
• adding markup that does not reduce ambiguity
• using persona as a substitute for behavior rules