System Prompt Design Guide: Patterns, Examples, and a Testing Checklist

Overview

A system prompt is the instruction layer that establishes an assistant's behavior before a user request is processed. In an LLM application, it should complement application logic, retrieved context, tool permissions, and human review rather than carry every responsibility alone. Good system message design makes the assistant's job narrower, observable, and easier to evaluate.

The most useful system prompts usually define six areas:

• Role: the assistant's function and intended audience.
• Scope: the tasks it should handle and the topics outside its remit.
• Instructions: the process or priorities it should follow.
• Output contract: format, fields, length, tone, and citation or evidence requirements.
• Tool behavior: when tools may be used, what inputs are allowed, and how results should be interpreted.
• Boundaries and escalation: when the assistant must refuse, qualify an answer, request missing information, or route the case to a person.

These elements should be written as operational rules rather than broad aspirations. “Be helpful” is difficult to test. “If the answer depends on information not present in the supplied documents, say that the documents do not establish it” is more specific and easier to verify.

For retrieval-augmented applications, keep the system prompt aligned with the retrieval design. A prompt can instruct an assistant to use supplied evidence, but it cannot repair missing, irrelevant, or poorly ranked context. For implementation background, see the RAG tutorial for beginners. Likewise, prompt quality should be assessed with representative test cases rather than judged from a few impressive conversations. A prompt evaluation framework can help teams record expected behavior, failure modes, and revisions.

Checklist by scenario

Support assistant

A support assistant needs a clear service boundary and a reliable handoff path. Include these instructions:

• Identify the product, plan, or workflow only when that information is available.
• Answer from the approved knowledge source or supplied account context.
• Do not invent policies, credits, timelines, or account details.
• Ask one focused clarifying question when the request lacks a necessary detail.
• Escalate cases involving account access, sensitive changes, complaints, or unresolved uncertainty.
• Use a consistent response structure, such as answer, next step, and escalation option.

Before: “You are a helpful customer support bot. Answer questions politely.”

After: “You are a product support assistant. Use only the supplied help content and account context. If the answer is not supported, state that clearly and recommend human review. Never guess account status, eligibility, refunds, or deadlines. For troubleshooting, provide numbered steps and ask for the product version only when it changes the next step. If the user reports a security concern or cannot regain access, stop troubleshooting and recommend the approved support route.”

Research assistant

A research assistant should separate evidence from interpretation. Specify that it must:

• Distinguish supplied evidence, reasonable inference, and unanswered questions.
• Preserve important qualifiers such as dates, populations, definitions, and limitations.
• State when the available material does not support a conclusion.
• Use a requested structure, such as findings, evidence, caveats, and follow-up questions.
• Flag conflicting passages instead of silently choosing one.

For assistants that use retrieval, define what counts as relevant context and how evidence should be attributed. The system prompt should not tell the model to cite sources that the application does not actually provide.

Coding assistant

A coding assistant benefits from explicit assumptions and verification steps. Add rules to:

• Ask for the language, runtime, framework, or version when it affects the solution.
• Preserve existing interfaces unless the user requests a breaking change.
• Explain assumptions before presenting code when requirements are incomplete.
• Provide tests or validation commands for non-trivial changes.
• Never claim that code was executed, compiled, or security-reviewed unless the application performed that action.
• Identify possible risks involving secrets, permissions, untrusted input, and destructive operations.

A useful output contract might require a short approach, a patch or code block, test cases, and known limitations. This is more dependable than asking the assistant to “write production-ready code” without defining what review means.

Data extraction assistant

Extraction prompts should prioritize schema compliance over conversational fluency. Define:

• The exact fields and data types to return.
• Whether missing values should be null, omitted, or marked as unknown.
• How to handle multiple values, conflicting values, and ambiguous references.
• Whether text should be copied exactly or normalized.
• A requirement to return valid JSON without commentary when machine parsing is expected.

Include a few representative examples only when they clarify difficult cases. Well-chosen few-shot prompting examples should demonstrate edge cases, not merely repeat obvious inputs.

What to double-check

Before shipping or revising a system prompt, run through this compact review:

1. Instruction priority: Are the most important rules easy to locate and free of contradictions?
2. Scope: Can the assistant recognize requests it should not handle?
3. Inputs: Does the prompt distinguish user text, retrieved content, tool results, and application-controlled values?
4. Unknowns: Is the assistant told what to do when context is incomplete or conflicting?
5. Output: Could a person or parser determine whether the response meets the contract?
6. Tools: Are permissions, confirmation requirements, failure handling, and irreversible actions explicit?
7. Safety: Does the assistant avoid exposing secrets, following untrusted instructions, or taking actions beyond its authority?
8. Escalation: Is there a concrete next step for high-impact or unresolved cases?
9. Evaluation: Do test cases cover normal requests, ambiguous requests, adversarial wording, missing context, and tool failures?
10. Observability: Can the team tell which prompt version, inputs, tools, and output produced a result?

Prompt text is only one part of an AI app architecture. Enforce permissions in application code, validate structured outputs, isolate secrets, and treat retrieved or user-provided text as untrusted data. Guidance on practical defenses is available in this prompt injection prevention guide.

Common mistakes

Writing a role instead of a procedure. A persona can establish context, but it does not define how the assistant should make decisions. Add inputs, steps, limits, and outputs.

Combining incompatible goals. Prompts often demand answers that are simultaneously exhaustive, brief, certain, creative, and strictly evidence-based. Set priorities and state which requirement wins when they conflict.

Relying on the prompt for enforcement. Instructions cannot replace server-side authorization, schema validation, rate controls, or confirmation gates for consequential actions.

Overloading the system message. Long prompts can contain duplicated rules, stale examples, and irrelevant background. Remove repetition and move dynamic data into the appropriate application-controlled context.

Testing only ideal inputs. Include incomplete requests, conflicting documents, malformed tool results, attempts to override instructions, and requests outside the assistant's scope.

Changing several variables at once. If the prompt, model, retrieval settings, and tools all change together, it becomes difficult to know which change affected the result. Version the prompt and record evaluation results for each revision.

When to revisit

Review a system prompt whenever the underlying workflow changes, not only when the wording feels outdated. Good update triggers include a new model, altered context limits, changed tools, a revised knowledge base, a new output schema, or a different escalation process. Revisit it before seasonal planning cycles when policies, product information, staffing, or expected request volume may change.

Also review prompts after a recurring failure appears in logs, human feedback, evaluation results, or support escalations. Keep a small regression set of important examples and run it after every meaningful revision. Track whether the change improved the target behavior without creating regressions in refusal quality, format compliance, factual grounding, or tool use. Teams building this workflow may also find the guides to LLM prompt testing and LLM observability useful references.

For a practical next pass, copy your current system prompt into a review document and mark each sentence as one of four types: role, instruction, boundary, or output requirement. Remove statements that do not fit one of those categories, resolve contradictions, then test the revised version against normal, ambiguous, and failure-case inputs. Record the prompt version and the reason for each change. That simple habit turns prompt engineering from ad hoc editing into a repeatable quality process.