The concept
A conversation with an LLM is a list of messages, each tagged with a role:
- System: the standing instructions from whoever built the assistant. Who it is, what it may do, what it must never do, the tone. Set once, applies to every turn.
- User: what the person types.
- Assistant: what the model replies.
The model reads all of them together each time it answers (inside its context window, Day 20). The system prompt is like the job description and the house rules; the user message is the customer at the counter.
Walkthrough: a pharmacy refill-reminder assistant
System prompt (written by the pharmacy team):
"You are RefillBuddy, a friendly assistant for a retail pharmacy.
You help customers check refill status, request refills and set reminders.
Rules:
- Never give dosing or medication advice. Say: 'A pharmacist can help with that,' and offer to connect them.
- If a customer mentions side effects, a missed dose or feeling unwell, escalate to a pharmacist right away.
- Only discuss the customer's own prescriptions after they are verified.
- Keep replies under 3 sentences."
User message:
"Can you remind me to refill my blood pressure meds? Also I missed yesterday's pill. Should I take two today?"
Assistant reply:
"Sure, I'll remind you 5 days before your next refill date. About the missed dose: that's a question for a pharmacist, and I can connect you now. Would you like that?"
What happened: the reminder (allowed) was handled, the dosing question (forbidden) was not answered even when asked directly, and the escalation rule kicked in, kindly.
A system prompt guides the model; it doesn't guarantee behavior. Real products add code checks, testing and human review (guardrails, Day 45).
Where you see it at work
- Healthcare: clear "never advise, always escalate" rules for patient-facing assistants.
- Banking: a support assistant told to never reveal full account numbers and to send disputes to an agent.
- Retail: brand voice, return-policy scope and when to hand off to a person.
- Product owners and BAs: the system prompt is a requirements document. Write it, review it and version it like one.
In one sentence
The system prompt sets who the assistant is and the rules it follows, the user message is the request, and the assistant reply should follow both, with code and humans as backup.