The concept
A pretrained model (Day 18) only continues text. Two more training steps make it useful:
Fine-tuning: extra training on a much smaller set of examples showing exactly the input and the output you want. Same idea as transfer learning (Day 13), applied to language.
RLHF (reinforcement learning from human feedback): people compare two answers from the model and pick the better one. Those choices train a "reward model" that scores answers the way people would. The main model is then tuned to earn higher scores (rewards, Day 6).
Model builders use both, at large scale, to turn a raw model into a helpful assistant. Companies more often just fine-tune for one job.
Walkthrough: discharge summaries in the house style
A hospital system wants AI to draft discharge summaries. A general model writes decent ones, but not in their format: section order, patient reading level, follow-up wording.
Fine-tuning:
- Collect, say, 3,000 clinician-approved past summaries, de-identified (patient details removed) under the hospital's privacy rules.
- Pair each with its input: the visit notes.
- Train the model on these pairs. It learns "notes like this → summary like that".
- Test on 200 held-out cases (Day 8), reviewed by clinicians.
RLHF-style feedback:
- For a new case, the model drafts two summaries, A and B.
- A clinician picks B: clearer medication list, plain-language follow-up.
- Thousands of such choices train a reward model that predicts which draft clinicians prefer.
- The model is tuned toward higher-scoring drafts.
Result: drafts closer to house style. A clinician still reviews and signs each one.
Before you fine-tune, ask
- Would clear instructions plus a few examples in the prompt do (Days 23–24)? Cheaper and easier to change.
- Is the problem style and format, or missing facts? Fine-tuning shapes behavior; for current facts, retrieval works better (Day 31).
Where you see it at work
- Healthcare: summaries and referral letters in an organization's format.
- Insurance: claim notes to company standards.
- Banking: complaint replies in an approved tone.
- Product management: thumbs up/down buttons on AI features collect preference feedback.
In one sentence
Fine-tuning teaches a model your format from approved examples, and RLHF teaches it what people prefer by learning from their choices between answers.