Prompt engineering best practices: fix the failure, not the phrasing

TL;DR: Put your instructions first, say the exact format you want, show one example of good, and set temperature to match the job. Those four fix most bad output. After that it is a testing job, not a writing job: pick a pattern to match the failure you actually see, lock structured output with a template plus a validator, and run every prompt version against the same 20 to 50 real inputs before you trust it.

Most bad AI output is not a model problem. It is a prompt problem. And the fix is nearly always boring.

Put your instructions first. Say what format you want. Show one example of what good looks like. Set your temperature to match the job. Those four moves fix most of it.

Everything else, the patterns, the schemas, the test loops, comes after. You add those when you need the same result a thousand times, not the first time.

Here's the thing. Most teams skip straight to clever phrasing. Then they wonder why the output changes every run.

Six habits that fix most bad output

Start here. These six cover the bulk of it.

1. Instructions first, data second, always use delimiters. Models weigh the start of a prompt heavily. Put the task before the data. Split the two with clear markers: XML-style tags, triple hashes, or plain headers. OpenAI's own guidance says the same. Wrap the task in an instructions block, the data in a data block, the examples in an examples block. It stops the model guessing which bit is which.

2. Be specific about format, reader, length and style. "Write a summary" gives you something new every time. "Write a 40 word summary for a non technical reader, plain sentences, no jargon" gives you the same shape every time.

3. Show one example of good. One worked example often beats a paragraph of rules. The model copies the shape instead of reading prose about the shape.

4. Prefill when a script has to read the output. If code parses the answer, start the model's reply for it. Open with a curly brace when you want JSON. The model finishes the pattern instead of picking one.

5. Use a short role line when tone matters. "You are a technical writer briefing a legal team" moves the vocabulary more than asking for "formal language" ever will.

6. Match the ask to the job. Tight and exact for extraction. Loose and open for ideas. They are not the same job, so they should not be the same prompt.

• Extraction, classification, coding: tight rules, no room to wander, one right answer.
• Ideas, drafting, brainstorming: looser rules, many right answers, room to explore.

None of this is exotic. It is the gap between a prompt someone wrote once and a prompt someone tested against ten real inputs.

Match the pattern to the failure, not the other way round

Patterns are not magic words. They are reusable shapes that fix one specific break. Prompt pattern catalogues treat them that way. Find the break first. Then pick the pattern.

• Role prompting fixes wrong tone. "You are a senior compliance officer reviewing this clause" tightens the language fast.
• Templates fix messy structure. Give the model a fill in the blank shape and it stops inventing layouts.
• Stepwise decomposition fixes jobs that are too big for one pass. Split "analyse this contract" into pull the clauses, then flag the risks, then summarise.
• Cognitive verifier fixes confident wrong answers. Make the model ask itself clarifying questions before it answers.
• Prompt chaining fixes jobs where step one needs cleaning before step two can trust it. Draft, then critique, then finalise.

One problem though. People copy a pattern because a guide told them to. Run your prompt first. Look at what actually broke. Then pick the pattern that targets that break. A cognitive verifier fixes confident errors. It does nothing for messy formatting.

Test each pattern on the model you are actually shipping. A pattern that shines on one model can flop on another. Newer reasoning models need less scaffolding than older ones did, so some of the old advice is now dead weight. If you are still choosing, how to pick an LLM is worth a read first.

Temperature, top_p and roles

System messages set the role and the rules. User messages carry the task and the data. Keep that split clean and the model stays predictable across a long chat.

For sampling, the general prompting guidance is steady. Temperature 0 for facts. Higher for creative work.

• Temperature 0 to 0.2: extraction, classification, code, factual answers.
• Temperature 0.5 to 0.8: drafting, brainstorming, tone changes.
• Top_p: leave it near 1.0 unless you are deliberately narrowing the pool. Moving both at once makes debugging harder, not easier.
• Max tokens: set it just above your expected length, not far above.

Then run the same prompt five to ten times at your chosen setting. Look at the spread. Decide whether that variance is a feature or a fault.

Long prompts have a second cost. Every example and every rule eats context window space that your actual data needs.

Lock the output format, do not hope for it

Ask for JSON in prose and half the time you get JSON with a sentence wrapped round it. Lock it down instead.

1. Give a template plus one filled example. Show the exact keys. Then show one real instance. The model copies a shape better than it reads a description of one.
2. Prefill the opening token. Starting the reply with a curly brace removes the model's option to chat first.
3. Validate, then loop. Run the output through a schema validator. If it fails, send the error back with "correct this to match the schema" rather than starting again.
4. Use examples over schemas when the content is subjective. A schema handles structure. An example handles judgement.

Microsoft's Foundry documentation backs the same approach. State the structure. Pair it with real source data when accuracy matters more than shape.

Format locking is also a security job, not just a tidiness one. A model that will happily wrap its answer in whatever a user asked for is a model that will follow a user's instructions. That is the door prompt injection walks through.

Test prompts like code

You would not ship code you never ran. Same rule. Decide what "correct" means before you compare versions.

• Set your bar first. Format, accuracy, and how much made up content you will tolerate for this job.
• Run small A/B batches. Two prompt versions against the same 20 to 50 real inputs. Not two versions against one input you eyeballed.
• Let the model say "not found". Telling a model to decline instead of guess is a documented way to cut hallucinations on lookup and extraction jobs.
• Escalate when the prompt cannot carry it. Two or three rounds under your bar means the answer is not better wording. It is grounding the prompt in real source data, or a different model.

Now here's the important bit. Keep a log of every version you test against the same input set. Six months on, that log is the only proof of which change lifted accuracy and which one just felt better.

A prompt that passed last quarter can quietly stop passing. Models get updated underneath you. That is model drift, and your test batch is how you catch it.

What this looks like in a real build

Devwiz has shipped 200+ apps, including work for the NSW Government, Briometrix, Vivid and Huskee. That is the track record, and it is app work. Prompt design is a separate craft. It turns up on any build where an AI feature reaches a real user.

In a live build that means prefilled structured output, schema validation before anything reaches a screen, and patterns picked from the failures a client's own data throws up. Not the pattern that reads best in a blog post.

The prompt is one part in a chain. It sits next to retrieval, validation and monitoring. On its own it is a trick. Wired into an AI app properly, it is a component. Runtime checks around it are a separate job again, covered in LLM guardrails.

If you are prompting a coding assistant rather than building a product feature, that is a different craft with its own rules. We covered it in prompt engineering for coding.

What actually matters

Most advice treats prompt engineering like a spell book. Find the right words and the model behaves. That is backwards.

The catalogue exists because different breaks need different fixes. No single pattern is better than the rest.

Here's what gets missed. This is a testing job, not a writing job. A prompt that works on five inputs and breaks on the sixth is not a good prompt. It is an untested one. The teams getting steady results are not the ones with the cleverest wording. They are the ones running batches against real data and logging what moved.

The other trap is over-building. People stack five patterns on a task that needed two clear sentences. Newer models handle a lot of the scaffolding that older ones needed. Test the plain version first.

If you take one thing from this: instructions first, plus one worked example. That pair fixes more real failures than every advanced pattern put together.

Once prompts are stable, the next question is how they fit the wider business. What prompt engineering means for a business covers that, and prompt engineering for builders covers the hands on side.

Want this built properly rather than bolted on? That is what our AI programs do.

Frequently asked questions

What is the single most important prompt engineering rule?

Put your instructions at the start of the prompt and separate them from the data with clear delimiters. That one change cuts more ambiguity than anything else you can do.

Should I always set temperature to zero?

No. Use 0 for extraction, classification and coding, where you want the same answer every time. Raise it towards 0.7 for drafting and brainstorming, where variety is the point.

When should I use a prompt pattern instead of a simple instruction?

Only after you have seen a specific failure repeat. Wrong tone, messy structure, confident errors. Match the pattern to that failure rather than applying one by default.

How do I stop a model making facts up?

Tell it explicitly that it can answer 'not found' instead of guessing, and give it real source data next to the question when accuracy matters.

How many test cases do I need before I trust a prompt?

Run each version against the same batch of 20 to 50 real inputs. A prompt that looks good on one or two examples has not been tested yet.

Is prompt engineering the same as prompt engineering for coding?

No. Prompting a coding assistant is about scoping a change and controlling what the model writes. Prompting a product feature is about reliability, structure and testing so the output is safe to parse.