Lesson 6 of 7
System & app design
7 min read
A single API call is easy. A reliable app on top of a model that's non-deterministic, occasionally wrong, and sometimes slow — that's the real engineering. Where do you even start?
Treat prompts as configuration
The prompts and schemas that shape your calls are the behaviour of your app, so treat them like config you can version, test, and change without a redeploy. Keep the system prompt in one place, template the variable parts, and write down what 'good output' looks like so you can tell when a change makes things worse.
Your prompts are your product's behaviour. Version them, test them, and keep them out of scattered string literals.
Design for the model being wrong
The model will sometimes stall, return junk, or hallucinate. A robust app plans for it: set timeouts and retries, validate every response before you trust it, and have a fallback — a smaller model, a cached answer, or an honest error — for when a call fails. Keep a human in the loop for high-stakes actions, and log the inputs and outputs so you can debug what actually happened.
Reliability isn't a better prompt — it's the scaffolding around the call: validate, retry, fall back, and fail honestly.
Never send a raw model output straight to a user or a destructive action without a check. 'The model said so' is not validation — treat every response as untrusted input until you've verified it.
The shape of it
- —Treat prompts and schemas as versioned, testable configuration.
- —Assume the model will be wrong sometimes — validate every response.
- —Build in timeouts, retries, and a fallback for failed calls.
- —Log inputs and outputs; keep a human in the loop for high-stakes actions.
You're moving your AI feature from a demo to production. Which change matters most for reliability?
Continue in the app
Take the whole Building AI Apps with the API course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.