At a glance
What you need to know.
AI development combines application engineering with a system whose answers may vary. Choose a task where errors can be detected, evaluate it with realistic cases, and keep authority in application code. A convincing demonstration is a starting point for investigation, not proof of production quality.
Define the task and the consequence of a wrong result
Describe the input, desired output and action a person takes next. Extracting fields from a supplier invoice is different from approving its payment. Identify errors that can be corrected easily and errors that could affect money, access or important decisions. Establish a baseline using the current process or deterministic software. Sometimes improved search, a form or a validation rule solves the task more reliably. Select AI because the task benefits from it, rather than because every new workflow needs a model.
Inspect the information the application is allowed to use
Identify source owners, quality, permission boundaries and update frequency. Test whether a user may see the source information before making it available for retrieval. Keep restricted records out of a shared index unless its access design is verified. Retrieval can supply relevant context, but it does not repair obsolete documents or resolve conflicting policies by itself. Decide how edits, removals and permission changes reach the application. Document provider data handling and retention requirements before sending real business content.
Create an evaluation set before optimizing prompts
Collect approved examples representing ordinary inputs, ambiguous requests, missing information and unusual formats. Define an expected result or a review rubric for each case. Keep evaluation cases separate from examples used to tune the application. Measure task-specific outcomes: correct required fields, useful cited answers, refusal when evidence is absent, review burden and processing time. Record model, prompt and source versions so a regression can be investigated. A single overall score can hide failures in the most consequential cases.
Design a useful review and fallback experience
Show users what the system found, the relevant evidence and what still needs checking. For extraction, allow corrections before data enters an authoritative workflow. For an assistant, provide a route to the source and a clear response when no suitable evidence is found. Avoid presenting a model-generated confidence number as calibrated certainty unless it has actually been validated. Define behavior for timeouts, provider outages and unavailable records. A failed AI call should leave the underlying business task recoverable.
Release by measured task performance and operating cost
Pilot with an agreed user group and approved data. Compare evaluated quality, manual correction, latency and cost per completed task with the baseline. Include retrieval, storage, observability and review effort in the cost estimate. Re-run important evaluations when changing the model, prompt, source corpus or permissions. Keep a disable or fallback path that does not depend on the same failing model. Production acceptance should name the allowed task, known limitations, accountable owner and criteria for stopping or expanding the pilot.
Before you start.
- Name the task and separate suggestions from business decisions.
- Approve source data, access boundaries and provider handling.
- Build representative evaluations with explicit expected outcomes.
- Test missing evidence, adversarial input and unauthorized requests.
- Provide correction, source inspection and a recoverable fallback.
- Measure quality, review effort, latency and total task cost.
Common questions.
Do we need to train our own model?
Often the first useful experiment can use an existing model with task-specific instructions and approved context. Training or fine-tuning needs a distinct reason, suitable data and evaluation evidence. It does not replace application permissions, source maintenance or testing.
Does retrieval-augmented generation prevent hallucinations?
No. Retrieval can improve access to relevant evidence, but the application may retrieve unsuitable material or produce an unsupported answer. Evaluate retrieval and answers separately, inspect citations, and allow the system to decline when the available evidence cannot answer the question.