At a glance
What you need to know.
Choose prompting, retrieval, tools or fine-tuning by the failure you need to address. Stable instructions, changing knowledge and task-specific behavior are different requirements. Compare approaches using the same held-out task set and operating constraints before accepting additional training and maintenance work.
Classify errors before changing the model
Sample accepted and rejected outputs and group the failures: unclear instructions, missing information, wrong formatting, reasoning errors, unauthorized actions or unsuitable task scope. Each category suggests a different intervention. A changing customer record requires authorized retrieval or a tool, not memorization in model weights. A consistent output format may improve with a clearer schema and validation. Define the task's quality rule and baseline first. Without that diagnosis, a new prompt or fine-tuned candidate can look better on a few demonstrations while preserving the actual operational failure.
Use prompts for task instructions and examples
A prompt can express the goal, relevant constraints, expected output and examples of accepted behavior. Keep instructions concise enough to inspect and version them with evaluation cases. Validate the actual output rather than assuming the model follows the requested format. Test contradictory input, missing evidence and changed context. Prompt improvements are a useful early experiment because they are comparatively easy to revise, but their cost and reliability still depend on context length and model behavior. A prompt cannot replace server authorization or guarantee factual accuracy.
Use retrieval and tools for current permitted facts
Retrieve approved information when the task needs changing knowledge or private records. Scope retrieval to the requesting role and show source references where evidence matters. Use a controlled tool when the task requires a calculation or business-system operation whose result can be verified. Evaluate stale, missing and misleading sources. Retrieval quality depends on source preparation, ranking and access, so adding more documents does not automatically improve answers. Keep facts and instructions distinct: an external document may contain content that attempts to steer the model outside the authorized task.
Consider fine-tuning for a documented behavior gap
Fine-tuning can be considered when the chosen model and provider support it and a stable task-specific behavior remains inadequate after simpler experiments. Confirm current capability, data format, limits and account access in the selected provider's documentation. Build representative approved examples and retain a genuinely held-out evaluation set. Include edge cases and define data ownership and permitted training use. Training introduces a versioned artifact and ongoing evaluation responsibility. Do not present it as a general cure for missing current facts, permission errors or an undefined acceptance rule.
Compare candidates under equal task conditions
Use the same held-out cases and success criteria to compare a prompt-only baseline, retrieval or tool changes and a fine-tuned candidate where applicable. Measure accepted quality, unsupported claims, reviewer effort, latency and total cost. Include training and maintenance in the longer-term comparison. Keep evaluation examples out of tuning and avoid selecting a candidate solely from the cases shown in a demo. A model that improves a frequent category can regress on a rare but consequential exception; segment results according to the actual business task.
Operate the chosen approach as a versioned system
Version the prompt, model, retrieval sources, tools and output validator together for the release. Rehearse source changes, provider unavailability and rollback to an accepted version. Keep destination effects separately authorized and traceable. In an illustrative support task, a fine-tuned style can produce a more consistent response while a policy answer remains wrong because its source is obsolete. Repair the source and re-evaluate the task rather than treating the improved wording as evidence of correctness. Document what the chosen intervention actually changed and what it did not resolve.
Keep evidence and execution under separate authority
An AI-generated proposal is not authorization to perform a business action. Validate record ownership, tool arguments and the permitted operation outside the model using the actual requesting identity. Restrict tools to the agreed task and require a reviewed path for privileged or irreversible decisions. Treat retrieved documents and tool responses as untrusted content that can contain misleading instructions. Preserve the evidence a reviewer needs with appropriate access, rather than exposing raw sensitive inputs in general logs. OWASP's prompt-injection guidance supports layered controls; retrieval, fine-tuning and a carefully written prompt do not establish an enforceable security boundary.
Before you start.
- Which observed error category needs improvement?
- Is the need instructions, current facts or stable task behavior?
- Are training examples approved and evaluation cases held out?
- Do candidates use comparable quality, latency and cost criteria?
- Can the full prompt/model/source/tool release be rolled back?
Common questions.
Does fine-tuning replace retrieval?
Not for a task that needs current or private facts whose access must be checked. Retrieval and tools supply permitted information or operations at runtime. Fine-tuning concerns a supported training intervention for behavior, with its own data and evaluation requirements.
Does either approach eliminate prompt injection?
No. OWASP's guidance explains that retrieval and fine-tuning do not fully mitigate injection. Enforce identity and allowed effects outside the model, treat external content as untrusted and test adversarial inputs with layered controls.