We are deploying a GenAI chatbot for customer support. Our biggest fear is the model making up fake refund policies (Hallucination). Besides RAG, what other techniques like Temperature settings, Top-P sampling, or Prompt Guardrails should we use to ensure the output remains grounded in reality?
3 answers
Start by lowering your Temperature. Temperature controls the randomness of the model: 0.0 makes it deterministic (choosing only the single most likely token), while 1.0+ makes it creative. For customer support, keep it between 0.0 and 0.2. Also, use Top-P (Nucleus Sampling) to limit the model's vocabulary to a small percentage of likely tokens, cutting off the "long tail" of incorrect associations.
You should implement System Guardrails. Tools like NeMo Guardrails or LlamaGuard allow you to define "Off-limit" topics. You can also create a "Verification Loop" where a second, smaller model checks the output of the first model against a set of "Facts" before the user ever sees it.
Never forget the power of the System Prompt. By explicitly telling the model, "If you do not know the answer based on the provided context, state that you do not know. Do not make up information," you can reduce hallucinations by up to 40% even without changing sampling parameters.
"Groundedness" is the key metric here. We track our "Groundedness Score" daily to see how often the model strays from the provided documentation.
The "Two-Model" check is expensive but highly effective. We use a "Judge" model that specifically looks for policy violations in the "Worker" model's output.