We want our users to rate LLM responses in our app. How do I link a 'thumbs up' click in my React frontend to a specific generation in Langfuse? I want to create a dataset of "good" vs "bad" responses to eventually use for fine-tuning our next model version.
3 answers
The process is straightforward: when your LLM generates a response, the SDK returns a generationId. You should store this ID in your frontend state. When the user clicks the feedback button, you call the langfuse.score() method, passing that ID along with a value (like 1 for positive, 0 for negative) and a name like "user-satisfaction." This links the score directly to the trace. Later, you can go into the dashboard and filter for all generations with a score of 1. You can then export these high-quality pairs as a JSONL file, which is the perfect format for fine-tuning models on platforms like OpenAI or Anyscale.
Can we add text comments to the feedback, or is it strictly numerical scores? Our users often want to explain why a response was helpful.
I suggest setting up an automated "Evaluation" in the dashboard that alerts you if the average user score drops below a certain threshold after a deployment.
Diane's advice saved us from a bad prompt release last week. We saw the satisfaction dip instantly and rolled back within ten minutes.
You can definitely add a comment! The score method accepts an optional comment field. We use it to capture "correction" text from our internal expert reviewers. It’s been vital for our R&D team to understand the nuances of why the model failed on certain legal queries. The ability to see the comment right next to the trace in the UI provides great context.