We are using GPT-4 for complex tasks and Claude 3 for faster ones. Can Langfuse accurately aggregate the total cost? I need to generate a weekly report showing our spend per user ID. Is there a way to map custom model prices if we are using a fine-tuned model on an internal server?
3 answers
Yes, this is a core feature. As long as you provide the model name in your generation calls, Langfuse will match it against its internal price list for major providers. For your internal or fine-tuned models, you can define custom "Models" in the project settings and specify the price per 1k tokens for both input and output. To get the report per user ID, make sure you are passing a userId string when you initialize the trace. You can then go to the "Analytics" tab and filter the dashboard by that user ID to see exactly how many tokens they consumed and what the total dollar amount was.
Does the cost calculation handle the recent price drops from OpenAI automatically, or do we have to manually update the model definitions?
You can also export all this data to a CSV or via API if you want to build a more custom billing dashboard for your finance department.
We actually use the API export for our monthly billing cycle, and it has been incredibly accurate compared to our direct provider invoices.
The cloud version of the platform updates the prices pretty quickly after the providers make their announcements. If you are self-hosting the open-source version, you might need to pull the latest Docker image or manually update the price table in the database. I personally prefer the cloud version just so I don't have to keep track of the constant price wars between Gemini and OpenAI.