I'm trying to add observability to my app without changing much code. Can I use Helicone by simply changing the baseURL in the OpenAI client? I am worried about whether the headers for caching and custom properties will interfere with the standard request structure when running on an AWS Lambda.
3 answers
Integrating this proxy is remarkably straightforward. In your OpenAI configuration, you just set the baseURL to https://oai.hconeai.com/v1 and add the Helicone-Auth header with your API key. For AWS Lambda, this is ideal because it adds negligible latency since it's a transparent proxy. To track specific segments like "user_id" or "job_type," you can use the Helicone-Property- prefix in your headers. The proxy strips these out before forwarding the request to OpenAI, so the model provider never sees your internal metadata. We’ve been using this for six months, and the visibility into our request costs and completion tokens has been a total game-changer for our monthly cloud budget.
Does the proxy approach handle the new "streaming" responses correctly? I’ve had issues with other middleware cutting off the stream before the final token arrives.
Make sure you set up the 'Cache' headers if you want to save money on repetitive prompts. It can serve results directly from the edge.
Alan is spot on. Using the Helicone-Cache-Enabled: true header reduced our testing costs by nearly 30% during our last development sprint.
It handles streaming perfectly. Because it acts as a pass-through proxy, the Server-Sent Events (SSE) reach your frontend just as they would normally. The only difference is that Helicone logs the full aggregated response in the background once the stream finishes. I tested it with the latest GPT-4o models, and the latency was virtually indistinguishable from a direct connection.