I’m building an agent that needs to access our private internal API. I can’t figure out the best way to pass the API keys securely into the tool function so the agent can use it. Should the keys be part of the tool's environment or passed as arguments? I’m worried about the LLM accidentally leaking the keys in its "Thought" process or response.
3 answers
You should never let the LLM see the actual API key. The best practice is to define your tool function to pull the key directly from an environment variable or a secret manager inside the Python code. The tool’s "description" tells the LLM what the function does and what parameters (like user_id or query) it needs to provide. When the agent calls the tool, the Python backend executes the request using the hidden key. This way, the key is never part of the prompt context and cannot be leaked in the model's output or reasoning trace.
Does this approach still work if the agent needs to handle Oauth2 tokens that expire every hour and require a refresh flow?
Just use .env files and os.getenv(). It keeps your credentials out of your code and out of the LLM’s reach entirely.
Simple and effective, Patrick. Keeping the "brains" (LLM) away from the "keys" (API) is the golden rule of AI security.
Victor, for Oauth2, I usually wrap the API logic in a class that handles the token refresh automatically. The tool itself just calls a method on that class. The agent remains completely unaware of the token exchange happening in the background; it just sees a successful data retrieval. It keeps the agent's logic simple while the heavy lifting of security is handled by standard Python libraries like requests or httpx.