We have a major problem with "Shadow AI"—employees using unauthorized LLMs or browser extensions that might be leaking sensitive proprietary code to public models. I’m looking for ways to block these without hurting the team's productivity. Has anyone implemented a "Corporate Proxy" for AI that logs all prompts and redacts sensitive info like API keys or PII before sending it to the cloud?
3 answers
We faced this exact issue last year. We decided to stop fighting the "blocking" war and instead provided a sanctioned internal "AI Gateway." This gateway uses a Data Loss Prevention (DLP) layer that automatically scans every prompt for regex patterns matching our internal IP addresses, customer emails, and private keys. If a match is found, the prompt is blocked, and the user gets a notification about why. This way, the devs get the tools they want, and the security team gets the visibility they need. It’s a win-win that stopped people from trying to bypass the firewall.
Sharon, that sounds like a massive undertaking to build. Did you buy an off-the-shelf solution for that DLP gateway or develop it in-house?
We just blanket-blocked all AI domains and only whitelisted them for the R&D department. It’s a bit harsh, but it’s the only way to be 100% sure for now.
Blanked-blocking usually leads to people using their personal phones or VPNs to bypass it. The "Gateway" approach Sharon mentioned is much more effective in the long run.
Philip, we actually started with an open-source tool called "LangChain" to build the basic proxy and then integrated our existing enterprise DLP rules. It took a team of two about a month to get it stable. Since then, we've seen a 90% drop in unauthorized AI traffic because the internal tool is actually faster and more context-aware than the public ones.