My Django (AI backend) often hits 504 Gateway Timeouts when calling complex LLM chains that take over 30 seconds. How are you guys handling these long-running AI tasks? Is it better to use WebSockets or a polling mechanism with a task queue?
3 answers
This is a classic architectural hurdle for a Django (AI backend). Standard HTTP requests aren't meant for 30-second waits. The most robust solution is using Celery with Redis. When a user triggers an AI task, the view should immediately return a task_id and a 202 Accepted status. Then, use Django Channels to push the result to the frontend via WebSockets once the worker finishes. This keeps your web workers free to handle other traffic. I implemented this for a legal document summarizer, and it completely eliminated our timeout errors while providing a much smoother "Loading" experience for the end users.
Kimberly, is the complexity of setting up Redis and Channels worth it if my Django (AI backend) only has a few users and infrequent long tasks?
Make sure you increase your Nginx or Gunicorn timeout settings as a temporary "band-aid" while you set up your task queue.
Correct, Karen, but that's a dangerous path! I've seen Django (AI backend) servers crash because all workers were tied up waiting for the AI to respond. Celery is definitely the professional way to go.
Daniel, if your traffic is very low, you could get away with simple "short polling" where the frontend pings an endpoint every 2 seconds to check status. But honestly, if you plan to scale your Django (AI backend) at all, starting with Celery is the right move. It prevents your server from locking up when multiple people trigger tasks at once. It’s a bit more setup initially, but it saves a massive headache later when your user base grows.