My users are uploading 500MB video files to my Django site, and it's making the whole application hang. I'm currently using a standard FileField in a ModelForm. I know I should probably be using an asynchronous approach or a task queue like Celery, but I’m not sure how to handle the initial upload part. Does the file have to hit my Django server first, or can I upload it directly to S3?
3 answers
You should use a client-side upload tool like Uppy or Dropzone. They handle chunked uploads and retries, which is essential for 500MB files to prevent failures.
For files that large, you definitely want to avoid routing the data through your Django process. The "Presigned URL" strategy is the industry standard for AWS S3. Your Django server generates a temporary, secure URL and sends it to the frontend. The user's browser then uploads the file directly to S3. Once finished, the frontend pings your Django API with the file metadata. This completely bypasses your server's memory and CPU, preventing the "hanging" you're seeing. It also saves you a ton on bandwidth costs since the data doesn't travel through your backend twice.
If you use presigned URLs, how are you planning to handle file validation? You can't easily check the file type or size until after it is already on S3.
Daniel, you can actually set constraints in the presigned post policy, like content-length-range. For post-upload processing like generating thumbnails or transcoding, that's where Celery comes in. You can trigger an S3 Lambda function or a Celery task that pulls the file from S3 once the upload is complete. This keeps your web worker free to handle other requests while the "heavy lifting" happens in the background. It's the most scalable way to build a media-heavy application.
Betty is right. Using a robust frontend library alongside S3 presigned URLs is the perfect combination for a smooth user experience with large media.