Django File uploads to S3 and Cloudfront CDN

In this article I will cover how to setup up Django 4.2 to upload files to an AWS S3 bucket and serve files via CloudFront CDN.

In this article:

Requirements¶

• A Django project with Django version 4.2
• django-storages package in requirements.txt
• AWS Account
• Application deployed with Appliku

Create AWS S3 Bucket for Django media files upload¶

1. Go to AWS interface for S3 service: https://s3.console.aws.amazon.com/s3/buckets?region=eu-central-1
2. Create a bucket https://s3.console.aws.amazon.com/s3/bucket/create?region=eu-central-1
3. Give bucket, pick a region, ACL enabled, object writer, remove checkboxes "Block all public access":
4. Scroll down and click the "Create bucket" button.
5. On the top right of the screen find your username, click on it to open a dropdown and open the link "Security Credentials" https://us-east-1.console.aws.amazon.com/iam/home?region=eu-central-1#security_credential
6. Click on "Policies" in the left sidebar
7. Click on the "Create policy" button
8. Open JSON tab. and paste this JSON into the field, replace mywriteonlybucket with your bucket name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::mymediabucket337/*",
                "arn:aws:s3:::mymediabucket337"
            ]
        }
    ]
}

Now click on "Next: Tags", then "Next: Review"

Give this policy a name ad click the "Create policy" button 9. Now go to Users section 10. Click on the "Add users" button 11. Give your new user a name, leave "Provide user access to AWS Management Console" unchecked and click "Next" 12. Pick "Attach policies directly" 13. Find the policy that we created earlier and check it in the list, click next 14. Click the "Create user" button 15. You will be taken to the list of all users. Find your new user, click on it. 16. Click on the tab "Security credentials"![] 17. Click "Create access key" 18. Pick "Command Like Interface (CLI)", check that you understand their recommendations and click "Next" 19. Click the button "Create access key".

Grab your keys and save them safely and don't let anyone get their hands on your keys!

Setup Django Storages for S3 Media uploads¶

Make sure to install and add these dependencies to your requirements.txt:

Requirements¶

boto3==1.26.142
boto3-stubs==1.26.142
django-storages==1.13.2

Create a file storages.py next to your settings.py file and put this code in it:

from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage


class PublicMediaStorage(S3Boto3Storage):
    location = 'media'
    default_acl = 'public-read'
    file_overwrite = False
    access_key = settings.MEDIA_S3_ACCESS_KEY_ID
    secret_key = settings.MEDIA_S3_SECRET_ACCESS_KEY
    bucket_name = settings.MEDIA_S3_BUCKET_NAME
    custom_domain = settings.MEDIA_HOST
    querystring_auth = False

In settings.py make sure to grab environment variables into respective variables: e.g. with django-environ:

import environ
env = environ.Env()

MEDIA_S3_ACCESS_KEY_ID = env('MEDIA_S3_ACCESS_KEY_ID', default=None)
MEDIA_S3_SECRET_ACCESS_KEY = env('MEDIA_S3_SECRET_ACCESS_KEY', default=None)
MEDIA_S3_BUCKET_NAME=env('MEDIA_S3_BUCKET_NAME', default=None)

Or with django-decouple package:

from decouple import config

MEDIA_S3_ACCESS_KEY_ID = config('MEDIA_S3_ACCESS_KEY_ID', default=None)
MEDIA_S3_SECRET_ACCESS_KEY = config('MEDIA_S3_SECRET_ACCESS_KEY', default=None)
MEDIA_S3_BUCKET_NAME=config('MEDIA_S3_BUCKET_NAME', default=None)

In settings.py make sure to set the STORAGES dict's default key to our media storage

STORAGES = {"default": {"BACKEND": "project.storages.PublicMediaStorage"}}

Replace project with the name of the folder your settings.py and storages.py files are located in.

Also in settings.py specify MEDIA_URL & MEDIA_HOST:

MEDIA_ROOT = 'media'
MEDIA_HOST=f'{MEDIA_S3_BUCKET_NAME}.s3.amazonaws.com'
MEDIA_URL=f'https://{MEDIA_HOST}/'

Commit and push your code.

Then in Appliku dashboard go to your Application, click on Settings, go to Environment Variables tab.

Add three environment variables with credentials and the bucket name:

Then click "Save and Deploy"

This should be enough configuration for your Django app to use S3 bucket for media files. You can try how it works with Django admin in any model where you have a FileField or an ImageField.

It should upload files fine and uploaded files should be accessible as well.

Setting up CloudFront CDN serving Django media files from S3 bucket¶

Go to AWS interface for CloudFront service: https://us-east-1.console.aws.amazon.com/cloudfront/v3/home?region=us-east-1#/distributions

Click "Create distribution"

Click on the "Origin Domain" and select your bucket:

Wait until distribution is deployed

Looks like distribution is ready:

I have uploaded a file into the bucket so we can test if the distribution works as expected.

If we access the file from the bucket we see this pixelart dude:

Let's copy the distribution domain and replace the domain name in the URL to make sure that our image opens

As you can see it does!

Now in order to educate Django to serve media files from the CloudFront CDN we need to edit the MEDIA_URL setting. Open settings.py and replace MEDIA_URL with this(example with django-environ)

MEDIA_ROOT = 'media'
MEDIA_URL=f'https://{MEDIA_S3_BUCKET_NAME}.s3.amazonaws.com/'
MEDIA_HOST=f'{MEDIA_S3_BUCKET_NAME}.s3.amazonaws.com'
CUSTOM_MEDIA_DOMAIN = env('CUSTOM_MEDIA_DOMAIN', default=None)

if CUSTOM_MEDIA_DOMAIN:
    MEDIA_HOST = CUSTOM_MEDIA_DOMAIN

MEDIA_URL=f'https://{MEDIA_HOST}/'

Set the env variable CUSTOM_MEDIA_DOMAIN in Appliku dashboard to the domain that you copied from CloudFront distribution. Pay attention to the fact that you need to provide it without https:// like this:

dsxl96eyyri74.cloudfront.net

That's it. Now when generating URLs to uploaded media files Django will use the CDN URL and your media files will be served faster and cheaper for you(refer to AWS CloudFront and S3 pricing for more details).