A complete guide to organizing settings in Django

A complete guide to organizing settings in Django

Published on November 11th, 2021 in Django/Python

How do you organize your Django settings? In this article, we describe a simple approach we refined through many iterations. Our goal is not only to present you with a solution, but also to explain why it's better than the alternatives, and what those alternatives are.

Django, by default, keeps all the project settings in a single monolithic settings.py file. This is what gets created when you generate a new project using Django's command-line tools. It is simple.

Problem is, it is too simple. The monolithic approach doesn't fit well with how we develop, share and deploy our code:

• It mixes project architecture and configuration (eg. installed apps and secret keys).
• It should be committed to the source code repository but contains secrets.
• Configuration differs between developers and various testing, staging, and production servers.

Those are old and known problems with the default approach. Over the years, people have come up with various improved approaches.

Throughout our work as a web development agency specializing in Django, we went through several approaches, adapting them as our needs and development practices changed.

local.py

The first improvement that many Django developers make to default is to have a local settings file. They create a local.py that gets imported at the end of settings.py, which only hosts the defaults. The local settings file is not committed to the repository, everyone has a separate copy.

This solves the difficulties with a single monolithic file:

• Project architecture is still kept in settings.py. Configuration is now done in local.py.
• The main settings file can be committed to the code repository without problems, as all the secrets are in the local settings.
• Everyone has their version of local settings.

But this created a new set of problems:

• On more complex projects, local settings contained more logic (for example, to select the correct variant of the setting for dev/prod/test) and that ended being duplicated between users, sometimes leading to slight inconsistencies
• local.py couldn't reference default setting values from settings.py as that would create a circular import
• It was easy to slip up and change something in local settings where the change ought to have been made in the global settings

We doubled down on this approach and created a more complex settings layout:

settings/__init__.py

This is another popular layout. Instead of a single settings.py module, have a settings package (directory with __init__.py), that contains the base settings, settings files specialized for a specific deployment type (development, testing, production), local settings file (like before), and a wrapper that just imported everything.

The structure looks like this:

settings/
    __init__.py
    base.py
    development.py
    testing.py
    production.py
    local.py

The base.py file is mostly the original settings.py. Deployment-specific settings then tweak the config as needed. For example, for development.py we might have:

from .base import *

DEBUG = True

# Other development settings go here ...

And the __init__.py module contains this:

from .base import *
from .local *

In local.py, we can then import whichever development type we want to extend and change the few extra settings.

This has the added benefit of being able to select alternative settings at invocation time. For example in development, we'd use local settings that are tweaked for our local use, but run tests with test settings so we have the same settings as on the test server:

DJANGO_SETTINGS_MODULE=settings.test python.manage.py test

Over time, this approach also showed drawbacks:

• it's complex
• it doesn't support the case where parts of the configuration are via environment variables (such as on Heroku, Docker, or other popular PaaS)
• it requires a local file to be present on the filesystem that's not in the repository (again a problem with Heroku and Docker)

Settings package with env support

To solve the problem with environment variables, we adapted our approach by also having an env.py that can read variables from the environment. Our local.py then became:

from .base import *
from .dev import *

# Any additional settings here ...


from .env import *  # Override from environment vars

To avoid the local file problem on Heroku, we modified the settings/__init__.py module to contain:

from .base import *

try:
    from .local import *
except ImportError:
    pass


from .env import *

This worked well. This worked so well that we just kept adding everything to env variables and kept our local settings file minimal.

This leads us at last to the current twelve-factor approach, of having everything in the environment.

python-dotenv

Juggling dozens of environment variables gets tiring pretty fast, so most systems that read from env support the .env file containing VARIABLE=value pairs. It gets loaded into the current environment at the service start.

There's a popular package python-dotenv that does just that. The only thing we have to do is to initialize the Django settings with the values obtained via dotenv.

We can now go back to only having settings.py that imports dotenv, fetches the configuration, and initializes Django settings as needed.

There's only a teeny-tiny problem left: dotenv gives us all strings and sometimes we want integers, booleans, file paths, lists... We want to keep this config parsing in a single place, which brings us to the current best practice for handling Django settings:

How to handle Django settings

If you've read everything so far, congrats! You've now seen a couple of popular ways how not to handle Django settings.

And for those that skimmed to here and are just joining us, here's how you want to organize your settings files in a Django project:

settings/
    __init__.py
    env.py
.env
.env.sample

Contents of settings/__init__.py are your standard old Django settings, but instead of hardcoding values there, use functions that parse them from the environment.

For example (this is not a complete settings file):

from .env import *

# Interpret DEBUG env variable as a boolean value and use it. Default to False if not set.
DEBUG=ENV_BOOL('DEBUG', False)

...

# Static path is usually `static/` in project root, but can be overridden
STATIC_ROOT = ENV_STR("STATIC_ROOT", ABS_PATH("static"))

...

The env.py loads the environment and provides helpers, and looks a lot like this:

import os
from dotenv import load_dotenv

load_dotenv()

__all__ = [
    "BASE_DIR", "ABS_PATH",
    "ENV_BOOL", "ENV_STR", "ENV_INT",
    "ENV_DEC", "ENV_LIST",
]

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))


def ABS_PATH(*args):
    return os.path.join(BASE_DIR, *args)


def ENV_BOOL(name, default=False):
    if name not in os.environ:
        return default
    if os.environ[name].lower() in ["true", "yes", "1"]:
        return True
    elif os.environ[name].lower() in ["false", "no", "0"]:
        return False
    else:
        return default


def ENV_STR(name, default=None):
    return os.environ.get(name, default)


def ENV_INT(name, default=None):
    try:
        return int(os.environ.get(name, default))
    except ValueError:
        return default


def ENV_DEC(name, default=None):
    from decimal import Decimal

    try:
        return Decimal(os.environ.get(name, default))
    except ValueError:
        return default


def ENV_LIST(name, separator, default=None):
    if default is None:
        default = []

    if name not in os.environ:
        return default
    return os.environ[name].split(separator)

By the way, this is a complete, working, example, released here unto Public Domain. Feel free to use and modify as you wish.

The .env file is the file holding actual configuration for the local machine and is not committed to the repository, and the .env.sample is an example version of .env that contains instructions for the reader on how to set the settings and provides reasonable defaults (where possible). For example:

# Environment-based settings; copy this file to .env in project root and edit as needed

# Whether the application runs in debug mode or not
DEBUG=true

# Set a unique secret key for the project
# SECRET_KEY=

# Database URL to use
DATABASE_URL=

The env.sample file should contain all the settings that can be influenced via environment variables, so it's easy for new team members to onboard and easy to set up new deployments or test pipelines.

Further work

This setup has stood the test of time for us in dozens of projects through many years of agency and in-house work. It's simple, follows the best DevOps practices, and is compatible with many tools and platforms.

That's not to say you couldn't push it a bit further. Introducing a bit of extra logic in settings can make defaults saner and further decrease the amount of configuration needed.

For example, the settings file for a project generated with API Bakery support different defaults based on the value of DEBUG. It also makes the .env file more user-friendly by cleverly interpolating or interpreting some settings like the log level or email backend.

Here's an excerpt from the generated env.sample:

# Set a unique secret key for the project, required for running outside DEBUG mode
# SECRET_KEY=

# Log level (default is INFO, DEBUG is very verbose)
# LOG_LEVEL=INFO

# Allowed hosts (list of comma-separated hostnames, or asterisk to match all hosts), only needed if DEBUG is false
# ALLOWED_HOSTS=

# E-mail backend to use, defaults to "smtp" if DEBUG is false, and "console" if DEBUG is true
# EMAIL_BACKEND=console

The point here is not to make the configuration too magical, but to make it easy to get started with a new project.

Speaking of, have you yet tried using API Bakery to generate a new Django project? Details like the one we've been discussing here are our bread and butter and combined make a real difference each time you start a project.

We offer a 14-day free trial with no strings attached so you can play around and see for yourself.

---

About API Bakery

API Bakery is a backend code generator for API services that creates clean, documented and tested code that you can easily adapt, extend and maintain.

Save hundreds of hours writing standard boilerplate code on every new project start, and focus on the real "meat" of your app.

Try for free No credit card required