Happiness is a good PYTHONSTARTUP script

Summary

Python shells are incredibly useful, but every time your start one, you tend to repeat the same things over and over.

PYTHONSTARTUP can help with this: set it up to a Python script, and it will be executed automatically if, and only if, a Python shell starts. Anything that is imported or defined in the start up script will be available in the shell.

Python is as good as the shell is

Even with the best IDE in the world and a lot of experience under your belt, the Python shell is still a formidable tool for exploratory programming. Testing a new API, mangling some data, automating some boring task...

That's why projects like ipython and jupyter became so successful, and I still use them to this day.

However, every time you start a shell, the first thing you usually do is import a bunch of stuff, or frenetically press the top arrow key to recall something from your history. This is aggravated by the fact Python has very limited support for reloading changed modules in a shell, so restarting it is a common thing.

But there is a rarely known shell feature that can make all those REPL back and forth a more enjoyable experience: the PYTHONSTARTUP environment variable.

This article is why I wrote "Environment variables for beginners" a few days ago: this trick should be available for everybody.

Running a script when a shell starts

PYTHONSTARTUP can be set to a python module path. Here is what is in mine:

echo $PYTHONSTARTUP
/home/user/Scripts/pythonstartup.py

The file doesn't need to be called "pythonstartup.py", but I like to make it explicit.

This file is a regular Python module, it's nothing special. It doesn't need to be in a specific location, it can contain any legal Python code, and it will work even if it's not importable.

Now the nice thing is, when a Python shell starts, this file will be automatically executed.

This happens only when a Python shell starts, not when a regular Python program starts. None of your scripts will be affected. This is solely a tool for making your life easier in an interactive session.

What to put in your startup script?

To understand how useful this is, I must add one detail: the entire namespace of the script is made available in your shell.

This means anything you import, anything you set in a variable or any function you define in this startup script will now always be available in your shell at startup.

You don't have to have a single script. You can have one script for each project if you wish, and make your shell load all the things you need to work on that specific project.

Because this works in ipython and jupyter, I've seen a lot of people at least preloading their favorite data manipulation library like this:

from pandas import pd
from numpy import np

You start your shell, numpy and pandas are ready to go.

You can do that in a more generic manner by catching ImportError:

try:
    from pandas import pd
except ImportError:
    pass
try:
    from numpy import np
except ImportError:
    pass

This way, if they are not installed, the script still works.

But even without going that far, you can imports things you use a lot from the stdlib, such as json, datetime classes, pathlib.Path and so on...

You can even open a file and load the data or configuration from it, or create a connection pool.

django_extensions provides the shell_plus command to load all the ORM models so you can query your DB in the current project, but you can do something similar with PYTHONSTARTUP manually with any project.

A real life PYTHONSTARTUP script

Tips and trick

• Be careful if you have a single startup script but several versions of Python. At least make sure to use a compatible syntax and try/except when it matters.

• Make some speed test. Some libraries like scipy or pandas are quite heavy, it may slow down your shell startup time.

• Use del if there is something you imported or defined but don't want to end up in the shell name space