TIL: You Can Build Portable Binaries of Python Applications

Contrary to popular belief, it’s possible to ship portable executables of Python applications without sending your users to Python packaging hell.

I’m hard at work wrapping up doc2dash 3.0 – a dear project of mine which is now over 10 years old. It’s also a Python application. And unless packaged into a more user-friendly format like Homebrew or a Linux distribution package, enduser Python applications have a bad reputation for good reasons: if you’re not somewhat competent in Python’s packaging landscape (or at least know how to use pipx) it can get unpleasant.

Given that doc2dash manages to be both my oldest and my least successful project at the same time, I always suspected the Python packaging angle to be part of the problem. Over the years, I’ve tried all application bundlers available, but none of them worked: they all failed with varying obscure errors1.

Then came PyOxidizer. Gregory Szorc has been patiently building a whole toolchain to make Python applications – and Python itself! – portable.

Long story short, doc2dash 3.0 is going to have portable binaries for Linux, macOS, and Windows.

Here’s how:

• pyoxidizer.bzl is the build script. python_config.run_command defines the CLI entry point that is called when the binary is started. exe.add_python_resources() installs my (per-platform pinned) requirements and the application itself.
• noxfile.py2: I have one target (pin_for_pyoxidizer) to pin the dependencies for reproducible builds and one (oxidize) to build a release binary. Passing standalone_static for Windows builds ensures that I get a single .exe file (the option doesn’t work on macOS and on Linux you get musl which can have performance problems). Finally, the download_and_package_binaries target downloads the artifacts for the latest Git tag, makes Linux and macOS binaries executable, and builds per-architecture Zip files together with the a concatenation of all licenses of all packages that are within the binary.
• .github/workflows/pyoxidizer.yml runs the oxidize target on Linux, macOS, and Windows, checks the binaries work, and uploads them as artifacts. If you want, you can try out a binary from a recent test build (and let me know if there’s problems!).

This is just a TIL and not a proper blog post, because there’s a lot I’d still like to improve in my PyOxidizer usage – like figuring out Apple Silicon builds. That said, people using it longer and more intensely than me seem happy!

Either way, as someone who’s maintained a Python application for 10 years whose majority of bug reports were packaging-related, it’s exhilarating that I even got this far. And I want the Python community to know about this gem.

Given it’s solving a monumental problem (Python packaging isn’t problematic because people are evil or obtuse – it’s problematic because of the diversity and complexity of the ecosystem), PyOxidizer’s can’t be perfect: as of writing it has 288 issues open. For instance, since it operates on somewhat virtual filesystems, packages that rely on the __file__ variable instead of the importlib packaging machinery (case in point: Sphinx) won’t work (there might be workarounds – I haven’t checked).

But Gregory’s toolchain is one of the most exciting things happening to Python right now and I hope he’ll be able to keep working on it (meaning: FAANG, please throw money at him!).

Hynek Schlawack

Code Bohemian in ❤️ with Python 🐍, Go 🐹, and DevOps 🔧. blogger 📝, speaker 📢, PSF fellow 🏆, big city beach bum 🏄🏻, substance over flash 🧠.

Is my content helpful and/or enjoyable to you? Please consider supporting me! Every bit helps to motivate me in creating more.