How to Use Python Packages
source link: https://code.tutsplus.com/how-to-use-python-packages--cms-26000t
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
How to Use Python Packages
Python packages allow you to break down large systems and organize their modules in a consistent way that you and other people can use and reuse efficiently. Python's motto of "Batteries Included" means that it comes preloaded with lots of useful packages in the standard library.
But there are also many amazing third-party packages you can take advantage of. In this tutorial, you'll learn all you need to know about what packages are exactly, how to import modules from packages, exploring the built-in package in Python's standard library, and installing third-party packages.
What Are Packages?
Before we can talk about packages, let's talk about modules. Modules are the source files with *.py
extension where you (and everyone else) put the functions and classes that comprise your program.
A package in Python is simply a folder with multiple Python files and should have an __init__.py
file. The __nit__.py
file indicates that the directory is a package. The __ini__.py
file can be empty or contain some executable code.
Packages are the manifestation of Python's hierarchical namespaces concept. To quote from the Zen of Python:
"Namespaces are one honking great idea -- let's do more of those!"
To view the whole Zen of Python, type import this
in a Python interactive session:
>>> import this |
|
2 |
The Zen of Python, by Tim Peters |
3 |
|
4 |
Beautiful is better than ugly. |
5 |
Explicit is better than implicit. |
6 |
Simple is better than complex. |
7 |
Complex is better than complicated. |
8 |
Flat is better than nested. |
9 |
Sparse is better than dense. |
Readability counts. |
|
Special cases aren't special enough to break the rules. |
|
Although practicality beats purity. |
|
Errors should never pass silently. |
|
Unless explicitly silenced. |
|
In the face of ambiguity, refuse the temptation to guess. |
|
There should be one-- and preferably only one --obvious way to do it. |
|
Although that way may not be obvious at first unless you're Dutch. |
|
Now is better than never. |
|
Although never is often better than *right* now. |
|
20 |
If the implementation is hard to explain, it's a bad idea. |
21 |
If the implementation is easy to explain, it may be a good idea. |
22 |
Namespaces are one honking great idea -- let's do more of those! |
23 |
>>> |
Namespaces help with organizing code and preventing naming conflicts. This is critical when multiple people work together or when using packages developed by other people.
While packages represents a hierarchy of sub-packages and modules, which are files the hierarchy doesn't have to file-system based where packages and sub-packages are directories and sub-directories. It is much more flexible than that.
Create a Python Package
Let's start with a simple example. Below we have a package called simple_package
with 2 Python modules.
simple_package |
|
2 |
. |
3 |
├── __init__.py |
4 |
├── tasks.py |
5 |
└── views.py |
6 |
|
7 |
0 directories, 3 files |
- __init__.py : indicates that its a package
- tasks.py and views.py are modules
Third-Party Packages
Let's take a look at a package called ansible. It is not a package from the standard library. You'll see later how to find and install third-party packages. Now, let's just check out the directory file structure.
The packages will typically be installed into the Python interpreter's site-packages directory, located somewhere (depending on version, OS, and distribution) under lib.
On the Mac, for example, Python 3.10 will be located in <interpreter root>/lib/python3.10/site-packages. Here is how the ansible package is organized:
tree ansible -L 1 |
|
2 |
ansible |
3 |
├── cli |
4 |
├── collections |
5 |
├── compat |
6 |
├── config |
7 |
├── constants.py |
8 |
├── context.py |
9 |
├── errors |
├── executor |
|
├── galaxy |
|
├── __init__.py |
|
├── inventory |
|
├── keyword_desc.yml |
|
├── __main__.py |
|
├── modules |
|
├── module_utils |
|
├── parsing |
|
├── playbook |
|
20 |
├── plugins |
21 |
├── __pycache__ |
22 |
├── release.py |
23 |
├── template |
24 |
├── utils |
25 |
├── vars |
26 |
└── _vendor |
27 |
|
28 |
18 directories, 6 files |
There are 6 modules and 18 directories. Each directory is a sub-package of the main ansible package. Looking inside the ansible/utils directory, we can see it contains additional modules and even one more sub-package:
tree ansible/utils -L 1 |
|
2 |
ansible/utils |
3 |
├── cmd_functions.py |
4 |
├── collection_loader |
5 |
├── color.py |
6 |
├── context_objects.py |
7 |
├── display.py |
8 |
├── encrypt.py |
9 |
├── fqcn.py |
├── galaxy.py |
|
├── hashing.py |
|
├── helpers.py |
|
├── __init__.py |
|
├── jsonrpc.py |
|
├── _junit_xml.py |
|
├── listify.py |
|
├── lock.py |
|
├── multiprocessing.py |
|
├── native_jinja.py |
|
20 |
├── path.py |
21 |
├── plugin_docs.py |
22 |
├── py3compat.py |
23 |
├── sentinel.py |
24 |
├── shlex.py |
25 |
├── singleton.py |
26 |
├── ssh_functions.py |
27 |
├── unicode.py |
28 |
├── unsafe_proxy.py |
29 |
├── vars.py |
30 |
└── version.py |
31 |
|
32 |
1 directory, 27 files |
33 |
The Search Path
When you import a module, Python will go through a search algorithm based on the search path, which is a list of directories to start the search. The search path is a list of directories available through sys.path
, and you can manipulate it dynamically (add, remove or move around items in the search path). The site-packages directory is always there.
To import the path.py module from ansible/utils, you'll need to use the following command:
import ansible.utils.path |
To import both path
and encrypt
modules,, you use the following commands:
import ansible.utils.path |
|
2 |
import ansible.utils.encrypt |
If you also want to use the standard os.path module, you'll use the following command:
import os.path |
Now you can use either or both path modules with no conflicts due to the difference namespace they belong to.
Exploring the Standard Library
The standard library has a lot of packages. It's worth exploring it whenever you need to accomplish some task and you're not sure how. There is a very high likelihood that for any general-purpose task like math, shell integration, OS integration, string manipulation, networking and common file formats, there is a well-designed, well-performing and well-tested package in the standard library.
You can really trust standard library packages because it is a big deal to get into the standard library. Either the package was designed by Python's core developers or it was heavily reviewed and often heavily used in the field as a third-party library before making it into the standard library.
Here are all the packages in the standard library organized by topic.
The standard library is awesome, but there'll often be some special functionality you need that is not standard. It doesn't mean you have to write it from scratch. Python has a vibrant and active community that develops and shares freely a lot of code. Enter PyPI - the Python Package Index. PyPI hosts all publicly available packages and provides a one-stop shop for browsing through them.
Browsing PyPI
PyPI organizes the packages in a browsable index. You can browse and search by topic, environment, framework, development, status, intended audience, license, natural language, programming language (yes, there are Python packages that support many programming languages), and operating system.
As of 2021, PyPI does not display download statistics for packages since it's ineffective due to the resources required to maintain the statistics.
Installing Packages
There are two ways to install packages from PyPI. You can download the package and then run python setup.py install
. But the modern way is to use pip, setuptools and wheel.
Pip and setuptools are included by default starting from Python 3.4 and Python 2.79, but you will need to upgrade to the latest version:
- Linux /macOS:
pip install -U pip setuptools
- Windows:
python -m pip install -U pip setuptools
Python 2 is, however, no longer supported, so you should already be using Python 3.0 or higher for improved performance.
Use pip to install wheel:
pip install wheel. |
To install a package with pip, you issue this command.
pip install <package_name> |
Where package_name
is the name of the package. For example, to install Ansible, the command would look like this:
pip install ansible |
If you need a specific version, you can also specify it as follows:
pip install ansible==7.0 |
Python packages are always installed into an environment. A common practice I will not cover here is to use virtual environments to manage multiple independent installations of Python with different interpreters and/or different sets of installed packages. You can read more about Virtual environments here
Best Practices
The Python packaging authority provides a lot of guidance on the best practices around packaging. This is important because it is an area of active development and recommendations evolve quickly.
Also, if you want to do something special like installing packages from alternative repositories instead of PyPI or using pip in a more sophisticated way, you'll find great discussions and practical advice.
Conclusion
When you're a Python beginner, you learn the core language and have fun playing with it. Pretty soon you discover the standard library, and as you gain more experience you benefit more and more from its richness.
The next stage in your evolution as a Pythonista is to incorporate the vast awesomeness the Python community has put on PyPI into your systems. Packages as the deployment unit of reusable Python code enable this ecosystem.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK