Why Iterables Are Powerful in Python — Understand These 5 Distinct Usages

Advance Your Python Skills

Why Iterables Are Powerful in Python — Understand These 5 Distinct Usages

Understand these common use cases of iterables to level up your Python coding skills

Photo by Alexey Ruban on Unsplash

What Are Iterables?

Iterables are a collection of important data structures in Python. For instance, built-in types such as strings, lists, and dictionaries are all iterables. If you have used higher-order functions, such as map and filter , you probably know that these functions create iterables too (i.e., map and filter objects). For another instance, you may have heard generators, which are also powerful iterables with memory efficiency.

These data types are common examples of iterables that we use in Python. But what exactly are iterables? The following definition is given by Luciano Ramalho, the author of Fluent Python — a nice book for more advanced Python. Basically, iterables are those that can be converted to iterators, such that their elements can be iterated to perform certain operations.

“Any object form which the iter built-in function can obtain an iterator. Objects implementing an __iter__ method returning an iterator are iterables. ”— Fluent Python by Luciano Ramalho

Iterables are so critical that they’re used in many places for varied purposes. The present article is trying to provide a systematic, but non-exhaustive review of common use cases of iterables in Python. After reviewing these use scenarios, I hope that you’ll feel more comfortable about using iterables in your Python projects.

1. Iteration Using For Loops

The most obvious usage of iterables is to iterate each item of the iterable in a for loop. Under the hood, each iterable is converted to an iterator (using the iter() function as mentioned before), which will render items to the for loops by calling the next() function. Certainly, the current article is not to provide a thorough mechanistic explanation of how iteration is implemented (see here if interested). Instead, let’s just see how iterables work in the iteration. As shown in the code snippet below, common built-in iterables can all be used in iteration. Two things are worth noting.

• Strings are iterables, and individual characters are considered items of the string objects.
• By default, dictionaries are iterated over their keys. If you need to iterate over their values and items (i.e., key-value pairs), you’ll have to use the dictionary’s values() and items() methods, respectively.

2. Collection Data Type Construction

We can use iterables to create collection data types, and some common used ones include lists, sets, tuples, and dictionaries. In simple scenarios that involve manageable amount of elements, we can use the literals to create these data. As shown at the beginning of the code snippet in the previous section, we just specified individual elements that were going to be stored in these data containers. Very straightforward, right?

However, when we need to create container data that have a large number of elements, it’s less convenient to use the literals. Notably, each of these collection data types have their own constructors using their respective class names, and they can simply take iterables to create new collection objects. The following code shows you how we can do that.

If you do data science or machine learning, you’re not probably unfamiliar with NumPy and Pandas packages. The most basic data structures in these packages all have the construction using iterables. The following code snippet shows you some basic usages.

3. Comprehensions and Generator Expressions

One particularly useful feature in Python is the comprehension technique. Specifically, we use comprehensions to create lists, sets, and dictionaries, and these techniques are termed as list comprehension, set comprehension, and dictionary comprehension. They have the following basic forms.

As shown above, all these forms include the use of one-line iteration. Every comprehension can have an optional condition evaluation — when it’s true, the item will be sent to the expression. Both list and set comprehensions have just one expression, while the key comprehension has two expressions, with one for key and the other for value. Another notable difference is the use of square brackets for list comprehension, while for the other two, curly brackets are used. You have to be clear with subtleties of these related techniques, because if you confuse with these syntaxes, you can get unexpected results or create bugs in your code. The following code snippet shows you use cases of these comprehension techniques.

Related to these comprehension techniques is the generator expression, which is used to create generators. Like the outputs from these comprehensions, the generator expression’s output (i.e., generators) is also an iterable. Unlike these iterables from comprehensions, generators can produce values when they’re requested to do so, and thus they’re very memory efficient. If you want to know more about generators, you can find more here . Let’s see the syntax and its usage of creating a generator below.

As shown above, generators need just a small amount of memory, because they don’t load all the elements as lists, in the example. Importantly, generators and lists can produce the same result when we calculate the sum of these squares using the built-in sum() function.

4. List and Tuple Unpacking

When we work with lists and tuples, we usually need to use particular portions of the data. The common approach is to use the indexing/slicing technique, which will allow us to get either individual elements or chunks of data. However, indexing and slicing can be tricky, because the used numbers appear to be magic (i.e., someone may find it hard to make sense why particular indices are used). Instead, we can use the unpacking technique, which involves assigning elements to specific variables sequence-wise. Let’s see a trivial example below.

In the above example, we use the same number of variables that match the number of elements in the iterable. However, to make the unpacking technique more powerful, Python supports unpacking using the catch-all approach. Specifically, if we use a number of variables fewer than the number of elements in the iterable, the one with an asterisk will capture all remaining elements. Notably, the particular variable will be created as an iterable (i.e., a list object). Let’s see this particular usage.

The first_number and last_number are assigned the first and last elements of the list, respectively. Interesting, the middle_numbers captures all the numbers in the middle and has a data type of list .

5. Higher-Order Functions

Another use case of iterables is their being the input (i.e., parameters) and output (i.e., return values) of some higher-order functions. The two common higher-order functions involving iterables are the map and filter functions. For the filter function, it takes an iterable and a filtering function, which will apply to each element of the iterable. When the condition is met, the item will be kept in the iterator. Let’s see how we use the filter function with iterables. As shown in the example, the filter function filters the sequence by only including those that are divisible by 3. Importantly, the generated filter object is an iterator and thus is able to be used in an iteration.

For the map function, it will take a mapping function and one or more iterables. Importantly, the number of iterables should match the number of parameters that the mapping function takes. Each element(s) from the iterable(s) will be passed to the mapping function for processing. The mapped items will be included in the output map object, which is an iterator object. Let’s see how the map function works in the following code snippet. Compared to the filter function, the map function is to convert the object using the mapping function without evaluating what elements are included.

As an important side note, even though we can use iterables in the map and filter functions, if the whole purpose of using these higher-order functions to create lists, dictionaries, sets and generators, we should consider use the corresponding comprehension techniques, because they’re more readable and considered more Pythonic.

Conclusions

In this article, we reviewed five common use cases of iterables in Python. They’re a very broad concept, and include built-in data types (e.g., strings, lists, and dictionaries). Thus, a better understanding of those usages can help you handle most of your jobs involving iterables in your Python projects.

Thanks for reading this article, and happy coding with Python.