7 Habits to Improve The Performance of Python Programs

Python doesn't excel at performance, but with some tips, you can try to improve program performance and avoid unnecessary waste of resources.

1. Use local variables

Trying to use local variables instead of global variables makes it easy to maintain and helps to improve performance and save memory.

Replace variables in the module namespace with local variables, such as ls = os.linesep. On the one hand, it can improve program performance, as the search speed of local variables is faster; on the other hand, lengthy module variables can be replaced with short identifiers to improve readability.

2. Reduce the number of function calls

When determining the object type, it's optimal to use isinstance() , sub-optimal to use id() , and worst to use type() to compare.

#Determine whether the variable num is an integer type
type(num) == type(0) #call the function three times
type(num) is type(0) #identity comparison
isinstance(num,(int)) #call the function once

Don't put the repeated operation as a parameter in a loop in order to avoid repeated calculations.

#Each loop needs to re-execute len(a)
while i < len(a):
    statement
#Only execute len(a) once
m = len(a)
while i < m:
    statement

To use a function or object Y in module X, you should use from X import Y directly instead of import X; X.Y. Thus, when using Y, you can reduce the query once (the interpreter doesn't have to find the X module first, and then look for Y in the X module's dictionary).

3. Use mapping to replace conditional search

The search speed of mappings (such as dict, etc.) is much faster than conditional statements (such as if, etc.). And there is no select-case statement in Python.

#if​ reach
if a == 1:
    b = 10
elif a == 2:
    b = 20
...

#dict reach，better performance
d = {1:10,2:20,...}
b = d[a]

4. Iterate sequence elements directly

For sequences (str, list, tuple, etc.), iterating sequence elements directly is faster than iterating element indexes.

a = [1,2,3]

#Iterate elements 
for item in a:
    print(item)

#Iterate indexes
for i in range(len(a)):
   print(a[i])

5. Replace list comprehension with generator expressions

List comprehension, which will produce an entire list, makes a negative effect on the iteration of large amounts of data.

But the generator expression doesn't. It doesn't actually create a list, but instead returns a generator which produces a value (delayed) when needed, which is more friendly to memory.

#Calculate the number of non-null characters in file f
#List analysis
l = sum([len(word) for line in f for word in line.split()])

#generator expression
l = sum(len(word) for line in f for word in line.split())

6. To be compiled first and then called

When using the function eval() and exec() to execute code, it is better to call the code object (compiled into bytecode in advance through the compile() function) instead of calling str directly, which will avoid repeating the compilation process multiple times and improve the performance of the program.

Regular expression pattern matching is similar. It is also best to compile the regular expression pattern into a regex object (via the re.complie() function) before executing comparison and matching.

7. Habits for module programming

The highest level Python statement in the module (no indented code) will be executed when the module is being imported (Whether it's really necessary to be executed or not). Therefore, you should try to put all the functional code of the module into the function (The functional code related to the main program can also be put into the main() function, and the main program itself calls the main() function).

Test code can be written in the main() function of the module. The value of __name__ will be detected in the main program. If it is '__main__' (indicating that the module is executed directly), the main() function will be called to test; if it is the name of the module (indicating that the module is called), then the test won't be executed.