Where does Python stand right now in terms of speed?
Compared to other programming languages like Java, Go, C++, etc, Python is a relatively slow programming language which makes its execution take more time.
When we compare popular frameworks like Django which is built on Python and Gin which is built on Go, we can see that the framework built on Go runs more requests per second which is approximately over 1,00,000 requests/sec, whereas Django runs approximately over 8,000 requests/sec, this shows that Go is a lot faster than Python.
The main reason behind this slow execution of Python code is because it is a dynamically typed language.
Let us take an example of Java which is a statically typed language, it runs all the required checks and compiles the code before runtime and this in turn optimizes the program and makes it run faster.
Whereas Python is compiled at run time, therefore any variable type or value can change while the program is running. For this reason, Python code cannot be compiled beforehand, and therefore, the code cannot be optimized at runtime.
Another reason behind this slow execution is, as it is an interpreted language the code needs to be converted into machine language before execution and this also takes time and makes the language slower.
How to install the beta version of Python 3.11?
The beta version of Python 3.11 is available for download from “python.org” website.
After we have both the versions installed on our system, to run speed tests we need to create separate virtual environments and set each Python version to each environment using these commands:
virtualenv env10 --python=3.10virtualenv env11 --python=3.11
Once the virtual environments are created, we can check the python versions in each of the environment.
After this is done, we are all set to run some performance benchmarks.
Is there any difference in the speeds of Python 3.11 and 3.10?
Python in its 3.11 release notes states that “CPython 3.11 is on average 25% Faster than CPython 3.10 when measured with the pyperformance benchmark suite and compiled with GCC on Ubuntu Linux. Depending on your workload, the speedup could be up to 10-60% faster. This project focuses on two major areas in Python: faster start up and faster runtime.”
To check those claims let us create a function to generate Fibonacci numbers and run some speed tests.
We will use this code to compare the performance between both the versions.
Let’s run our program with Timeit to check the execution times in each version. To repeat the generation process 10 times and get the best execution time, this command can be used:
python -m timeit -n 10 "from <filename> import fib;fib(n)"
Here are the results for each of the tests.
Python 3.10 results:
Python 3.11 results:
We can see that the newer Python outperforms its older counterpart in every attempt. Python 3.11 seems to be about 1.5x faster.
But let us go a bit further and try one more test.
Using Bubble Sort to Check Speed Improvements.
In the code below, we will try to measure the time taken to sort 10,000 numbers.
This code will generate random numbers in range 1 to 10,000, but the “timeit” function will only measure the time taken to execute the bubble sort function.
The output that we get after running this program in both python versions is given below.
Python 3.10:
Python 3.11:
Python 3.11 took about 17.8 seconds to sort while Python 3.10 took over 24 seconds. And this difference seems to only increase as the numbers generated are increased.
So, how did they achieve this drastic increase in speed?
This increase in speed can be understood by looking into the Faster CPython project.
This project involves 3 important enhancements:
- Faster Startup
- Faster Runtime, and
- Python Enhancement Proposal (PEP) 659: Specializing Adaptive Interpreter
Faster Startup:
The compiled versions of the modules under the name “module.version.pyc” is cached by Python in “pycache” directory to increase the pace of loading the modules where the format of compiled file is encoded by the version.
Python module execution in previous versions looked like this:
Read __pycahe__ -> Unmarshal -> Heap allocated code object -> Evaluate
In Python 3.11, steps for module execution process have been reduced drastically by the introduction of “frozen” core modules which are crucial for Python startup, which translates to static allocation of code objects by the interpreter.
This reduced module execution process can be seen as:
Statically allocated code object -> Evaluate
According to “python.org” this makes interpreter startup 10-15% faster in Python 3.11.
Faster Runtime:
Another separate improvement is how Python uses less stack space and fewer execution frames for each function call for a greater efficiency.
Frames are objects that hold information about program state during a function call and with Python 3.11 they are only created when they need to be. Earlier, they used to be created with every single call whether they were being used or not.
Most function calls now do not even use any stack space in the runtime either. So, they are more memory efficient, and they are more CPU efficient. And we can clearly see these improvements in the tests that we did earlier.
Python Enhancement Proposal (PEP) 659: Specializing Adaptive Interpreter:
One of the biggest new changes in Python 3.11 is the introduction of a new mechanism in the Python interpreter called the Specializing Adaptive Interpreter. It is defined in Python Enhancement Proposal 659.
According to this, whenever Python encounters a situation where the types of a given operation are predictable it will use byte codes that are optimized for those specific types and this can yield up to a 25% speed up in your code automatically. You don’t have to do anything to your code, all you have to do is run it on python 3.11.
Let us see an example of Adaptive Interpreter at work.
In the above code we have a loop that performs a numerical operation. All the types in this loop are stable and predictable.
If we run this code with Python 3.10, this is the result we get:
We can see that it completes it in a few seconds.
But, when we run it with Python 3.11, this is the result:
We can clearly see that it is consistently a little bit faster. That is about 15-20%, and this is with no work on our part. This is entirely the interpreter handling this.
Let us see another example of the Adaptive Interpreter at speeding things up. We can see it when we have calls to common built-in functions implemented in C, such as the “len()” function which gets the length of an object. The runtime will swap in byte codes that makes calling to those C function faster, so they sidestep the normal calling mechanisms.
In this example I have a simple function call that just returns the length of a list.
When we run it with Python 3.10 it runs quick already.
But, when we try running it with Python 3.11, it runs even faster.
This was all the improvement with no work on our part, all we did was swap in the new interpreter.
Final Thoughts:
Python 3.11 seems like a terrific release in Python’s history.
It is worth noting that Python 3.11 is still not out yet, so we are comparing a stable version to a possible release candidate. But still, we can see significant improvements when we compare the two versions.
In the tests we can see a good amount of improvement in execution speeds. A significant amount of overhead has been shaved right off. These may just be the first steps towards many other powerful optimizations in Python that have not yet been implemented or considered, so this could just be the tip of the iceberg.
And finally, I should point out these kinds of isolated examples are just illustrations. They are like benchmarks in the sense that they do not measure performance of real programs in the wild, they are just isolated examples that are just used to show how dramatic the differences can be. Your own programs may not show these kinds of improvements when you run it in Python 3.11. But what they do is point out how a lot has changed under the hood in just this one version and how many more changes can be on their way too.
References:
Disclaimer: This is a personal [blog, post, statement, opinion]. The views and opinions expressed here are only those of the author and do not represent those of any organization or any individual with whom the author may be associated, professionally or personally.
Top comments (1)
Very informative and helpful