Context managers in Python

Python, with its simplicity and versatility, has various constructs and features that facilitate the readability and maintainability of code. These include context managers, which undoubtedly form the most influential of all the tools to manage resources and handle exceptions exceptionally well. One of the nicest features of contexts is the fact that they are really easy to use. For example, we can freely switch to any work directory by using context manager or easily access the database. In this article, we will describe in detail context managers so you can go back to your projects with practical examples that will help you to use them to their full potential. Therefore, the theme of this article is learning context managers.

What is a Context Manager?

A context manager is actually a resource manager in Python. For example, when we read or write some content from a file, we don’t care about closing the file. If we are using one file, then this is not a big issue. However, when it comes to multiple files, this will cause problems as we are not releasing the resources we are no longer using. This is also true for database access. After a database is used, we should close the database. Otherwise, it can slow down our code. So, the context manager helps us maintain the resource adequately. Below is an example of a context manager used to access the file.

Here, we are trying to open the file in read mode using the Python context manager and reading the first line from the file. The file will automatically close when we leave the context.

The contextlib Module

Python's contextlib module provides utilities for working with context managers. It offers decorators, context manager factories, and other tools to simplify the creation and usage of context managers.

‘contextlib.contextmanager’

One of the key features of the contextlib module is the ‘contextmanager’ decorator. This decorator allows you to define a generator-based context manager with minimal boilerplate code.

In this example, the my_context_manager() function serves as a context manager. The yield statement separates the code for entering and exiting the context. Any code before the yield statement is executed before entering the context, while any code after the yield statement is executed after exiting the context.

‘contextlib.ExitStack’

Another useful tool provided by the ‘contextlib’ module is the ExitStack class. It allows you to manage multiple context managers simultaneously, even when the number of managers is unknown at compile time.

In this example, the ‘ExitStack’ class manages a list of file objects opened using the ‘open()’ function. All files are automatically closed when the block of code completes execution.

Why do we need to use context manager?

• Context managers in Python serve several important purposes, making them a valuable tool in programming. Here are some key reasons why context managers are essential:
• Resource Management: Context managers play a crucial role in the proper management of files, databases, network sockets, and locks. They ensure that these resources are acquired and released correctly, even in complex scenarios and error conditions. This not only reduces resource losses but also enhances the overall trust and quality of the code, leading to improved performance.
• Automatic Cleanup: Context managers help to form a code that takes care of the initialization and deallocation of resources. In summary, this mechanism makes the code compact. It eliminates the need for explicit resource-releasing procedures, thus reducing the risk of omitting this step, which could result in memory leaks or other problems.
• Exception Handling: Context managers are designed to help clean up a specific context whenever an exception occurs. They are guaranteeing that resources are being properly cleaned up in case of any exception, even if it happens. This aspect is especially beneficial in situations where several resources are managed simultaneously by the context managers, as coordination is done correctly.
• Readability and Maintainability: Wrapping up the resource handle within a context manager not only makes the code easier to read and maintain but also enhances your understanding of the code. The context manager clearly delineates the resource usage in the initial and final states, thereby improving the code structure and making it easier to comprehend and debug. This is a significant benefit that context managers bring to your code.
• Reduced Boilerplate: Context managers are a boon for developers as they eliminate the need to write repetitive, standard code, known as 'boilerplate code '. By encapsulating widely recurring resource management patterns into their reusable constructs, context managers enable developers to focus on the main application logic of their programs. This not only results in a more elegant and concise code
• structure but also reduces the chances of errors due to repetitive code, thereby improving code development efficiency.
• Concurrency and Thread Safety: Context managers can effectively help implement thread safety and concurrency by managing locks and other synchronization primitives in a particular block. This allows for the easy design of concurrent algorithms and greatly minimizes the chances of race conditions and other synchronization problems.

Creating Context Managers

In the previous example, we used the open() function as a Python context manager. There are two ways to define a context manager – one is class-based, and the other is function-based. Let’s see how we can create it using class.

Python Code:

For the class-based, we have to add two methods: enter () and exit(). In the enter() method, we define what we want to do in the context. Here, we open a file. In the exit() method, we have to write what will happen after leaving the context. In this case, we will close the file after leaving the context. Now, let’s see what a function-based Python context manager looks like.

For a function-based context manager, you define a function with the @contextlib.contextmanager decorator. Within that function, you place the code you want to execute within the context. Following that, you use the yield keyword to indicate that this function serves as a Python context manager. After yield, you can include your teardown code, specifying actions to take when exiting the context. This teardown code ensures proper resource cleanup and is executed when leaving the context.

Examples

We discussed the context manager's theoretical part. Now, it is time to see some examples. Let’s see how we can access the database.

You already understand what is happening in the function. This context manager will give access to a database in its context. When we leave the context, the connection to the database is ended. You don’t have to return value whenever creating a context manager.
Now, here is another example. While using Google Colab, if the data size is too large, we attach our Google Drive to the notebook and create a folder for the data. Then, we download the data in that folder and want to return it to our working directory. This is easy with a context manager.

See the above example. We don’t return any value from the “yield” keyword. I hope you now understand how to use Context Manager.

What does '@contextmanager' do in Python?

In Python, the '@contextmanager' decorator is a powerful tool provided by the contextlib module. It allows you to create custom context managers using generator functions with minimal boilerplate code.
When you decorate a generator function with '@contextmanager,' it transforms the function into a context manager factory. This means that when the decorated function is called, it returns a context manager object that can be used with the with statement.

Here's how '@contextmanager' works:

• Decorator Syntax: You apply the '@contextmanager' decorator directly above the definition of a generator function.
• Generator Function: The function decorated with '@contextmanager' must be a generator function. This means it contains at least one yield statement.
• Yield Statement: The generator function supplies the yield statement to indicate which part of the code will run before yield and which part will run after yield. With the yield statement, whatever is being yielded will be the resource that the context manager is managing.
• Setup and Teardown Logic: The code that is to be executed before the yield declaration runs during setup and the code that is to be executed after the yield declaration runs during teardown. This gives you a place to either do the initialization or clean up any operations.
• Returning a Context Manager: In the generator function, by calling the context manager object, an object will be returned that can be used with the 'with' statement. The with statement is designed to call the enter method of the context manager before entering the context and the exit method after exiting the context to correct resource management.

Here's a simple example to illustrate the usage of '@contextmanager':

In this example, my_context_manager() is a generator function decorated with '@contextmanager.' When called, it returns a context manager object. Inside the with the statement, the setup code executes before the yield statement, and the teardown code executes after it. The value "resource" yielded by the yield statement becomes the resource managed by the context manager. Finally, the with statement prints the message indicating that it's inside the context, along with the resource value.

Managing Multiple Files

Sometimes, we need to open multiple files. So what should we do? Of course, we have to use multiple context managers. Here is an example of copying the content of a file.

This will work, but if we have a large file to copy, it will cause pain. That will make the code slower. You can also face device freezing, as that process takes a lot of resources. So what should we do? If we can copy the file line by line, that doesn’t cause any problem. To do this, we have to open two files simultaneously. See the example below.

This is called the nested context. This method is fine when you have to open two or three files. But when the number of files is much higher, the code becomes messier, and you have difficulties editing it. In Python 3.10, they introduced a new method, “parenthesized context,” in which you can write multiple contexts in one line using brackets. There is no need to use looping.

The above code is much cleaner. This is the best option for working on multiple files. Now it may happen that you will prefer others. That’s totally up to you.

Conclusion

Using context manager in your code ensures the proper utilization of the resource. Resource utilization is not a big deal when you are working on a small project, but when it comes to large projects, like reading a very large file that we discussed previously, context manager comes in handy. In this article, we learned-

• The basics – what and how.
• The types of context managers, according to their implementation – are class-based and function-based.
• How to implement the different types of context managers.
• How to handle multiple files using nested context manager and parenthesized context manager.