DEV Community

Cover image for Different Ways to Fix Circular Imports in Python
Sachin
Sachin

Posted on • Originally published at geekpython.in

Different Ways to Fix Circular Imports in Python

Have you ever come across circular imports in Python? Well, it’s a very common code smell that indicates something’s wrong with the design or structure.

Circular Import Example

How does circular import occur? This import error usually occurs when two or more modules depending on each other try to import before fully initializing.

Let’s say we have two modules: module_1.py and module_2.py.

# module_1.py
from module_2 import ModY
class ModX:
    mody_obj = ModY()
Enter fullscreen mode Exit fullscreen mode
# module_2.py
from module_1 import ModX
class ModY:
    modx_obj = ModX()
Enter fullscreen mode Exit fullscreen mode

In the above code snippets, both module_1 and module_2 are mutually dependent on each other.

The initialization of mody_obj in module_1 depends on module_2 and the initialization of modx_obj in module_2 depends on module_1.

This is what we call a circular dependency. Both modules will stuck in the import loops while attempting to load each other.

If we run module_1.py, we’ll get the following traceback.

Traceback (most recent call last):
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
  File "module_2.py", line 1, in <module>
    from module_1 import ModX
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
ImportError: cannot import name 'ModY' from partially initialized module 'module_2' (most likely due to a circular import)
Enter fullscreen mode Exit fullscreen mode

This error explains the situation of circular import. When the program attempted to import ModY from module_2, at that time module_2 wasn’t fully initialized (due to another import statement that attempts to import ModX from module_1).

How to fix circular imports in Python? There are different ways to get rid of circular imports in Python.

Fix Circular Imports in Python

Move code into a common file

We can move the code into a common file to avoid import errors and then try to import the modules from that file.

# main.py ----> common file
class ModX:
    pass

class ModY:
    pass
Enter fullscreen mode Exit fullscreen mode

In the above code snippet, we moved the classes ModX and ModY into a common file (main.py).

# module_1.py
from main import ModY

class Mod_X:
    mody_obj = ModY()
Enter fullscreen mode Exit fullscreen mode
# module_2.py
from main import ModX

class Mod_Y:
    modx_obj = ModX()
Enter fullscreen mode Exit fullscreen mode

Now, module_1 and module_2 import the classes from main which fixes the circular import situation.

There is a problem with this approach, sometimes the codebase is so large that it becomes risky to move the code into another file.

Move the import to the end of the module

We can shift the import statement at the end of the module. This will give time to fully initialize the module before importing another module.

# module_1.py
class ModX:
   pass

from module_2 import ModY

class Mod_X:
   mody_obj = ModY()
Enter fullscreen mode Exit fullscreen mode
# module_2.py
class ModY:
   pass

from module_1 import ModX
Enter fullscreen mode Exit fullscreen mode

Importing module within the class/function scope

Importing modules within the class or function scope can avoid circular imports. This allows the module to be imported only when the class or function is invoked. It’s relevant when we want to minimize memory use.

# module_1.py
class ModX:
  pass

class Mod_X:
   from module_2 import ModY
   mody_obj = ModY()
Enter fullscreen mode Exit fullscreen mode
# module_2.py
class ModY:
   pass

class Mod_Y:
   from module_1 import ModX
   modx_obj = ModX()
Enter fullscreen mode Exit fullscreen mode

We moved the import statements within classes Mod_X and Mod_Y scope in module_1 and module_2 respectively.

If we run either module_1 or module_2, we’ll not get a circular import error. But, this approach makes the class accessible only within the class’s scope, so we can’t leverage the import globally.

Using module name/alias

Using the module name or just an alias like this solves the problem. This allows both modules to load fully by deferring circular dependency until runtime.

# module_1.py
import module_2 as m2

class ModX:
    def __init__(self):
        self.mody_obj = m2.ModY()
Enter fullscreen mode Exit fullscreen mode
# module_2.py
import module_1 as m1

class ModY:
    def __init__(self):
        self.modx_obj = m1.ModX()
Enter fullscreen mode Exit fullscreen mode

Using importlib library

We can also use the importlib library to import the modules dynamically.

# module_1.py
import importlib

class ModX:
    def __init__(self):
        m2 = importlib.import_module('module_2')
        self.mody_obj = m2.ModY()
Enter fullscreen mode Exit fullscreen mode
# module_2.py
import importlib

class ModY:
    def __init__(self):
        m1 = importlib.import_module('module_1')
        self.mody_obj = m1.ModX()
Enter fullscreen mode Exit fullscreen mode

Circular Imports in Python Packages

Usually, circular imports come from modules within the same package. In complex projects, the directory structure is also complex, with packages within packages.

These packages and sub-packages contain __init__.py files to provide easier access to modules. That’s where sometimes arises circular dependencies among modules unintentionally.

We have the following directory structure.

root_dir/
|- mainpkg/
|---- modpkg_x/
|-------- __init__.py
|-------- module_1.py
|-------- module_1_1.py
|---- modpkg_y/
|-------- __init__.py
|-------- module_2.py
|---- __init__.py
|- main.py
Enter fullscreen mode Exit fullscreen mode

We have a package mainpkg and a main.py file. We have two sub-packages modpkg_x and modpkg_y within mainpkg.

Here’s what each Python file within modpkg_x and modpkg_y looks like.

mainpkg/modpkg_x/__init__.py

from .module_1 import ModX
from .module_1_1 import ModA
Enter fullscreen mode Exit fullscreen mode

This file imports both classes (ModX and ModA) from module_1 and module_1_1.

mainpkg/modpkg_x/module_1.py

from ..modpkg_y.module_2 import ModY
class ModX:
    mody_obj = ModY()
Enter fullscreen mode Exit fullscreen mode

The module_1 imports a class ModY from module_2.

mainpkg/modpkg_x/module_1_1.py

class ModA:
    pass
Enter fullscreen mode Exit fullscreen mode

The module_1_1 imports nothing. It is not dependent on any module.

mainpkg/modpkg_y/__init__.py

from .module_2 import ModY
Enter fullscreen mode Exit fullscreen mode

This file imports the class ModY from module_2.

mainpkg/modpkg_y/module_2.py

from ..modpkg_x.module_1_1 import ModA
class ModY:
    moda_obj = ModA()
Enter fullscreen mode Exit fullscreen mode

The module_2 imports a class ModA from the module_1_1.

We have the following code within the main.py file.

root_dir/main.py

from mainpkg.modpkg_y.module_2 import ModY

def mody():
    y_obj = ModY()

mody()
Enter fullscreen mode Exit fullscreen mode

The main file imports a class ModY from module_2. This file is dependent on module_2.

If we visualize the import cycle here, it would look like the following ignoring the __init__.py files within the modpkg_x and modpkg_y.

Import cycle

We can see that the main file depends on module_2, module_1 also depends on module_2 and module_2 depends on module_1_1. There is no import cycle.

But you know, modules depend on their __init__.py file, so the __init__.py file initializes first, and modules are re-imported.

Import cycle changed due to __init__.py file

This is what the import cycle looks like now.

Final import cycle

This made module_1_1 depend on module_1, which is a fake dependency.

If this is the case, empty the sub-packages __init__.py files and using a separate __init__.py file can help by centralizing imports at the package level.

root_dir/
|- mainpkg/
|---- modpkg_x/
|-------- __init__.py  # empty file
|-------- module_1.py
|-------- module_1_1.py
|---- modpkg_y/
|-------- __init__.py  # empty file
|-------- module_2.py
|---- subpkg/
|-------- __init__.py
|---- __init__.py
|- main.py
Enter fullscreen mode Exit fullscreen mode

In this structure, we added another sub-package subpkg within mainpkg.

mainpkg/subpkg/__init__.py

from ..modpkg_x.module_1 import ModX
from ..modpkg_x.module_1_1 import ModA
from ..modpkg_y.module_2 import ModY
Enter fullscreen mode Exit fullscreen mode

This will allow internal modules to import from a single source, reducing the need for cross-imports.

Now we can update the import statement within the main.py file.

root_dir/main.py

from mainpkg.subpkg import ModY
def mody():
    y_obj = ModY()

mody()
Enter fullscreen mode Exit fullscreen mode

This solves the problem of circular dependency between the modules within the same package.

Conclusion

Circular dependency or import in Python is a code smell which is an indication of serious re-structuring and refactoring of the code.

You can try any of these above-mentioned ways to avoid circular dependency in Python.


🏆Other articles you might be interested in if you liked this one

Template Inheritance in Flask with Example.

Difference between exec() and eval() with Examples.

Understanding the Use of global Keyword in Python.

Python Type Hints: Functions, Return Values, Variable.

Why Slash and Asterisk Used in Function Definition.

How does the learning rate affect the ML and DL models?


That’s all for now.

Keep Coding✌✌.

Top comments (0)