Our world is entirely of dependencies. When you want to bake a cake, you rely on the ingredients and the recipe. When writing a blog post, you depend on your objective and the value you want to give your readers.

In the realm of programming, more often than not, you rely on libraries and frameworks to utilize the functionalities you need. Reusing existing libraries and frameworks is a good practice that saves time and effort.

These libraries and frameworks are often called Dependencies.

Dependencies may not pose a significant challenge if the project is small, personal, or short-term. However, as the development term extends, the complexity of managing these dependencies can increase significantly.

Separating concerns and defining naming conventions are essential for learning and understanding things. So, in this article, we will separate the term Dependencies and discuss it in the context of programming. Next, we will discuss managing external and internal dependencies. Finally, we will discuss managing dependencies in Python and Node.js.

What are Dependencies?

Dependencies are libraries, frameworks, or other software components a project relies on. Commonly, dependencies compose external dependencies, internal dependencies, and transitive dependencies.

External dependencies are libraries or frameworks not part of the code you or your team write. Assume you are writing a function that requires a specific functionality. Instead of writing the code from scratch, you can use an external library with the necessary functionality. This library is an external dependency of your project.

Develop-time dependencies are part of external** dependencies**, which you primarily require during development. For example, you may need a testing library when writing unit tests. When writing documentation, you may need a documentation generator. These dependencies are essential during development but not required in the production environment; they are develop-time dependencies.

About Build-time dependencies:
You may need additional dependencies in some programming languages during the build process. These dependencies are required to compile the code but are unnecessary in the production environment. Sometimes, these dependencies are also called build-time dependencies.

Transitive dependencies are dependencies of dependencies. (It isn't apparent, I know.)
Assume you are using a library as an external dependency. This library, in turn, uses another library to provide the functionality you need. The second library is a transitive dependency of your project.

Internal dependencies are part of the code you or your team writes to reuse any other part of the code within the project. Assume you have a function that you want to use in multiple parts of your project. You can create a separate file for this function and import it wherever needed. This function is an internal dependency of your project.

Managing Dependencies

For External / Transitive dependencies

In the development phases, managing external dependencies is often straightforward. Specify the dependencies in a configuration file if you use a package manager like npm, pip, Maven, or Composer. Afterward, we will discuss managing dependencies with package managers in Python and Node.js.

Managing dependencies becomes more critical and complex in the operations phase. Here, we consider some concerns when managing dependencies in the operations phase.

Dependency update for Security Vulnerabilities: Security vulnerabilities in dependencies can pose a significant risk to your project. When we discover that a dependency has a security vulnerability, updating it to a secure version is essential. However, updating dependencies can sometimes break the project. Automated dependency update tools can help manage this process. (And package managers often have built-in tools for this.)

Dependency Licenses: Dependencies may have different licenses. Some licenses may not be compatible with your project's license. If you update dependencies for security reasons, you may also need to check the licenses of the updated dependencies. For compliance reasons, it is essential to check the licenses of your dependencies and ensure they are compatible with your project. Tools or services that automatically output the licenses of dependencies can help with this process, like a license-checker for Node.js, pip-licenses for Python, and cargo-about for Rust.

For Internal dependencies

Maintaining a clear structure and naming convention is essential for internal dependencies. When you have many internal dependencies, having a clear structure and naming convention is vital. Module systems in programming languages and package managers can help manage internal dependencies, but how you structure your code is up to you.

It is the area of software architecture and design. There are many design patterns and principles to help you manage internal dependencies. For example, the SOLID principles and the Clean Architecture pattern can help you manage internal dependencies. This topic is too vast; we will not delve into it in this article.

Dependency Management in Programming Languages

In modern languages, package managers help manage dependencies. Package managers are tools that help you install, update, and remove dependencies for your project.

Here, we will discuss dependency management in Python and Node.js, two popular programming languages.

In Python

You can manage dependencies in Python with the package manager pip, which comes pre-installed with Python. Pip allows you to install and uninstall Python packages, and it uses a requirements.txt file to keep track of which packages your project depends on. However, pip does not have robust dependency resolution features or isolate dependencies for different projects; this is where tools like pipenv and poetry come in. These tools create a virtual environment for each project, separating the project's dependencies from the system-wide Python environment and other projects.

For example, with poetry, you can add a dependency to your project by running poetry add <package-name>. This command will add the package to your virtual environment and update the pyproject.toml and poetry.lock files, which keep track of your project's dependencies and their exact versions. You can add the --group dev flag to the poetry add command to separate develop-time and production dependencies.

In Python, creating a virtual environment for each project is standard practice. A virtual environment is a self-contained directory tree that contains a Python installation for a particular version of Python, plus some additional packages. Creating a virtual environment helps to isolate your project and its dependencies from other projects, avoiding potential conflicts between dependencies.
It also allows you to work with different versions of Python and various sets of dependencies for different projects.

In Node.js

Node.js manages dependencies using package managers like npm (Node Package Manager), yarn, and pnpm. npm comes pre-installed with Node.js and allows you to install and uninstall Node.js packages. It uses a package.json file to keep track of which packages your project depends on. Yarn and Pnpm are alternative package managers that aim to improve on npm in various ways, such as improved performance and better lock file format.

However, none of these tools isolate dependencies for different projects by default, and this is where tools like nvm (Node Version Manager) and n come in. These tools allow you to install multiple versions of Node.js and switch between them, isolating the Node.js version and the associated dependencies for each project. This functionality is beneficial when working on various projects requiring different Node.js versions or different sets of dependencies.

With npm, you can add a dependency to your project by running npm install <package-name>. This command will add the package to your node_modules directory and update the package.json and package-lock.json files, which keep track of your project's dependencies and their exact versions. You can add the --save-dev flag to the npm install command to separate develop-time and production dependencies.

Summary

In this article, we delved into the concept of Dependencies in software engineering, a fundamental aspect that plays a crucial role in developing and maintaining software projects.

We started by defining dependencies and differentiating between external, internal, and transitive dependencies. We then discussed the importance of managing these dependencies, including tracking security vulnerabilities and licenses for external and transitive dependencies. We emphasized the need for automation tools to make dependency management more efficient.
Additionally, we highlighted the significance of maintaining a clear structure and naming convention for internal dependencies.
We provided examples of dependency management in Python and Node.js, discussing the tools and practices commonly used in these languages.

Managing dependencies is not just about installing and uninstalling packages.
It's about:

• managing versions to ensure compatibility
• resolving conflicts that can break your application
• isolating environments to prevent interference between projects
• staying aware of license and security considerations

To maintain the integrity and legality of your project.

Effective dependency management ultimately contributes to your software projects' robustness, maintainability, and quality. It's a skill that every developer should strive to master.

What's Next?

Here are some topics you can explore to deepen your understanding of dependency management:

More deep dive into Package Managers: Package managers are essential for managing dependencies. Each tool has unique features and best practices, including performance, dependency resolution, and lock file management. Understanding these aspects can help you make informed decisions when choosing a package manager for your project.

Dependency Management in Other Languages: We've discussed Python and Node.js in this article, but dependency management is a universal concept in programming. Exploring how you handle dependencies in other languages like Java, C#, or Rust could be beneficial. (I think Rust's cargo is an excellent example of a package manager.)

Dependency Injection: This design pattern helps manage dependencies between classes and components in your application. It promotes loose coupling and makes your code more testable and maintainable. It is also related to the Internal dependencies we discussed earlier.

Continuous Integration and Continuous Deployment (CI/CD): CI/CD pipelines automate the build, test, and deployment processes. Managing dependencies in a CI/CD pipeline can help you streamline your development workflow.

Semantic Versioning: Semantic Versioning is a versioning scheme for software that aims to convey meaning about the underlying changes with each new release. Understanding this can help manage version updates of dependencies more effectively.

Containerization and Dependency Management: Tools like Docker can help isolate your application and its dependencies, making it easier to manage and deploy. Understanding how containerization affects dependency management could be a valuable topic.

Microservices and Dependency Management: Each service is a separate application with dependencies in a microservices architecture. Managing dependencies in such a system presents unique challenges and strategies.

Monorepos and Dependency Management: A monorepo is a repository containing multiple logical projects. These projects can be unrelated, loosely coupled, or as tightly coupled as applications in a microservices architecture, each with its dependencies. Understanding managing dependencies in a monorepo could be an exciting topic. (Recently, I have been using moon, a monorepo management tool.)