Recent I've spent quite some time working on a CLI tool designed to evaluate code complexity. This project made me think about factors which impact the overall cognitive load required to understand and maintain a codebase. I've realized that complexity largely comes from two key areas: how we organize our code and the amount of knowledge required to understand it. This knowledge could involve domain expertise needed to grasp the science or business rules behind the code, or it could be the technical know-how related to the tools and techniques used in implementation.

A lot is said on structuring code effectively, while I believe the "knowledge factor" is often overlooked. This is critical because, although messy code can be untangled with time and effort, no amount of refactoring will help if the underlying code is built on complex scientific concepts or advanced coding techniques that you do not understand. Mastering such knowledge can take years, making it far more challenging to deal with.

Considering the knowledge requirements associated with a codebase, I divide them into two categories: natural complexity and artificial complexity introduced by engineering decisions. Natural complexity is unavoidable, such as domain-specific knowledge necessary to solve the problem. However, the more dangerous form of complexity arises from optional engineering choices—like unnecessary third-party libraries or hard-to-master coding techniques. These choices can impose a steep learning curve on the team, resulting in long-term costs and complexity that can drain businesses significantly.

Code Complexity Grades

Let me explain my thoughts abstractly. Imagine we have a gradation of code based on complexity levels:

1) Code that is instantly clear to anybody.
2) Code that requires closer inspection but is still accessible to most developers.
3) Code with complex logic or requiring specialized domain knowledge that is commonly available within the organization.
4) Code with highly intricate logic or demanding advanced, specialized domain expertise.
5) Code so complex that it is comprehensible only to a select few.

A healthy software project should have the majority of the code at level 2, with a small portion at level 3, and only a minimal amount at level 4. This balance ensures that most of the codebase remains accessible and maintainable, while only a small part requires specialized knowledge or extra attention.

Advanced Coding Techniques

In my view, one of the selling points for many advanced coding techniques like functional programming (FP) is that by applying the techniques developers intend to "simplify" the code.

For example functional programming offers powerful abstractions that can encapsulate complex logic in a more declarative and concise manner. This can lead to cleaner, more maintainable code that reduces the cognitive load required to understand intricate systems.

However, there is a caveat: according to the complexity scale, any FP code inherently falls into level 3-4. It requires specialized knowledge and skills, which are not as widespread as one might hope. Therefore, when building a team for a project that heavily relies on functional programming, it's not enough to simply find smart and capable engineers — they also need to be proficient in the advanced coding techniques that the project demands.

Using advanced approaches, the project gets higher amount of level 3-4 code. Such a code requires more cognitive load for those, who is not too experienced with a selected paradigm, and not every programmer can understand it.

Third-Party Libraries

Third-party libraries are often introduced with the goal of speeding up development and reducing the need to reinvent the wheel. By leveraging pre-built solutions, developers can focus on higher-level tasks, potentially simplifying certain aspects of the project.

However, these libraries come with their own complexity. Each one introduces new abstractions, APIs, and sometimes even entire ecosystems that developers need to understand and manage. According to the complexity scale, integrating third-party libraries inherently pushes parts of the codebase up by expanding amount of knowledge necessary to understand system. It requires understanding how these libraries work, their limitations, and how they interact with other components in the application.

While third-party libraries are generally easier to pick up compared to advanced coding techniques. However, even with good documentation and community support, developers must invest time to learn the library’s nuances and ensure it fits the project's needs. Over-reliance on these libraries can still increase the cognitive load, especially for team members unfamiliar with them, adding complexity that comes not from the problem domain itself but from managing external tools.

Usage of a library can be justified when it helps to reduce complexity, especially by covering areas that would otherwise require specialized domain knowledge. For example, Three.js is a popular JavaScript library that simplifies working with 3D graphics. Creating 3D scenes, animations, and effects from scratch would typically require deep knowledge of computer graphics, rendering pipelines, and low-level WebGL programming. However, Three.js abstracts much of this complexity, offering a more accessible API that lets developers create sophisticated 3D visualizations without needing to master computer graphics. This allows teams to focus on business logic and the creative aspects of the project rather than the technical challenges of rendering graphics.

Embracing Simplicity

Over the years, I have come to appreciate the value of simplicity in software design. This doesn't mean avoiding advanced techniques altogether, but rather using them judiciously and ensuring that the overall system remains understandable and maintainable.

I think the knowledge requirements we casually introduce by using unnecessary third-party libraries or advanced coding techniques is the worst, since nothing requires it to be there, it is not essential to solve the business problem. The balance lies in knowing when to apply these techniques and ensuring that the overall design remains accessible to the entire development team.