Project-wide Refactor: JavaScript to TypeScript Migration

Introduction

In today's fast-paced development landscape, keeping your codebase up-to-date and adaptable is essential. But this is not just any refactor. We're talking about the migration from JavaScript to TypeScript, a leap that promises enhanced type safety, improved maintainability, and a more robust development experience.

Let's dive into the nitty-gritty details of how we tackled this endeavor head-on.

In this blog post, we will unveil our dynamic approach to this transformation. We will also shed light on how we seamlessly embedded ourselves into the client's team, taking ownership of the product and fostering proactive collaboration.

These numbers tell a story of resilience, growth, and proven reliability that we must respect:

• A decade-old codebase.
• Over 5,000 files of intricate code.
• Meticulously nurtured by three dedicated teams.
• Steadfastly battle-tested in production for over seven years.

Understanding the Landscape

Before embarking on any significant project, a comprehensive understanding of the existing landscape is paramount. Our journey began with an in-depth analysis of the project. We scrutinized the JavaScript codebase, identifying its core components, dependencies, and areas that could benefit from TypeScript's enhanced type safety.

This phase wasn't just about lines of code; it was about deciphering the underlying architecture and intricacies that defined the project's DNA. By unraveling these complexities, we laid the groundwork for a strategic migration plan that would seamlessly integrate TypeScript's capabilities while preserving the project's essence.

To delve deeper into the project's dependency structure, we leveraged the power of the deprank node library. This tool allowed us to visualize the dependencies graph, revealing the intricate relationships between various modules and components. Armed with this visual representation, we gained insights into potential points of impact for the migration. It enabled us to make informed decisions about which parts of the codebase would benefit the most from TypeScript's strong typing.

Example:

To help convert your codebase to TypeScript whilst minimizing the amount of effort required, we suggest converting files in deprank --deps-first order.

The following command will find all .js or .jsx files in a src folder, and sort them in dependency-first order.

Charting the Migration Roadmap

After gaining a solid understanding of the intricacies of the project, we focused on the critical task of charting the migration roadmap. Our approach was not limited to a mere technical checklist; rather, it was a holistic strategy that considered the project's current state, future aspirations, and evolving user needs. Collaboration played a pivotal role in this phase. We engaged in extensive discussions with the client's team to align our vision with theirs. Together, we set achievable goals, outlined milestones, and identified potential challenges. This collaborative roadmap became our guiding light, ensuring that every step of the migration was purposeful and aligned with the project's overarching objectives.

Codebase insights:

• The Project was structured as Microservices.
• There were ~500k lines of JavaScript code at the time we started.
• Services were divided into:
  • Workers: worker.sms-sender, worker.email-sender,…
  • Microservices: microservice.sms, microservices.email,…
  • API gateways: api.server, api.form-server,…
  • Multiple private libraries with thousands of lines of code.

We perform migration gradually, starting with the least dependent services and moving toward the most dependent ones.

Here's a simplified version of our migration roadmap:

• Identify Low-Hanging Fruits: Begin with simpler, standalone components or modules that could be migrated to TypeScript without extensive changes.
• Model Definition Creation: Develop TypeScript type definitions for existing JavaScript models (Class-based JSON schema), enhancing type safety and documentation.
• Incremental Migration: Gradually migrate more complex modules, focusing on core functionality and high-priority areas.
• Refactor and Optimize: Leverage TypeScript's features to refactor and optimize code for improved performance and maintainability.
• Comprehensive Testing: Rigorously test the migrated code, using automated tests and manual verification to ensure functionality remains intact.

Seamless Integration

Seamless integration into the client's team was essential for the migration's success. As embedded team members, we collaborated on code reviews, design discussions, and implementation decisions.

Leveraging the airbnb/ts-migrate tool as our core migration engine

To ensure the migration process was as seamless as possible, we leveraged the power of the airbnb/ts-migrate tool as our core migration engine. This tool enabled us to:

• Automatically convert JavaScript code to TypeScript
• Provide detailed and actionable feedback about type safety issues
• Generate TypeScript type definitions for existing JavaScript code with JSdoc.

The ts-migrate tool made it possible to migrate the codebase incrementally, one module at a time. This approach allowed us to verify that each migration was successful before moving on to the next. It also enabled us to focus on high-priority modules, ensuring that we were making the most significant impact with the least amount of disruption.

Adding more commands to solve additional requirements

• CommonJS to ES6
• Generator Functions to Async/Await
• Default export to named export

Here is an example of how the ts-migrate tool works with self-implemented commands:

Iterative Progress

Despite the benefits of the tool ts-migrate. Throughout the migration, regular iterations and feedback loops were key. Continuous communication allowed us to address challenges promptly and refine our approach as needed. The result was a transformed codebase that seamlessly combined JavaScript's functionality with TypeScript's power.

Going from basic to advanced functions

At the beginning of the migration, we attempted to implement our commands to work out of the box as simply as possible.

From basic - A straightforward command for converting all CommonJS export syntax to ES6 is to use RegEx to match and replace:

To advance - A command for converting all Generator Functions to Async/Await is to use typescript methods to travel and replace nodes:

Comprehensive Testing

We do not only test the migrated code using automated tests and manual verification to ensure its functionality remains intact but also write unit tests to ensure that self-implemented commands work correctly.

Based on our experience, the migration process also involves Code Formatting plugins. Tests should not concern themselves with formatting, but rather just ensure the expected syntax replacement.

Improve CI

"Success is not final, failure is not fatal: it is the courage to continue that counts." - Winston S. Churchill

Any successful result, such as the migration from JavaScript to TypeScript, requires time, effort, and a willingness to persist through challenges and setbacks:

• Time spent: ~250h
• Duration: 8 months
• Side effect: CI runs 3 three-fold slower

Despite the fact that our CI runs three times slower, we managed to resolve the problem by leveraging the latest library, SWC.

SWC is an extensible Rust-based platform for the next generation of fast developer tools. It's used by tools like Next.js, Parcel, and Deno, as well as companies like Vercel, ByteDance, Tencent, Shopify, and more.

Result

In the end, our technical prowess, proactive collaboration, and strategic adoption of SWC produced remarkable results. The JavaScript to TypeScript migration led to a codebase that seamlessly blended enhanced type safety with improved maintainability.

Moreover, by addressing the challenge of slowed CI times through SWC, we ensured that our development process remained efficient and productive.

Conclusion

"TypeScript is JavaScript that scales." - Anders Hejlsberg

Our journey through the JavaScript to TypeScript migration for the project was marked by technical innovation, collaborative spirit, and unwavering dedication. As we continue to uncover the layers of our approach. Are you ready to embark on your own migration journey, armed with technical expertise and innovative solutions? Join us as we delve deeper into the realm of software transformation, and learn how to overcome challenges while achieving technical excellence and efficiency!