Ville M. Vainio

Posted on Jul 17, 2019

Trivial, typesafe translations in Angular

#angular

The official Angular translation system (angular-i18n) is problematic in various respects (read this ticket and thread in ngx-translate project for background):

Discussion: future of the library #783

ocombe posted on Jan 30, 2018

Hello everyone, I'd like to discuss about the future of this library and get your opinion on my plans.

(Long) history

You can skip that part if you don't care about how this library came to life

When I started this library 3 years ago, I wanted to learn Angular and I thought that working on an open source project would be the best way to do that. It had worked for me on AngularJS with my library ocLazyLoad that was a huge success (2600 stars) and whose core principles were finally integrated into the framework in v1.6.7.

At the time I was looking for a way to translate my Angular apps and found out that there was nothing in Angular to do that (i18n wasn't even existing in the framework). I asked my good friend Pascal Precht if he was interested in porting angular-translate from AngularJS to Angular but he wasn't and told me that I should do it.

My library quickly became popular for various reasons (no real alternative, I had a good reputation from my work on ocLazyLoad, my appearances in Angular Air and my talks at conferences). And I guess that the code was good enough and easy to start with. It was also a time where the framework was still in beta, and there were no real guidelines for publishing libraries for Angular, it was so hard to do it that not a lot of people were trying and I didn't have any real competition on i18n for a long time.

Some time later, the library ecosystem started to settle and Angular was stable, they decided to drop the name Angular 2/4/... in favor of just "Angular". I decided to rename this library ngx-translate and to use npm scopes in order to deliver a modular experience. I rewrote the whole library and made it possible to replace some parts (loader, parser, compiler, ...). It was still a side project for me, but I was using it at work because we had finally started using Angular in production and needed to translate the app in multiple locales.

The official i18n implementation was super complicated (and still is), it didn't support json, code translation, changing the locale without reloading the app, and you needed multiple app bundles (one per locale). My library was so popular that I wanted to work on it full time because I could see that it was a necessity for the Angular community. It solved a lot of use cases, and was simple enough to be usable in a few minutes.

I decided to quit my job and see if I could make a living out my open source work. I knew that it was possible because ag-grid had done it and they were very successful. It was in December 2016, I had 2 months left at work before I was free, and I went to the conference NG-BE where Igor Minar from the core team was present. I wanted to ask him some advices on how to turn this into a profitable OSS project, but before I could tell him about my plans, he told me: "Ah Olivier, I wanted to talk to you, you've got to stop working on your library [ngx-translate], Companies come to us and want to do i18n, but they only talk about your library, we want them to use our solution [Angular i18n] because it's more efficient".

I was a bit speechless, I was there about to tell him that I had quit my job and wanted to work full time on my lib, and he was asking me to stop working on it. So I did what anyone with a bit of common sense would have done, I told him that Angular i18n sucked, that it failed to deliver the promise of easy-to-use-yet-powerful that they wanted Angular to be. I probably didn't use those words, but he agreed with me that it still needed some work, but they didn't have enough resources to work on it (Victor Savkin and Jeff Cross had just left the team to create Nrwl). I jumped on the occasion and told him that I would love to work with them to improve Angular i18n. He told me that he would consider it, and we left it at that.

A few weeks later, and recontacted me, and told me that they were ok to hire me as a contractor to work on i18n. It was like Christmas for me (actually it was around Christmas, so that might explain it) because working for Google is a once in a lifetime opportunity and a dream job. But working on open source, from home, for Angular, it was even better and I didn't even thought that it was possible for me to do that.

I joined the team in February 2017 (almost a year ago), and started working on i18n. There was a lot of work to do, but I was confident that I could make a difference. It was clear to me at the time that it would take a few months to make i18n easier to use for everyone, with the features that people wanted, and that I could deprecate my library after that.

It turns out that working on a framework like Angular is super complicated. The codebase is huge, not highly documented, with a lot of different packages. The development process is complicated (it doesn't even work on Windows yet), and whenever you make a change you have to be extra careful not to break anyone. We even write tests, like a lot of tests! It was a big shift for me, because I had always worked on projects that needed to ship fast, and tests were clearly not the priority, you fixed bugs instead. But working on a library like Angular is totally different. Millions of people depend on your code, when you make a mistake you can break applications that loose thousands or hundred of thousands of dollars because of you. The responsibility is overwhelming and that's why it is much more complicated to release new features. I also wasn't the one deciding the priorities for the framework. Any change that you want to make has to work with Google applications, and it's not like you can duplicate functionalities to make it work for everyone (who said forms and http?).

Anyway it was complicated to change the way i18n worked in Angular. I learned a lot and I feel like we're going in the right direction now. I understand the choices that they made for Angular i18n, it's the most efficient and stable way to translate applications. It works very well for huge companies like Google, but it just isn't what most developers need/want.

During this year, I haven't really worked on ngx-translate. The issues are pilling up, I've only merged a few PRs and some of them have been waiting there for a long time without any activity because of me. There are some long term issues that I haven't taken the time to fix (code splitting / lazy loading, testing, ICU expressions, bugs with ionic...).

But before we get to that, let's quickly compare my library with the native implementation.

Angular

In Angular, you translate your templates using an extraction tool that generates xmb or xliff files, which are well defined formats that have been battle-tested. Those files are translated using professional tools, by real translators. At the end you merge the translations at build time and get a translated bundle.

Advantages:

The translation process is well defined, it is easy to detect missing translations and you can integrate that in your plans before deploying your app (extract, send to translators, merge)
It scales well, you never look directly at the files since you use a software for that, you can translate in x languages, you just need to extract once and create the translation files for each locale (which your professional tool should do for you)
it's super efficient because the app is pre-translated when it bootstraps, there's no extra work to apply your translations and it works well with pre-rendering because of that
since you merge the translations at build time, it means that it can support directives, components and other html elements, you can move them around in the different languages, or even remove them if needed
you don't need to worry about lazy loading, your bundles are translated at build time, and that includes your lazy loaded modules

Issues:

it takes a lot of work to setup, you need to extract, send to translators, and then you can merge
you create one app bundle / locale, which means that you cannot change the locale at runtime, you need to reload a different bundle for that. And to do that you need to handle server side routing to deliver the correct locale bundle. Google doesn't feel like it's an issue because you're not supposed to change your locale frequently. And in fact most of your users will change it maybe only once, or not at all. But it still complicates the workflow a lot.
no code translations, only template are translated since that's all you can do at build time for now
harder to use in development because you need to stop your dev server, extract translations, merge them. And the setup is different between JIT and AOT...
you cannot translate libraries, and that's a real problem for components libraries (like material, ng bootstrap, ...)

ngx-translate

With ngx-translate you load the module and configure it, you then load the json files containing your translations. You need to manually define keys for your translations and to write them in your json files that you will probably translate yourself, or transform into other formats that you'll send to your translators. But it's ok because json is easy to manipulate and generate.

Advantages:

you can choose how you want to handle your translations (bundled, http-loaded, pre-cached using service workers or localstorage, etc...)
you can change the locale at any time
it works in your templates AND in your code
json (that's all I need to say)
the community can work on plugins, custom loaders, tooling, etc... everything is hackable and you should be able to make it work the way you prefer

Issues:

it uses bindings (via a pipe or a directive) which means that it works at runtime and you can experience some kind of FOC effect (flash of content) when the translations are loaded and applied. Also it takes some memory and some processor time when change detection is triggered
if you want to split translations, it doesn't work well, the setup is complicated and there are bugs in the library
json is not an official translation format, and most professional translation tool don't handle that, but you can use different loaders to support other formats (like po) so that's not a super huge issue
it's hard to test
if it breaks, then your app will have no text!
it doesn't support angular elements (components, directives, pipes) because there's no "compile" at runtime for AOT applications

Changes in Angular

Like I said, I've been working on Angular i18n for almost a year now. I've actually worked on a lot of other things as well, and the changes in i18n were clearly not the priority for the framework. A lot of internal changes were also necessary before we could do the changes that we wanted.

I've greatly improved the documentation (i18n guide), I've created demo projects with and without the cli to help people get started. I've worked on bugs, and made a few changes, but we've not released anything major for translations.

Finally for v6 the big changes are coming, translating will be done at runtime. Having access to the internals of the framework means that it will be way more efficient than what ngx-translate does. You shouldn't experience the FOC effect, and it won't take any resource during change detection. You'll be able to do code translations, at last! There will only be one bundle per application for all of the locales (or multiple, if you prefer). And libraries should be translatable.

What will not change:

you'll still have to extract translations using the extraction tool and merge them at build time, which means that's it's not very easy to use in development
there's no json format yet (we've been discussing internally about opening the API so that developers can write their own serializers if they want)
it'll still be more complicated to setup and to use than ngx-translate
you can translate at runtime in your code, but the templates will be translated at bootstrap, which means that you cannot change the locale without reloading the app
new features will still take a long time to be implemented, because we don't have the freedom that a library like ngx-translate has

So what about ngx-translate future?

This has been a long post, and we're finally getting to the part where we talk about the future of ngx-translate. The way I see it there are 2 solutions:

1/ we continue like it has been for the past year, only merging a few PRs, not working on new features and not improving the workflow. I keep working on making Angular i18n better and hopefully people will stop using this library because the native solution will be better. Or maybe a real alternative will emerge (like https://github.com/robisim74/angular-l10n).

2/ I change the way I work and decide to spend more time on the library (at least 1 day/week) to improve it: fixing lazy loading, code splitting, adding functions to help with testing, support ICU expressions, create well documented and maintained examples, finally tackle ionic and other mobile issues, ... I focus on making sure that the library stays a viable alternative to Angular i18n by providing solutions to the "problems" that Angular i18n will not fix. I write better tooling, maybe even an editor for the translations. I rewrite the documentation, on a real website with shiny examples.

To do all of that, I would need to make this library profitable somehow. I was thinking about changing the license to require companies to pay for the library (while keeping it free for open source and personal projects) in exchange for priority support (for bugs fixing, and for working on the features that they want the most). Obviously the current version would remain free and MIT-licensed, since there's no legal way to retroactively change a license, and it would be stupid to do that anyway. But new versions would be using this new license.

This library is downloaded ~350 000 times / month on npm now, if you compare that to the ~2 400 000 downloads of @angular/core, it means that my library is used in 1 out of 7 Angular projects! That's crazy, and it gives me confidence that this is viable.

What do you guys think? Would your companies be willing to pay for a better ngx-translate? Do you see any other alternative?

View on GitHub

Since we can't wait for Ivy to happen, we deployed our own translation system that has barely any code or logic (as opposed to ngx-translate that has a bunch).

As preparation, grab files from ngx-pttx github project and copy them to your project (no npm package created yet).

The approach

You need to have the translations available in a hierarchical JSON structure. An example (in en-US.json) would be:

{
  "mygroup": {
    "mykey": "Hello my english text"
  }
}

Your application will download this once it knows what file to get (i.e. what language is chosen). The format may be familiar from angular-translate from your wild Angular 1 years.

Then, you set the strings to be available globally by injecting TxService and doing setStrings():

export class AppComponent {
  constructor(
    private txService: TxService
  ) {}
  ngOnInit() {
    this.fetchStringsSomehow.then(strings => this.txService.setStrings(strings));
  }

Now you can just use txService in various components:

const text = this.txService.instant("mygroup.mykey");

// warning: this can crash at runtime
const text2 = this.txService.txs.mygroup.mykey;

In templates you can use ptTx pipe:

<span>{{ "mygroup.mykey" | ptTx }}</span>

Or ptTx directive:

 <span ptTx="mygroup.mykey"></span>

Now you may ask - how is that type safe? Well, it isn't.

Since you want type safety (to ensure your strings keep working and to get intellisense), you need to create type definitions. We do it in a build step, with quicktype.io command line application. Point it to the translation json file (one, typically english, is enough for type generation):

yarn quicktype --lang ts --no-maps --just-types --out gen/translations_common.ts ../../some_dir_to_translations/en-US.json

Now, we can use txService.make() to create a function that checks the type and gives you the string. It takes the quicktype-generated type as type argument:


import { MyUiStrings } from "./gen";

  private getLocalizedTexts() {
    const tx = this.txService.make<MyUiStrings>();

    return {
      someText: tx(s => s.mygroup.mykey)
    };
  }