Running Typescript programs with babel-node aka how does babel register work

As a member of a frontend infrastructure team, (aka DivOps), I write a lot of Javascript tooling. I also really really like Typescript. In fact most of the tooling I write these days is written in Typescript.

The problem is, when the tools we work on are handed off to our end users, they have to either be compiled with tsc, or babel first. Generally we typecheck with tsc and transpile with the @babel/preset-typescript preset.

Sometimes though, that compilation step causes a delay, and we'd rather just run the code as is, Typescript, ES modules and all. That's where tools like @babel/node come in.

@babel/node allows you to "transpile" code from Typescript, or "ES Next" JavaScript with modules, or whatever to Javascript that can run in nodejs, and it does this at runtime as opposed to pre-transpiling.

Aka you can do stuff like this, and run Typescript files without "pre" compiling them.

babel-node -x .ts -- ./path/to/script.ts

So, how does @babel/node actually work?

npm i -D @babel/node

When you install this pacakage, it will add a babel-node command to your ./node_modules/.bin folder. This .bin folder is used for all CLI tools you install, things like webpack, babel-node, etc. The .bin then just symlinks the babel-node command to ./node_modules/@babel/node/bin/babel-node. This works because of the "bin" being set to "babel-node": "./bin/babel-node.js" right here.

Then, babel-node is simply a program with a shebang that tells the system to what interpreter to use for the program, in this case the /usr/bin/env command is what actually runs first, and makes sure that it can find node.

#!/usr/bin/env node

The babel-node program itself then just imports require("../lib/babel-node");

This is where we start getting into the actual runtime of @babel/node.

The lib/babel-node.js script is responsible for reading in any runtime arguments for v8 runtime stuff like --harmon or node env variables like --require.

Ultimately the lib/babel-node.js uses a program called kexec, a C++ programwhich swaps out a process at runtime, to actually run ./lib/_babel-node.js

./lib/_babel-node.js is where things really start to happen as this is really the meat of what @babel/node is actually doing. It uses a CLI helper utility called commander and sets it up with the following possible args.

You'll notice a few of those are similar to ones you'd use when working directly with node itself. Things like -e, -p, and -r are all flags that node itself also accepts, other flags like --inspect fall through to the node process that will ultimately get spun up.

Eventually after some args parsing, an internal Module.runMain function) happens and runs your script by overwriting the arguments passed with process.argv.

process.argv = [
 '/path/to/node/v12.17.0/bin/node',
 '/path/to/node_modules/@babel/node/lib/_babel-node',
 ...args,
];

Then what happens in this line is, node will now run /path/to/_babel-node, and send it your actual runtime arguments.

So, that's how _babel-node is actually invoked, but how does it transpile Typescript to Javascript on the fly?

The answer there is @babel/register.

@babel/register

@babel/register is the special sauce of how @babel/node is able to actually read in Typescript, or ES Next Javascript, transpile it to commonjs, and run it on the fly.

Using @babel/register is one way the docs say to setup your code.

Basically instead of babel-node -- path/to/your/script.ts you can manually call the @babel/register module itself by importing it in your code and call it with node /path/to/your/script.ts, in the case of Typescript though

require("@babel/register")({
  extension: ['.ts', '.tsx', '.js', '.jsx']
});
const { main } = require("./src/cli");

main().catch((e) => console.error(e));

Keep in mind with register, you have to separate your entry point from your code because register has to run and add the require hooks to the NEXT files you'll load, aka you can't have import and require("@babel/register"); in the same file. Note to use Typescript and Typescript with React, you'll have to add the "extensions". The other arguments you can pass to register are the same as the regular babel options.

In terms of @babel/node, the register function gets called here and is passed the command line args, and some additional configuration for babel.

@babel/register uses a library called pirates to add a compile hook using some internal Module.__extensions magic to change the underlying Javascript loader in node. It does that here.

So basically every time a file gets require'd going forward in your program, it'll first run through the compile function, get compiled, and finally executed.

Running and debugging babel-node

There's a couple of different ways to run a script with babel-node, just like with node itself.

Passing it the name of a script will invoke the script just like it would with node.

babel-node -x .ts -- ./path/to/script.ts

👆note the -x .ts or --extensions .ts there to make it recognize the Typescript .ts extension as well as .js

You can also add the --inspect flag and in VS Code or using chrome://inspect, you can debug your command line script. In VS Code make sure you have on Auto Inspect, and you'll then be able to throw breakpoints into your code.

If you want to use the Chrome developer tools to debug your script, make sure you throw a debugger; statement somewhere in your code, then you can run...

babel-node -x .ts --inspect-brk -- ./path/to/script.ts

The --inspect-brk will force the process to pause at the very first line of execution, which actually in this case will pause in the _babel-node.js script since that's technically the first line of code that gets executed. Then you can visit chrome://inspect and find your node script in the list, and click inspect.

You'll then get dropped into a Chrome Debugger.

You can run babel-node with -e and execute code on the fly.

babel-node -e "console.log('oh hai');"

You can run a script in combo with -p and print the results...

babel-node -e "new Date().getTime()" -p
// 1607531838009

Lastly you can run babel-node by itself, to spin up a repl for playing around in node with an environment that will transpile code on the fly for you.

Gotchas

One thing in particular that has bit me many times, and partly is the reason why I am writing this post to begin with is,

When using babel register, call the register function before your code

As mentioned previously, make sure when using @babel/register, call the require("@babel/register") first, then require your actual code. Otherwise it won't work at all.

babel-node's command line ignore overrides the ignores in your babel.config.js

This one seems like a bug, but basically, if your babel.config.js has any kind of ignore in it...

module.exports = {
  /* ... blah blah babel stuff */
  ignore: [function(filepath) {
    return filepath.includes('some-string-you-wanna-ignore');
  }]
}

You have to pass an empty ignore in your babel-node --ignore ' ' path/to/script.ts. It probably has to do with this line. Register is expecting ignore not to be undefined, and it's not checking that against the ignore from the babel.config.js.

babel-node will add your process.cwd() location to the --only param by default.

This might be fine in your codebase, but if you're expecting babel-node to be able to transpile code from outside of your cwd, it simply won't work because @babel/register . This goes back to the same line above. If no --only flag is passed at run time, babel will by default ignore anything outside your current working directory AND anything in $(cwd)/node_modules.

So, say you have a project structure like this.

packages/tools/my-awesome-tool/
packages/utils/
package.json

If you have a script in my-awesome-tool/scripts/cool-script.ts which imports from packages/utils and you invoke it FROM the directory, i.e. if you did cd packages/tools/my-awesome-tool, the scripts inpackages/utils will NOT get transpiled by default.

There are 2 ways around this. First, just run all your scripts from the root of your project. Second, you can pass --only /path/to/project. That way it won't ignore stuff.

Make sure to add the -- thing when running a script

If you forget to add -- in between your options i.e. babel-node -x .ts path/to/scripts.ts instead of babel-node -x .ts -- path/to/script.ts, then babel-node is going to try to parse through the options you passed to your OWN script. This is especially important when the script your calling takes it args.

For example...

babel-node -x .ts -- ./node_modules/.bin/build-storybook \
  -c packages/tools/storybook/src/config \
  -o packages/tools/storybook/dist/storybook

In this case, we're calling the build-storybook command with babel-node, and the -o flag is setting the output dir for storybook. If you forget the --, the -o param will actually be parsed as the --only flag for babel!

Conclusion

@babel/node is a nice convenient way to run scripts on the fly without pre-compiling them. Just be aware of some its nuances and you'll be able to build some amazing tools.

This post was written primarily because many of the things mentioned are things I've dealt with in real life, and I kept running into them over and over again, and finally decided it was time to write them down for myself! Hopefully it can help you too.