ECMAScript modules. Going beyond the ordinary

Let’s say you’re working on a static site generator, and while implementing new features you decide to import text files (HTML, MD, CSS, SVG, etc) as dependencies into JavaScript files. Sure, you may use a bundler with some loader, but what if your bushidō mandates you not to type npm install something every time you have to solve a problem? Well, there is a solution.

But first, make sure that:

1. You know what ESM is about.
2. You have Node.js installed, and this Node.js is not outdated.
3. You’re not using .mjs as a file extension for your JavaScript code, browsers do not like them, even though the syntax of the files are the same as .js. However, it’s important only if you want to write isomorphic code.

Now, let’s continue.

Exercise one. Node.js. Custom loader

Create a loader:

// loader.js

import { URL } from 'url';
import { readFile } from 'fs/promises';

function isAllowedURL(url) {
  return ['.html', '.htm', '.md', '.css', '.svg', '.json'].some(x => {
    return url.endsWith(x);
  });
}

export async function load(url, context, defaultLoad) {
  if (!isAllowedURL(url)) {
    return defaultLoad(url, context, defaultLoad);
  }

  const content = (await readFile(new URL(url))).toString();
  const json = url.endsWith('.json') ? content : JSON.stringify(content)

  return {
    format: 'module',
    source: `export default ${json};`,
    shortCircuit: true,
  };
}

Then create a file where you want to import some non-JS stuff:

// app.js

import html from './index.html';
import doc from './doc.md';
import css from './styles.css';
import svg from './image.svg';
import data from './data.json';

console.log(html, doc, css, svg, data);

Now, boom:

$ node --loader ./loader.js ./app.js

Huh?

import { URL } from 'url';
^^^^^^

SyntaxError: Cannot use import statement outside a module

Oh, yeah, don’t forget to set "type": "module" in the package.json of your project — otherwise imports don’t work.

But with this small key in the package.json everything just works, without the need for additional packages, bundlers, etc.

As you see on the screenshot, you may get this warning:

(node:22662) ExperimentalWarning:
Custom ESM Loaders is an experimental feature.
This feature could change at any time

That’s because the feature is still experimental, but, well, it works.

If you want to read more about plans of Node.js team related to loaders, check nodejs/modules#351 issue.

Exercise two. Browser. Request interception

Now move to a browser. The task remains the same. We want to load modules, but not JS modules, rather we want static files with styles, templates and plain text. Browsers do not support special “hooks” for ES Modules. What’s the plan?

Well, let’s create a dragnet using Service Workers.

The tools are different, but the approach is the same.

Create a Service Worker file:

// sw.js

function isAllowedURL(url) {
  return ['.html', '.htm', '.md', '.css', '.svg', '.json'].some(x => {
    return url.endsWith(x);
  });
}

async function handleRequest(req) {
  const content = await (await self.fetch(req.url)).text();
  const json = req.url.endsWith('.json') ? content : JSON.stringify(content);

  return new Response(
    `export default ${json};`,
    {
      headers: {
        'Content-Type': 'text/javascript'
      },
    },
  );
}

self.addEventListener('fetch', e => {
  if (e.request.destination === 'script' && isAllowedURL(e.request.url)) {
    e.respondWith(handleRequest(e.request));
  }
});

Register it somewhere on an HTML page:

<!-- index.html -->

<script>
  navigator.serviceWorker.register('./sw.js');
</script>

Use the same imports as in the Node.js example above:

// module.js

import html from './index.html';
import doc from './doc.md';
import css from './styles.css';
import svg from './image.svg';
import data from './data.json';

console.log(html, doc, css, svg, data);

But this time, instead of marking package.json as a “module,” do the same on the script tag:

<!-- index.html -->

<script src="./module.js" type="module"></script>

Now serve the page, open DevTools and check the logging:

No bundlers, no additional stuff. Just pure JavaScript and browser API. Cool, isn’t it?

If you decide to use this solution in a real project, read Service Worker’s docs carefully. There are some important limitations and nuances of compatibility with different web browsers. For instance, currently Firefox does not run Service Workers in private browsing mode. This is by design, but probably will be changed in the future.

Exercise three. Node.js + Browser. Request parameters

Okay, now it’s time for sick stuff. It’s just an experiment and I don’t encourage you to try it, but why not try when you can?

If you don’t know, ES Modules support top level await. It means that before a module “returns” its export, you can do something asynchronous in it. For instance, make a network call and get a response. Where does this lead us? Right.

Create a loader module:

// load.js

let result = null;

const path = import.meta.url.split('?')[1];

if (path) {
  let content;

  if (typeof window === 'object') {
    content = await (await fetch(path)).text();
  } else {
    const fs = (await import('fs')).default;
    content = fs.readFileSync(path).toString();
  }

  result = path.includes('.json') ? JSON.parse(content) : content;
}

export default result;

Import everything else using this module as a proxy:

// module.js

import html from './load.js?./index.html';
import doc from './load.js?./doc.md';
import css from './load.js?./styles.css';
import svg from './load.js?./image.svg';
import data from './load.js?./data.json';

console.log(html, doc, css, svg, data);

Even though the example is simplified, it’s already isomorphic. Which means you can use the code above in a browser and in Node.js without any changes. Try it.

I don’t think that anyone will seriously use this method to load modules, but the approach itself is at least interesting to think about.

An important thing to note here is that each module’s exports are cached. The cache is based on the module’s URI. Therefore, if the imported files are changed dynamically, their URIs should be changed too. That means that it’s possible to use dynamic imports only in this kind of situation. But when we have a top level await support, it’s not a huge problem.

Conclusion. Kind of

You may ask, “Is there any value in this at all? It’s easier to use fs.readFile or fetch instead of importing a module!”

I would like not to answer. You never know when the weird knowledge you have will help you in the future.

Anyway, I hope it was a fun ride! Maybe you’ve learned something new and useful.

Use more ESM and less CJS. Peace!

Originally published by Sergey Matiyasevich in Uploadcare Blog