Betterer v5.0.0

Oof, this one feels like it's been a while coming, but after a whole bunch of work, and a whole bunch of breaking changes I've just released v5.0.0 of Betterer! 🎉

What is Betterer?

Betterer is a test runner that helps make incremental improvements to your code!

The first time Betterer it runs a test, it will take a snapshot of the current state. From that point on, whenever it runs it will compare against that snapshot. It will either throw an error (if the test got worse ❌), or update the snapshot (if the test got better ✅). That's pretty much it!

You can check out the (newly updated!) documentation at https://phenomnomnominal.github.io/betterer/

What happened to v2, v3, v4...?

"But Craig", I hear you say, "The last time you posted about Betterer, it was at v1.0.0!? What's been going on?!". 🔥🔥🔥

That's very astute of you dear reader, and let's just put it this way - I sure do love breaking APIs! One of the interesting things about Betterer is that it is a tool designed for problems that emerge in large and old codebases. That means it has to be able to handle large and old codebases from the get go! So I've had a lot of fun as I've tried to figure out the best workflows and APIs for using Betterer.

Between v1.0.0 and now, I've released a bunch of features, consolidated and simplified APIs, and just generally made Betterer more usable and flexible. I'm pretty happy with where it is at now, so I figured it was about time for an update. I've even been talking about it at a few conferences now that they're coming back! What a world 🌍!

What's in v5.0.0?

Parallel tests:

Performance is hard. Prior to v5, the default Betterer reporter would struggle pretty badly, especially when lots of tests were running and producing lots of issues. That was because the main thread was responsible for updating the reporter output and running all the tests.

To fix this, Betterer will now execute all your tests using Node.js Worker Threads! That frees up the main thread to focus on rendering and also means that multiple tests can run at the same time. Getting this to work required breaking some APIs, so your test definition file needs to change:

Before:

// .betterer.ts
import { BettererTest } from '@betterer/betterer';

export default {
  'my test': new BettererTest({
    // ... test config
  }),
  'my other test': new BettererTest({
    // ... test config
  })
};

After:

// .betterer.ts
import { BettererTest } from '@betterer/betterer';

export default {
  'my test': () =>
    new BettererTest({
      // ... test config
    }),
  'my other test': () =>
    new BettererTest({
      // ... test config
    })
};

But never fear, you can use the betterer upgrade command to do this migration for you! Just running betterer upgrade will show you what the migration will look like, and betterer upgrade --save will actually update your files. Easy ✨. The betterer upgrade command will be used in the future when I (most probably) break more stuff.

Check out the beast of a PR here (and yes, it took me three branches to get it right 😅)

Betterer ❤️ Angular

I've published a new Betterer test for incrementally adding Angular compiler configuration to a project! I'm pretty excited by this, as there are a lot of big Angular codebases out there that don't utilise the full power of the Angular compiler. In particular, I think Betterer could be a good way to introduce the strictTemplates option. You can now do that with the following:

//.betterer.ts
import { angular } from '@betterer/angular';

export default {
  'strict templates': () => angular('./tsconfig.json', {
    strictTemplates: true
  }).include('./src/**/*.ts', './src/**/*.html')
};

Expect to see a full post detailing this in the near future!

Simpler BettererFileTest:

The old BettererFileTest API was a bit clunky and confusing due to the BettererFileResolver thing. I've hidden that away in the internals, so now the public API is less clunky and confusing:

Before:

import { BettererFileResolver, BettererFileTest } from '@betterer/betterer';

function myFileTest() {
  const resolver = new BettererFileResolver();
  return new BettererFileTest(
    resolver,
    async (filePaths, fileTestResult) => {
      // test implementation...
    }
  );
}

After:

import { BettererFileTest } from '@betterer/betterer';

function myFileTest() {
  return new BettererFileTest(
    async (filePaths, fileTestResult, resolver) => {
      // test implementation...
    }
  );
}

Smaller public API, less magic, and you only have to use it if you know why, choice! 👍

Improved workflow:

I'm still working on figuring out the ideal Betterer workflow. For now, I recommend running Betterer in Pre-commit mode as a pre-commit hook (perhaps using husky and lint-staged) and in CI mode on your build server.

But one thing about chonky codebases is that they often have lots of contributors! Lots of contributors making changes (and making things better) means that 👻 merge conflicts 👻 in the results file are quite common!

To try to help with resolving merge conflicts, I've introduced the betterer merge command. You can still fix merge conflicts manually, but betterer merge will do it for you! If you're as lazy as me, you can even enable automerge and you'll never have to think about merging the results file ever again (I hope, this could still be buggy 🐛😅.

To enable automerge run:

betterer init --automerge

Improved caching:

Betterer got some cool (ish) caching implemented in v4, but turns out caching is a hard problem (😅), so it's taken a little bit to get right.

It works by passing the --cache flag when running Betterer:

betterer --cache

That will create a file something like this:

{
  "version": 2,
  "testCache": {
    "no hack comments": {
      "packages/angular/src/angular.ts": "b66de728222febdecb3cf11d3aa510b3a8a6ae0e37c0539e37787964573a56ad1b7eb6ee378a9087",
      "packages/angular/src/index.ts": "b66de728222febdecb3cf11d3aa510b3a8a6ae0eb9494122f82a750085fc20d2c3b0f14b34897431",
      "packages/betterer/src/betterer.ts": "b66de728222febdecb3cf11d3aa510b3a8a6ae0e94efcd2f99a4cf14222c400693335ac1b94696bb",
      // ...
    }
  }
}

Betterer will use this cache to only re-test files that have actually changes, so it can be much faster (useful for running on pre-commit!) I suspect there are still issues here, so please try it out and create issues. 🙌

Bug fixes and improvements:

• betterer results command to get a quick summary:

• Negative filters. I already suspect I'll regret this, but you can now use "!" at the start of a filter to negate it. Now --filter myTest will just run "myTest", and --filter !myTest will run every other test.

• Rewrote most of the public API docs. These are now generated from the code, so should hopefully be easier to keep up to date. 🤞

• Removed a bunch of stuff from the public API. This means more consistency, and I'll be less likely to accidentally break stuff in the future. 😇

Thanks ❤️

Huge thanks to everyone who has helped me with this stuff, if you've read my rambling, cryptic tweets, opened issues on Github, chatted to me about Betterer at conferences, it's all meant a lot! Maybe I'll print some stickers or something? ☀️

Love 🥰 this? Hate 🤬 this? Go off in the comments, DM me on Twitter, or be the third person to join the Betterer Discord. Catch you on the line 💻!