MikroORM 5: Stricter, Safer, Smarter

The next major version of MikroORM has been just released. The title says: Stricter, Safer, Smarter — why?

• Greatly improved type safety (e.g. populate and partial loading hints)
• Auto-flush mode (so we never lose in-memory changes)
• Automatic refreshing of loaded entities (say goodby to refresh: true)
• Reworked schema diffing with automatic down migrations support
• and many many more…

This time it took almost a year to get here — initial work on v5 started back in March 2021.

In case you don’t know…

If you never heard of MikroORM, it’s a TypeScript data-mapper ORM with Unit of Work and Identity Map. It supports MongoDB, MySQL, PostgreSQL, and SQLite drivers currently. Key features of the ORM are:

• Implicit transactions
• ChangeSet based persistence
• Identity map

You can read the full introductory article here (but note that many things have changed since that was written) or browse through the docs.

Quick summary of 4.x releases

Before we dive into all the things v5, let’s recap what happened in 4.x releases:

• Result cache
• Automatic transaction context
• Nested embeddables and many other improvements in this domain
• Using env vars for configuration

But enough of the history lesson, let’s talk about the future!

Improved type safety

Let’s jump right into the most interesting feature — strict typing (almost) everywhere! em.create(), toJSON(), toObject(), populate, partial loading, and order by hints, all of that (and even more!) is now strictly typed.

Let’s check the following example:

First, we use em.create() to build the whole entity graph in a single step. It will validate the payload for both types and optionality. Some properties on the entity might have default values provided via hooks or database functions — while we might want to define them as required properties, they should act as optional in the context of em.create(). To deal with this problem, we can specify such properties that should be considered as optional via OptionalProps symbol:

Some property names are always considered as optional: id, _id, uuid.

Then we load all Author entities, populating their books and the book tags. All of the FindOptions here are strictly typed, moreover, we could even skip the populate hint as it can be inferred from fields option automatically.

We might still need some type casting for DTOs. The serialized form of an entity can be very unpredictable — there are many variables that define how an entity will be serialized, e.g. loaded relation vs reference, property serializers, lazy properties, custom entity serializer/toJSON method, eager loading, recursion checks, … Therefore, all relations on the EntityDTO type are considered as loaded, this is mainly done to allow better DX as if we had all relations typed as Primary | EntityDTO (e.g. number | EntityDTO), it would be impossible to benefit from intellisense/autosuggestions. Imagine this scenario:

Validation improvements

Adding on top of the compile-time validation, we also get a runtime validation right before insert queries are fired, to ensure required properties have their values. This is important mainly in mongo, where we don’t have optionality checks on the schema level.

When we try to use the CLI without installing it locally, we also get a warning. And what if we forget to update some of the ORM packages and ended up with version mismatch and multiple installed core packages? We now validate that too!

Reworked schema diffing

Schema diffing has been one of the weakest spots. Often, additional queries were produced or it was even impossible to get to a fully synchronized state.

Schema diffing has been completely reworked to address all currently known issues, and adding a bit more on top of that:

• Diffing foreign key constraints
• Proper index diffing (before we compared just names)
• Custom index expressions
• Comment diffing
• Column length diffing (e.g. numeric(10,2) or varchar(100))
• Changing primary key types
• Schema/namespace diffing (Postgres only)
• Automatic down migrations (no SQLite support yet)
• Check constraints support (Postgres only)

Smarter migrations

In the production environment, we might want to use compiled migration files. Since v5, this should work almost out of the box, all we need to do is to configure the migrations path accordingly. Executed migrations now ignore the file extension, so we can use both node and ts-node on the same database. This is done in a backward-compatible manner.

Creating new migration will now automatically save the target schema snapshot into the migrations folder. This snapshot will be then used if we try creating a new migration, instead of using the current database schema. This means that if we try to create new migration before we run the pending ones, we still get the right schema diff (and no migration will be created if no additional changes were made).

Snapshots should be versioned just like the regular migration files.

Auto-flush mode

Up until now, flushing was always an explicit action. With v5, we can configure the flushing strategy, similarly to how JPA/hibernate work. We have 3 flush modes:

• FlushMode.COMMIT - The EntityManager tries to delay the flush until the current transaction is committed, although it might flush prematurely too.
• FlushMode.AUTO - This is the default mode, and it flushes the EntityManager only if necessary.
• FlushMode.ALWAYS - Flushes the EntityManager before every query.

FlushMode.AUTO will try to detect changes on the entity we are querying, and flush if there is an overlap:

More about flush modes in the docs.

Automatic refreshing of loaded entities

Previously, when an entity was loaded and we needed to reload it, providing explicit refresh: true in the options was required. Refreshing of entity also had one problematic side effect — the entity data (used for computing changesets) were always updated based on the newly loaded entity, hence forgetting the previous state (resulting in possibly lost updates done on the entity before refreshing).

Now we always merge the newly loaded data with the current state, and when we see an updated property, we keep the changed value instead. Moreover, for em.findOne() with a primary key condition, we try to detect whether it makes sense to reload an entity, by comparing the options and already loaded property names. In this step the fields and populate options are taken into account to support both partial loading and lazy properties.

For complex conditions in em.findOne() and for any queries via em.find(), we always do the query anyway, but now instead of ignoring the data in case such entity was loaded, we merge them in the same manner.

Seeder package

MikroORM v5 now has a new package for seeding your database with initial or testing data. It allows creating entities via the same EntityManager API as usual, adding support for entity factories, and generating fake data via faker (the newly release community version).

See the seeder docs for more examples.

Polymorphic embeddables

Polymorphic embeddables allow us to define multiple classes for a single embedded property and the right one will be used based on the discriminator column, similar to how single table inheritance works. While this currently works only for embeddables, support for polymorphic entities will be probably added in one of the 5.x releases.

Check out the documentation for a complete example.

There are many other small improvements in embeddables, as well as many issues were addressed. Two examples:

• Support for many-to-one relations (storing only primary key and being able to populate the relation same as with regular entities)
• Support for onCreate and onUpdate property options

Populating lazy scalar properties

Previously, the only way to populate a lazy scalar property was during the initial load of containing entity. If such entity was already loaded in the identity map (without this property), we needed to refresh its state — and potentially lose some state. MikroORM v5 allows to populate such properties via em.populate() too. Doing so will never override any in-memory changes we might have done on the entity.

Creating references without EntityManager

When we wanted to create a reference, so an entity that is represented only by its primary key, we always had to have access to the current EntityManager instance, as such entity always needed to be managed.

Thanks to the new helper methods on the Reference class, we can now create entity references without access to EntityManager. This can be handy if you want to create a reference from an inside entity constructor:

The Reference wrapper is an optional class to allow more type safety over relationships. Alternatively, we can use Reference.createNakedFromPK().

This will create an unmanaged reference, that will be then merged to the EntityManager once owning entity gets flushed. Note that before we flush it, methods like Reference.init() or Reference.load() won’t be available as they require the EntityManager instance.

Smarter expr helper

The expr() helper can be used to get around strict typing. It was an identity function, doing nothing more than returning its parameter — all it did was to tell TypeScript the value is actually of a different type (a generic string to be precise).

We can now use the helper in two more ways:

• With a callback signature to allow dynamic aliasing of the expression
• With an array argument to allow comparing tuples

Awaitable QueryBuilder

QueryBuilder is now aware of its type, and the getResult() and execute() methods are typed based on it. We can also await the QueryBuilder instance directly, which will automatically execute the QB and return the appropriate response. The QB instance is now typed based on usage of select/insert/update/delete/truncate methods to one of:

• SelectQueryBuilder — awaiting yields array of entities
• CountQueryBuilder — awaiting yields number
• InsertQueryBuilder — awaiting yields QueryResult
• UpdateQueryBuilder — awaiting yields QueryResult
• DeleteQueryBuilder — awaiting yields QueryResult
• TruncateQueryBuilder — awaiting yields QueryResult

Wildcard schema entities

Up until now, we were able to define entities in a specific schema, or without a schema. Such entities then used the schema based on ORM config or FindOptions. This allowed us to read entities from a specific schema, but we were missing the power of Unit of Work here.

With v5, entity instances now hold schema name (as part of WrappedEntity). Managed entities will have the schema from FindOptions or metadata. Methods that create new entity instances like em.create() or em.getReference() now have an options parameter to allow setting the schema. We can also use wrap(entity).getSchema() and wrap(entity).setSchema().

Entities can now specify wildcard schema via @Entity({ schema: '*' }). That way they will be ignored in SchemaGenerator unless the schema option is specified.

• If we specify schema, the entity only exists in that schema
• If we define * schema, the entity can exist in any schema, always controlled by the parameter
• If we skip schema option, the value will be taken from global ORM config

More about this topic can be found here.

Deep assigning of entities

Another weak spot was assigning new values to existing entities. While wrap().assign() was originally designed to update a single entity and its values, a lot of users wanted to assign an entity graph, updating relations in a single step too.

With v5, the way how EntityAssigner detects what entity should be updated has changed. Assigning a deep entity graph should be possible by default, without any additional options. It works based on matching entity primary keys, so if you want to issue an update for a relationship instead of creating new relation, make sure you first load it and pass down its primary key to the assign helper:

If we want to always update the entity, even without the entity PK being present in data, we can use updateByPrimaryKey: false:

More examples on this topic can be found in the docs.

Experimental support for ES modules

While MikroORM v5 is still compiled and published as CommonJS, we added several improvements that should allow using it with ESM projects too. Namely, we use the gen-esm-wrapper package to allow using named imports, and we use one nasty trick to keep dynamic imports instead of compiling them to require statements — for that we need to use MIKRO_ORM_DYNAMIC_IMPORTS env var. This should allow us to use folder-based discovery with ES modules, which was previously not possible.

Other notable changes

• Partial loading support (fields) for joined loading strategy
• AsyncLocalStorage used by default in the RequestContext helper
• onLoad event (like onInit, but allows async and fires only for loaded entities, not references)
• Exporting async functions from CLI config
• Configurable aliasing strategy for SQL
• Allow providing customLogger instance
• persist option inem.create() andpersistOnCreate global configuration
• M:N support in entity generator
• Support for specifying transaction isolation level
• Controlling where condition for populate hints
• Revamped API docs
• and many many more, see the full changelog here

Also be sure to check the upgrading guide.

What’s next?

Here is a list of things I would like to focus on going forward:

• allow specifying pivot entity for M:N relations (so we can have additional columns there, but still map it as M:N for reading purposes)
• support for database views (or maybe just entities representing SQL expressions)
• more drivers — namely better-sqlite3 and cockroach sounds like low hanging fruit, given knex now supports those natively

Like MikroORM? ⭐️ Star it on GitHub and share this article with your friends. If you want to support the project financially, you can do so via GitHub Sponsors.

---