The Challenge of Versioning Expandable API's in Umbraco

Since Umbraco v12 there has been a big push towards making all the core Umbraco products headless. A cool feature of many of these API's is the expansion functionality.

What the expansion functionality provides is an ability to selectively expand nested entity reference objects to their fully populated counterparts.

Lets take a snippet of an Order entity from the Umbraco Commerce Storefront API as an example:

// [GET] /umbraco/commerce/storefront/api/v1.0/order/599f90b3-2647-420a-b7ef-0cd88d69a50b
{
    "id": "599f90b3-2647-420a-b7ef-0cd88d69a50b",
    "orderNumber": "ORDER-01541-34602-5F73L",
    "orderLines": [],
    "currency": {
        "$type": "CurrencyRef",
        "id": "e9368f1a-dcf4-4dc6-ba1e-018dfff40f12",
        "code": "GBP"
    },
    ...
}

Here we can see an order has a connection to a Currency entity which by default will just return a reference object

{
    "$type": "CurrencyRef",
    "id": "e9368f1a-dcf4-4dc6-ba1e-018dfff40f12",
    "code": "GBP"
}

If we now made the same request but this time pass in a query parameter of expand=currency this will now expand the currency to the full entity and return the following:

// [GET] /umbraco/commerce/storefront/api/v1.0/order/599f90b3-2647-420a-b7ef-0cd88d69a50b
// -param expand=currency
{
    "id": "599f90b3-2647-420a-b7ef-0cd88d69a50b",
    "orderNumber": "ORDER-01541-34602-5F73L",
    "orderLines": [],
    "currency": {
        "$type": "Currency",
        "id": "e9368f1a-dcf4-4dc6-ba1e-018dfff40f12",
        "code": "GBP",
        "name": "GBP"
        "culture": "en-GB",
        "allowedCountries": [
            {
                "country": {
                    "$type": "CountryRef",
                    "id": "11959743-c7f9-4a23-9b93-018dfff40f3d",
                    "code": "GB"
                }
            }
        ],    
    },
    ...
}

We can go even further than this too by expanding even deeper entities:

// [GET] /umbraco/commerce/storefront/api/v1.0/order/599f90b3-2647-420a-b7ef-0cd88d69a50b
// -param currency[allowedCountries[country]]
{
    "id": "599f90b3-2647-420a-b7ef-0cd88d69a50b",
    "orderNumber": "ORDER-01541-34602-5F73L",
    "orderLines": [],
    "currency": {
        "$type": "Currency",
        "id": "e9368f1a-dcf4-4dc6-ba1e-018dfff40f12",
        "code": "GBP",
        "name": "GBP"
        "culture": "en-GB",
        "allowedCountries": [
            {
                "country": {
                    "$type": "Country",
                    "id": "11959743-c7f9-4a23-9b93-018dfff40f3d",
                    "code": "GB",
                    "name": "United Kingdom"
                    "defaultCurrency": {
                        "$type": "CurrencyRef",
                        "code": "GBP",
                        "id": "e9368f1a-dcf4-4dc6-ba1e-018dfff40f12"
                    },
                    "defaultPaymentMethod": {
                        "$type": "PaymentMethodRef",
                        "alias": "invoicing",
                        "id": "92037283-e693-4e25-8386-018dfff41062"
                    },
                    "defaultShippingMethod": {
                        "$type": "ShippingMethodRef",
                        "alias": "pickup",
                        "id": "236e5ca1-e340-4da1-8de4-018dfff41113"
                    },
              }
        ],    
    },
    ...
}

This has however recently presented a bit of a challenge that we don't yet have a solution for (which I'm aiming to try and suggest an approach in this post) and that is to do with versioning.

Versioning

When it comes to versioning REST APIs it generally boils down to two possible approaches.

Endpoint Versioning

With this strategy the version is controlled via a version number usually somewhere in the endpoints URL structure. eg /umbraco/commerce/storefront/api/v1.0/order/. When a model changes, you duplicate the old endpoint, incrementing the version number and return the new model from the new endpoint.

This is the approach Umbraco have currently chosen.

Model Versioning

With model versioning you tend to have a single endpoint but then either pass a header or a custom mime type to control which version of the model to return. This reduces the number of endpoints you have to maintain, but it's a lot less obvious.

The Problem

Both of these strategies however are problematic when you take expansion into account.

These versioning strategies are great so long as you only ever access an entity via it's own endpoint, but with the expansions API, the nested entities are retrieved within the output of another endpoint.

Lets take our original example of an Order and a Currency. Now lets say I recently made a change to the Currency entity and so now it's endpoints are all on v2.0 but the Order entity itself hasn't had any changes and so its endpoints are still on v1.0.

If I request an order from /umbraco/commerce/storefront/api/v1.0/order/599f90b3-2647-420a-b7ef-0cd88d69a50b and ask it to extend it's currency property, which version of the currency entity should it return?

Suggestion 1: Version the whole API

You could say "Why not just version the whole API so any change results in a version bump for the whole API?". Well, apart from the fact this would just create hundreds of endpoints we'd have to maintain, there is also the issue of cross product references.

In Umbraco, we have property editors such as pickers than can reference entities in other products e.g. the Umbraco Commerce Store Picker. When an Umbraco content item is returned, it can return a store entity from the Umbraco Commerce Storefront API as it's model value.

So now, if someone was accessing v1 of the Umbraco Content Delivery API, which version of the Store entity from the Storefront API should it return?

Suggestion 2: Switch to Model Versioning

So what if we switched to model versioning. Could this work?

Lets say we introduced some headers to control the versions of models to return. It might look something like this:

// [GET] /umbraco/commerce/storefront/api/order/599f90b3-2647-420a-b7ef-0cd88d69a50b
// -header Order-Version=1.0
// -header Currency-Version=2.0
{
    "id": "599f90b3-2647-420a-b7ef-0cd88d69a50b",
    "orderNumber": "ORDER-01541-34602-5F73L",
    "orderLines": [],
    "currency": {
        "$type": "Currency",
        "id": "e9368f1a-dcf4-4dc6-ba1e-018dfff40f12",
        "code": "GBP",
        "name": "GBP"
        "culture": "en-GB",
        "allowedCountries": [
            {
                "country": {
                    "$type": "CountryRef",
                    "id": "11959743-c7f9-4a23-9b93-018dfff40f3d",
                    "code": "GB"
                }
            }
        ],    
    },
    ...
}

So here, instead of having the version in the endpoint URL, we pass header parameters that dictate the exact versions of the entities we want to return.

This could work, but I think this starts to add some major overhead for implementors.

Every time a developer expands an entity, they would have to add a new header to state which version of that entity they want to support. This would again be even more cumbersome when you take the cross product scenario into account.

My Suggestion: Date Based Versioning

So, what do I suggest? Well, first of all, let me start by saying this is currently just the seed of an idea and I haven't worked out the major details. I'm really using this blog post to document my thoughts, but ultimately, I am wondering if a date based versioning strategy could work.

So my thought is, what if when people build a project using the headless APIs on every request they pass a single date based parameter which is the date their integration started?

// [GET] /umbraco/commerce/storefront/api/order/599f90b3-2647-420a-b7ef-0cd88d69a50b
// -header Integration-Date=2024-03-31 06:16:00

Each endpoint could then use that date to work out what the latest model was up to, but not later than that date and then return that version of the model.

When someone upgrades their project, they could then update their integration date header and it would start to use models released up to that integration date, but not after.

This would ensure that no matter what endpoint an entity was accessed from there was a single flag that could control the model version to return.

Further Considerations

This does introduce some other questions of it's own however.

How would we manage model changes? How do we signal the date of a model change? Could we introduce a C# attribute to decorate models with the date of when that model was introduced? then could we have something that just automatically picks the right model for us based on those attribute?

There is also the question of how this affects the Swagger docs / Open API spec generation. There is great value in having the generated Open API specifications as they allow developers to auto generate API clients. Could we make this work with the date headers? Could we pass a querystring to the Swagger endpoint and then generate the Open API spec dynamically?

Conclusion

As I said at the beginning, this is still very much open for debate and I don't yet have all the answers. The only thing I can say is the later considerations feel much more like coding challenges than fundamental API issues so maybe those would be easier to solve.

I'd love to hear peoples thoughts on this and whether anyone has seen any solutions to this? (I'm guessing API's supporting expansion aren't all that common).