Published: / Updated:
Speed up your Craft CMS Templates with Eager Loading
Eager-Loading Elements allows you to speed up your Craft CMS templates by fetching entries from the database more efficiently.
Andrew Welch / nystudio107
One of the wonderful things about Craft CMS is that you have great flexibility in designing both the frontend and the backend of your website.
This is due in large part to Craft’s concept of Elements. Most of the things you interact with on the backend such as Entries, Categories, Assets, Users, and even Matrix blocks are all Elements. Elements are all stored separately, so any reference from one Element to another is a relation.
So if you create an Entry with an Asset field in it, the Asset isn’t actually stored in the Entry itself; instead, the Entry Element just points to the Asset Element.
This is a key concept, because understanding how Elements are stored allows you to understand how they are loaded
The following things in Craft CMS are all Elements:
- Assets
- Categories
- Entries
- Global Sets
- Matrix Blocks
- Tags
- Users
There are also Elements added by some plugins, such as Craft Commerce Products.
Let’s take a very simple example of a code pattern you probably see every day:
{% for entry in craft.entries.section('blog').limit(null) %}
<img src="{{ entry.blogImage.first().url }}" alt="{{ entry.title }}" />
{% endfor %}
Here we’re just looping through all of our Entries in the blog section, and outputting the blogImage Asset url in an <img> tag.
In order to do this, Craft has to first fetch all of the blog Entries, so it knows how many blog Entries there are. Then each time through the loop it accesses the blogImage Asset to get its URL.
But remember the Asset Element isn’t actually stored in the Entry Element, it’s a relation that points to the Asset Element. So each time through the loop, Craft has to do another database access to get the Asset Element. Only then can it output the URL.
Database accesses are costly, because they require building a query, then accessing the database, which then reads from the storage device
This is a classic n+1 problem, which just means that as the number of things goes up, the time it takes to do something with them increases linearly.
This is roughly equivalent to asking your significant other to run to the store to pick something up for you… and then as soon as they get back, you ask them to run back out and pick up something else. And then repeat this dozens, or even hundreds of times. Don’t try this at home, folks.
Normally this type of “lazy loading” that Craft does is a good thing, because it can’t know what relations you’ll need to access ahead of time. If Craft loaded everything every time, it’d be like buying out the entire store without regard for what you actually need.
So Craft just loads relations like Assets as you access them. But this also can be inefficient if we know we’re going to need certain things.
If only there was a way to tell Craft what we need ahead of time…
Enter Eager-Loading Elements
Consider our theoretical (and potentially dangerous) shopping analogy. Instead of adding another round trip to the store for our significant other for each thing we want, wouldn’t it be nice if we did something sensible? Something like preparing a shopping list ahead of time, and potentially saving our relationship?
This is exactly what Eager Loading allows you to do: tell Craft ahead of time what you’re going to need
Craft CMS introduced the concept of Eager-Loading Elements in Craft 2.6. Leveraging Eager-Loading Elements, our example would change to be this:
{% for entry in craft.entries.section('blog').with(['blogImage']).limit(null) %}
<img src="{{ entry.blogImage[0].url }}" alt="{{ entry.title }}" />
{% endfor %}
Two things to note here:
- We added .with(['blogImage']) to our craft.entries, which tells Craft “Hey, while you’re fetching these entries, we want all of the blogImage Asset Elements, too”
- We changed .first() to [0] (technically, we’re going from Object syntax to Array syntax)
What’s happened here is we’ve asked Craft to give us our Entries, and while it’s fetching them, to also fetch each blogImage Asset Element in each Entry, too.
Under the hood, when you ask for Entries that have relations to other Elements, the thing you get back is an ElementCriteriaModel that tells Craft how to retrieve the Element (in this case, an Asset). But it doesn’t actually retrieve it until you access it. This is lazy loading.
When you ask for Entries with relations to other Elements that are Eager Loaded, instead of an ElementCriteraModel, Craft fetches the relations and returns an array of the Asset elements. Thus the change from .first() Object syntax to [0] Array syntax.
For details on the syntax you can use for Eager Loading, check out the Craft help article Eager-Loading Elements.
Real World Examples
The reason we want to use Eager Loading is to reduce the number of SQL database queries needed to load a page, and thus increase performance.
So let’s take a look at some real world examples.
We will once again use this website you’re reading right now as our guinea pig. Here’s what the Field Layout looks like for our blog section:
Here’s what the fields actually are:
- Blog Category → Category Element
- Blog Tags → Tag Element
- Blog Summary → Rich Text Field
- Blog Image → Asset Element
- Blog Content → Matrix Block Element
- Blog SEO → SEOmatic Meta
So it’s a pretty simple setup, we only have 6 fields, but 4 out of the 6 fields are Elements, and thus are relations that we could be Eager Loading. The Blog Content field is a “content builder” Matrix Block for our blog content, as described in the Creating a Content Builder in Craft CMS article.
The Matrix Blocks for the Blog Content look like this:
Only the image Matrix Block type has a relation, in that it has an Asset Element in it (actually a Focus Point field, but it’s the same thing).
The first thing we should do before we address any performance issues is to gather data to see where the performance bottlenecks are. All of the timing tests are done in local dev on our Homestead VM, with devMode on and all caching is disabled so we can see the raw results.
Because we’re doing all of these timing tests in an environment with all sorts of debugging enabled, it’s going to be significantly slower in absolute terms than on live production. However, the relative timings should be roughly similar.
Here are the database queries for the Blog Index page:
Wow, so we have 120 database queries just to display a simple page of the 9 most recent blog entries, and a thumbnail image of each. Also don’t just look at the number of queries, look at the time: 1.24399s
Here’s what the code looks like that loads our entries:
{% set entries = craft.entries ({
section: 'blog',
order: 'postDate desc',
limit: limitBlogs,
relatedTo: relatedBlogs,
}) %}
{% for entry in entries %}
Let’s apply a bit of what we’ve learned, and Eager Load some of these Elements that are in our Entry as relations:
{% set entries = craft.entries ({
section: 'blog',
with: ['blogImage', 'blogCategory', 'blogTags'],
order: 'postDate desc',
limit: limitBlogs,
relatedTo: relatedBlogs,
}) %}
{% for entry in entries %}
So we’ve just added the line with: ['blogImage', 'blogCategory', 'blogTags'], to tell Craft we want to Eager Load these things, because we know we’ll be using them.
Note that even though the blogContent is a Matrix Block Element, and thus it could be a candidate for Eager Loading, we’re not asking for it to be Eager Loaded. That’s because we’re not using the blogContent at all on the Blog Index page, so there’s no reason to load it!
So how’d we do? Here’s what the timings look like after we’ve added Eager Loading:
Holy moly! We nearly cut the number of queries in half, down to 62 queries from 120, and the time down to 1.03165s from 1.24399s. That’s a 49% savings in the number of queries, and an 18% savings in the amount of time that it took.
While an 18% savings in time may not seem like a big deal in a vacuum, in real world conditions where there are multiple people hitting your website concurrently, there will be contention for the database that snowballs. Everyone has to get in line to have their database queries fulfilled, so any delay can have a cascading effect on the website’s performance.
Our Blog Index page is pretty lightweight anyway, it only loads the most recent 9 blog entries. Let’s scale it up a bit, and have it load all of the blog entries on the page (about 40 at current count):
Now we’re up to a whopping 352 queries that takes 3.25565s. Let’s see what it looks like if we add in Eager Loading:
With Eager Loading added, we’ve gotten it down to 120 queries from 352 queries, and the time down to 2.42931s from 3.25565s. That’s a 76% savings in the number of queries, and a 26% savings in the amount of time that it took.
I think it’s pretty clear that as the number of things goes up, the savings we get from Eager Loading just gets better and better. This will make our website scale up nicely as the client adds content.
Eager Load All The Things!
Keep in mind that although the example we used was with Entry Elements, anything that’s an Element can potentially have relations to other Elements.
For example, you might have added Category fields to your Assets. These too are things that are relations, can be eager loaded, for example:
{% set asset = entry.blogImage.with(['someCategory']).first() %}
You can even Eager Load Nested Sets of Elements like this:
{% set entries = craft.entries ({
section: 'blog',
with: ['blogImage.someCategory', 'blogCategory', 'blogTags'],
order: 'postDate desc',
limit: limitBlogs,
relatedTo: relatedBlogs,
}) %}
{% for entry in entries %}
The “dot syntax” 'blogImage.someCategory' tells Craft to Eager Load the blogImage Asset from the Entry, and while it’s at it, also Eager Load the someCategory Category from the Asset.
Most everything in Craft is an Element; you can Eager Load any Element into any Element, even nested Elements!
This ends up being a pretty powerful way to design your backend conveniently with nested Elements, but not suffer a performance penalty on the frontend when you go to load them.
Just tell Craft what you want Eager Loaded — no matter how deeply nested — and away you go!
What About Auto-Injected Entry’s?
One nice thing that Craft does for you is it auto-injects an entry or category variable for Entry or Category Templates. It’s a nice convenience, because then you can just access the entry variable without worrying about parsing the URL, and loading the appropriate Entry.
The only downside is because the Entry has already been loaded, we don’t get a chance to tell Craft what we want Eager Loaded with that Entry.
Remember, though, that our Entry will get preloaded with the ElementCriteriaModel for any of the Elements that are lazy loaded. So we can just do something like we did in the first example to eager load Elements that are in our entry:
{% set blocks = entry.blogContent.with(['image:image']) %}
{% for block in blocks %}
What we’re doing is asking Craft to load all of our blogContent Matrix Block Elements with the field image for any Matrix Block of the type image.
This is certainly better than lazy loading the image Asset every time through the loop, but it can end up being a little verbose in your templates, especially if you have a whole lot of Elements in your Entries that you have to explicitly Eager Load.
So I wrote a small plugin Eager Beaver for Craft 2.6.x & Eager Beaver for Craft 3.x. This plugin just lets you use the same Eager Loading syntax for already loaded entry (or whatever) Elements that you use for Elements that you load yourself.
For example, this is what I use on _entry.twig template for each Blog entry:
{% do eagerLoadElements(entry, [
'author.userPicture',
'blogCategory',
'blogImage',
'blogContent.image:image'
]) %}
The first parameter to eagerLoadElements() is just the Element we want to eager load things into, and the second parameter is the same with that you use for normal Eager-Loading Elements.
I prefer Eager Loading everything in one fell swoop this way, rather than doing it à la carte throughout my templates. But either method works, use whatever you prefer.
Note that on Craft 3, you can do the exact same thing that the Eager Beaver plugin does by using craft.app.elements.eagerLoadElements:
{% do craft.app.elements.eagerLoadElements(
className(entry),
[entry],
['assetsField', 'categoriesField.assetField', 'matrixField.blockType:assetsField']
) %}
The first parameter is the class name of the element type we’re eager loading elements into (in this case, an entry). The second parameter is an array of elements we’re eager loading elements into (in this case, an array with just our entry in it). Finally, the third parameter is dot-notation of what elements we want to eager load. c.f.: eagerLoadElements()
What About the Element API?
Something that is getting more and more common these days is Craft being accessed via the Element API by frontend JavaScript or other services. This is exactly how we did things in the Autocomplete Search with the Element API & VueJS article.
Well, we can use Eager Loading in the Element API as well! We can just do something like this:
<?php
namespace Craft;
return [
'endpoints' => [
'api/search' => [
'elementType' => 'Entry',
'paginate' => false,
'criteria' => [
'section' => 'blog',
'with' => [
'blogCategory',
'blogTags'
],
'limit' => 9,
Note the with key; look familiar? Yep, we can eager load things here just like we can everywhere else.
It’s important not to overlook optimizing your API endpoints like this as well, especially as they become more and more commonly used.
Don’t Be Too Eager
We’ve focused on using caching to help with performance in the The Craft {% cache %} Tag In-Depth and the Static Page Caching with Craft CMS articles. It’s important to not use caching as a way to mask or varnish-over performance problems.
We should optimize our templates with things like Eager Loading before we add a caching layer on top to help with concurrency.
We don’t want our cache misses to bog down the website, and really, things like Eager Loading are pretty simple to do.
Go forth, and Eager Load!
Further Reading
If you want to be notified about new articles, follow nystudio107 on Twitter.
Copyright ©2020 nystudio107. Designed by nystudio107
Top comments (0)