For more in-depth content on web development check out CodeCurious
Surely you've heard of Hotwire
by now and perhaps thought "cute name but what the heck is that? and do I need to care about it?"
I went about answering that question for myself and wrote down what I learned. If you have 10 minutes and are curious about Hotwire and Turbo, read on.
You do not need to be Ruby on Rails developer to read this. The ideas around Hotwire and server-side rendering are making waves across different web development stacks - PHP/Laravel, Elixir/Pheonix, Python/Dyango have equivalent tech that implements Hotwire. Even React has added support for server-side rendering it seems.
The following writing is for wrapping our heads around the concepts. So you can sit back with your coffee/tea and just read (not much code to type-along).
What is Hotwire and What is Turbo
HTML-over-the-wire or Hotwire is a technique for building web applications. It's not one technology, it's an umbrella term for Turbo, StimulusJS, and Strada (in the Rails world). Turbo is the actual technology that makes the Hotwire approach possible. Turbo is the actual code.
Yes Turbo is created by the team behind Ruby on Rails and it's an evolution of Turbolinks with bunch of other functionality added. But Turbo can be adapted to other languages and backend frameworks (and already has!). This Turbo thing is not Rails specific.
So what does the Turbo code do? What problem does it solve? How does it help us?
Officially "Turbo is a collection of techniques for creating fast, progressively enhanced web applications without using much JavaScript. All the logic lives on the server, and the browser deals just with the final HTML."
This approach can, of course, be contrasted with the Single Page Application (SPA) approach. Where we'd get JSON from the server, use client-side JavaScript (like a boatload of JS involving frameworks and libraries that need to keep state and do routing) to eventually transform that JSON data into DOM updates.
The promise of Turbo is writing less JavaScript and more of your preferred backend language - Ruby or PHP or Python or what have you. All of your logic for interacting with your domain model and other business logic can live in one place, on your server. The other promise is to sidestep the complexity of full-fledge client-side JavaScript frameworks and the associated tooling (cough webpack). The only JavaScript you need is Turbo.js
Of course we'd want to get these benefits without sacrificing any of the speed or responsiveness associated with modern web applications with rich user experiences. And that is just what hotwire.dev promises.
If we go all the back to AJAX, the official documentation (and by that I mean wikipedia) says
"With Ajax, web applications can send and retrieve data from a server asynchronously (in the background) without interfering with the display and behavior of the existing page...Ajax allows web pages and, by extension, web applications, to change content dynamically without the need to reload the entire page."
So in other words, the big promise of AJAX was to 'update content without full-page reload'. So that it feels smooth and fast for the user. That is still what we are after. With Turbo we can do this in a simpler, more elegant way and automatically get the benefits by including Turbo.js and following some convestions (without writing much if any JavaScript!).
Turbo does its job with the following components:
Turbo Drive speeds up links and form submissions by not requiring page reloads.
Turbo Frames decompose pages into independent contexts, which scope navigation and can be lazily loaded.
Turbo Streams deliver page changes over WebSocket, SSE or in response to form submissions using just HTML and a set of CRUD-like actions.
[aside] SSE is server-sent events. The major difference between WebSockets and Server-Sent Events is that WebSockets are bidirectional (allowing communication between the client and the server) while SSEs are one-directional (only allowing the client to receive data from the server).
Turbo Native lets your majestic monolith form the center of your native iOS and Android apps, with seamless transitions between web and native sections.
Turbo 7.1 was released November 24, 2021 (keeping the Turbolinks versions) and Hotwire ships by default in Rails 7, released December 15, 2022. Let's see how each of the 4 Turbo components work. We also cover some constraints and nuances to keep in mind when building applications with Turbo.
Turbo Drive: Navigate Within a Persistent Process
Turbo Drive gives us the same speed of SPA by using the same persistent process. The persistent process is managed by Turbo (there is no client-side router, no state to carefully manage).
Following Links
This works by intercepting all clicks on <a href>
links to the same domain. When you click an eligible link, Turbo Drive prevents the browser from following it, changes the browser’s URL using the History API, requests the new page using fetch, and then renders the HTML response.
[aside] History API - allows manipulation of the browser session history, that is the pages visited in the tab or frame that the current page is loaded in. I see the pushState()
function is part of this API (which is what pjax
used to update browser URL, etc.)
Form Submission
During rendering, Turbo Drive replaces the current <body>
element outright and merges the contents of the <head>
element. The JavaScript window
and document
objects, and the <html>
element, persist from one rendering to the next.
The speed with Turbo happens free just by following a few conventions. Though it is possible to interact directly with Turbo Drive to control how visits happen or hook into the lifecycle of the request.
Turbo Frames: Decompose Complex Pages
Turbo Frames is a way to scope independent segments of a page inside a turbo-frame
element such that they can be lazily loaded and their navigation be scoped. Scoped navigation means all interaction within a frame (e.g. clicking links, submitting forms) happen within that frame, keeping rest of the page from reloading.
I heard DHH say on Remote Ruby interview that the origins of Turbo Frames are based on making mobile stuff work for Hey email.
[aside] the use case for which this is designed, reminds me of the 'aync dashboard' work I did to speed up the Dashboard of our web app at Castlight. Loading bits of the page 'later', in parallel instead of in-band with the initial page load. I recall using .js.erb files I think and looking at the network tab a lot.
Scoped Navigation
<turbo-frame id="new_message">
<form action="/messages" method="post">
...
</form>
</turbo-frame>
When we submit the form above, Turbo extracts the matching turbo-frame
element with the id
of new_messages
from the HTML response and swaps its content into the existing new_message
frame element. The rest of the page stays as it was.
Deferred Loading
<turbo-frame id="messages" src="/messages">
<p>This message will be replaced by the response from /messages.</p>
</turbo-frame>
To differ loading we add a src
attribute to the turbo-frame
element. The HTML response from the URL value will be used to automatically load content into the matching frame id
.
Are Turbo Frames like iframes then?
They sound like iframes but no Turbo frames are part of the same DOM, styled by the same CSS, and part of the same JavaScript context. So they don't suffer the weirdness associated with iframes.
Other advantages of Turbo Frames:
- Efficient caching - each segment is cached independently, so you get longer-lived caches with fewer dependent keys.
- Parallelized execution - each defer-loaded frame is generated by its own HTTP request/response and handled by a separate process. So different segments on a page load in parallel without having to manage the process.
- Ready for mobile - Each segment can appear in native sheets and screens without alteration, since they all have independent URLs.
Turbo Streams: Deliver Live Page Changes
While Turbo Frames give us partial page updates in response to direct interactions within a single frame (link clicks, form submits), Turbo Streams let us change any part of the page in response to updates from a WebSocket connection (or SSE).
[aside] Streams is a conceptual continuation of what was first called RJS and then SJR.
- The big idea around RJS, JavaScript written in Ruby, from that 2006 post announcing Rails 1.1 is that you don't have to write JavaScript to Ajaxify things in Rails, you can write Ruby. (So yeah, we are still after the same thing!).
- The idea behind Server-generated Javascript Response SJR from this post from 2013 is pretty reminiscent of what we have with Turbo. The flow is something like this - a form is submitted via AJAX request, server generates a JavaScript response that includes an updated HTML template, client evaluates the JavaScript returned by the server, which then updates the DOM. We can see how Streams is a conceptual continuation of this flow with eliminating more JavaScript. RJS was a poor man's CoffeeScript and it turned people away from the concept of server-generated JavaScript, but in 2013 rails re-committed to SJR while leaving behind RJS. The last sentence of that 2013 post says:
"The combination of Russian Doll-caching, Turbolinks, and SJR is an incredibly powerful cocktail for making fast, modern, and beautifully coded web applications. Enjoy!"
Unlike RJS and SJR it's not possible to send custom JavaScript as part of Turbo Stream actions, by design! Turbo focuses on sending HTML and updating the DOM and then if needed we can connect additional behavior using Stimulus actions and lifecycle callbacks.
How It Works?
Turbo Streams introduces a <turbo-stream>
element with an action
and a target
attribute. The actions could be append, prepend, replace, update, remove, before, after. We include the HTML to insert or replace in a template
tag and Turbo does the rest.
[aside] HTML template tag is a way for holding HTML that is not to be rendered immediately when a page is loaded but may be added to the page later using JavaScript.
Advantages of Turbo Streams
- Reuse server-side templates - same templates that created the first-load page are used to generate live page updates (in practice some CSS is needed to show/hide certain elements in the two cases).
- HTML over the wire - don't need any client-side JavaScript (other than Turbo.js) and save all the effort it takes to fetch JSON and turn it into HTML.
- Simpler control flow - clear to follow what happens in response to WebSocket or SSE by looking at the HTML to be changed. No indirection with client-side routing, event bubbling and such.
Turbo Native: Hybrid apps for iOS and Android
I am not a mobile developer. But here are some points I took away from reading about Turbo Native.
- Ideal for building hybrid apps where server-rendered HTML can be used to get baseline coverage of your app's functionality in a native wrapper. Along with a few native screens that can really benefit from the high fidelity.
- Going hybrid allows you the freedom to upgrade your app without going through the slow app store release process. Anything done in HTML can be changed in your web application and be instantly available to users.
- Turbo Native is not a framework that abstracts native APIs. It also does not try to create native code that's sharable across platforms. (In other words, you still need native devs for iOS and Android, but those devs just have less work do). The part that's sharable is the HTML that's rendered server-side.
So that covers the main concepts around Turbo Drive, Turbo Frames and Turbo Streams. Let's learn about some nuances to keep in mind when building applications with Turbo.
Also by the way, while Turbo is not Rails specific, turbo-rails gem is the reference implementation of Turbo for Ruby on Rails. The Turbo documentation says you don't need a backend framework to use Turbo. (I don't quite get that tbh, doens't the backend need to return HTML with the appropriate elements <turbo-stream>
, <turbo-frame>
along with the correct ids
to replace the right things on the page.)
Constraints and Nuances of Building Turbo Applications
Since there is no full page load, the JavaScript window
and document
objects retain their sate across page changes (any other objects we leave in memory will stay in memory). We can't rely of full-page reload to reset our environment, our application is a persistent, long-running process in the browser. Below are things we need to be aware of to design our application to gracefully handle this constraint:
1. Script Elements
When navigating with Turbo, <script>
elements in the <head>
are okay, Turbo Drive appends them to the current <head>
and the browser loads and evaluates them.
<script>
elements in the <body>
are not good. To install behavior, or to perform more complex operations when the page changes, avoid script elements and use the turbo:load
event instead.
For <script>
elements we don't want Turbo to evaluate after rendering, annotate them with data-turbo-eval="false"
. The browser will still evaluate those scripts on the initial page load.
For loading our application's JavaScript bundle, do this in the <head>
tag always. Also if we add a fingerprint to each script and data-turbo-track="reload"
attribute, Turbo can force a full page reload when we deploy a new JavaScript bundle.
2. Caching
Let's define some terms like Restoration visits and Application visits first.
- Restoration visits are when you navigate with the Browser's back or forward button. Turbo Drive renders a copy of the page from the cache if possible. (Drive also saves the scroll position of each page before navigating away and automatically returns to this saved position. nice)
- Application visits are the onces initiated by clicking a Turbo Drive-enabled link or programmatically calling
Turbo.visit(location)
. Always issues a network request. These result in a change to browser's history, Turbo Drives pushes a new entry onto the browser's history stack usinghistory.pushState
.data-turbo-action
is the attribute to decide what type of visit it is, default isadvance
, it could also bereplace
. - When Turbo Drive saves a copy of the current page to cache, it uses
cloneNode(true)
, which apparently means attached event listeners and data are discarded. (cloneNode
clones the DOM node)
Preparing the Page to be cached - we can listen to the turbo:before-cache
event with document.addEventListener
if we need to do things to the document before Turbo caches it. For example, reset forms, collapse UI elements, etc. so the page is ready to be displayed again.
Detecting when a preview is visible - Turbo Drive adds a data-turbo-preview
attribute to the <html>
element when it displays a page from cache. We can check for the presence of this if we want to enable/disable behavior.
Opting out of caching - We can control caching behavior on a per-page basis by including a meta
element in the page's head
. Like this:
<head>
...
<meta name="turbo-cache-control" content="no-cache">
</head>
3. Installing JavaScript Behavior
Since the usual window.onload
, DOMContentLoadeded
, or JQuery ready
events will only fire after initial page load, we need a strategy for installing JavaScript behavior on Turbo page loads. There are two options:
Observing Navigation Events
There is an event, turbo:load
that fires after the initial page load and again after every Turbo Drive visit. We can use this like this:
document.addEventListener("turbo:load", function() {
// ...
})
- avoid using
turbo:load
event to add other event listeners directly to elements on the page body. Instead use event delegation to register event listeners once ondocument
orwindow
.
Attaching Behavior with Stimulus
New DOM elements can appear on the page at any time from frame navigation, stream messages, client-side rendering, or Turbo Drive page loads. Stimulus with its lifecycle callbacks and conventions can handle all of these in a single place.
How Stimulus Works - it connects and disconnects its controller and event handlers whenever the document changes using the MutationObserver API. This allows it to handle all types of DOM updates.
4. Making Transformations Idempotent
The context here is performing client-side transformations to HTML received from the server. The example we can imagine is grouping a collection of elements by date using browser's knowledge of current time zone.
[aside] Making something, a function, idempotent means that regardless of how many times we apply the function to a given input the result will be the same as applying it just once. Meaning there are no more changes beyond its initial application.
We have to think about this because of caching. "Consider what happens if you’ve configured this function to run on turbo:load. When you navigate to the page, your function inserts date headers. Navigate away, and Turbo Drive saves a copy of the transformed page to its cache. Now press the Back button—Turbo Drive restores the page, fires turbo:load again, and your function inserts a second set of date headers."
We can solve this by detecting if the transformation is already present on the HTML before adding it. In this we'd check for the presence of a date divider. (We can also add a flag via data
attribute but that's less robust)
5. Persisting Elements Across Page Loads
The context here is that we may not want certain elements to change across page loads. Turbo Drive allows us to mark them permanent. The example we can imagine is a shopping cart icon with a counter. We don't want the counter to change to a previous/lower number if the user navigates with a back button after adding an item to the cart on the current page.
We do this by marking the counter element permanent.
<div id="cart-counter" data-turbo-permanent>1 item</div>
The HTML id is needed as Turbo Drive matches all permanent element by ID before rendering, and transfers them from the original page to the new page. Preserving data and event listeners.
Summary
That's all, that covers most of the concepts for understanding Turbo and Hotwire. It's all about updating things on the page in a way that feels fast and smooth to the user, without needing to write a lot of client-side Javascript.
HTML-over-the-wire or Hotwire is a concept, not a specific technology. The main idea is to send server-rendered HTML to the client, instead of JSON, leaving application logic to the server.
Turbo is the technology that implements Hotwire in the Rails world. And other backend stacks have their own version of Turbo. Turbo consists of Drive, Frames, and Streams.
Turbo Drive is an evolution of Turbolinks. When enabled, it intercepts link clicks and form submission, maintains a persistent process that leaves window
and document
unchanged, but replaces the <body>
element of the response.
Turbo Frames allow different sections of the page to be lazily loaded and elements on the page to be automatically scoped by using <turbo-frame>
element so that a response updates only the frame with a matching id
, leaving rest of the page untouched. Turbo Frames are similar to iframes
but different, in that they're part of the same DOM, styled by the same CSS, and part of the same JavaScript context.
Turbo Streams take things even further. They allow for partial page updates from not only user interactions (like form submission) but also in response to websocket and SSE. <turbo-stream>
elements support automatically changing the HTML of a target element by various operations like replace, append, prepend, remove, etc.
When building applications with Turbo we have to keep in mind that our application is a persistent, long-running process in the browser. We can't rely of full-page reload to reset our environment and have to approach a few things with care, like caching, script element, and installing JavaScript behavior, etc.
Finally, it's worth noting that Hotwire and Turbo approach is not suitable for all web applications. Sometimes the complexity of client-side rendering with a JavaScript SPA is needed for a high-fidelity user experience of text editor, for example. (such as the Trix editor from the Rails community). And yet, for many modern web applications it would be worthwhile to reach for Hotwire/Turbo for the conceptual compression of complexity offered.
If you made it all the way to the end, good job! 💜 Hope you learned something!
The above is a conceptual post around Hotwire and Turbo, meant for the job of wrapping our heads around what the big ideas are in the Hotwire approach. Check back for the accompanying 'code' post where we'll build an application using streams and frames and share the code step-by-step.
Top comments (2)
This is a great write up, thank you! And what a blast from the past. I haven't thought about
pjax
or*.js.erb
files in ages!Helpful and straightforward! Thanks for the explanation