Photo by Paweł Czerwiński on Unsplash
In all the time that I've been building and maintaining Rails apps, I've somehow never noticed that ActiveRecord attributes backed by columns with default values are nil
on create
.
I'm going to demonstrate a number of ways to avoid this bit of confusion, but first let's start with an example.
Magic Links and Missing Tokens
This is a real example from a recent Rails side project. I added a MagicLink
model and magic_links
table for implementing magic sign in links for authentication. A magic link is a short-lived link that gets emailed to a user which when clicked will sign the user into the app.
Here is the schema description for the magic_links
table in PostgreSQL.
\d magic_links
Table "public.magic_links"
Column | Type | Collation | Nullable | Default
-------------+--------------------------------+-----------+----------+------------------------------------
id | uuid | | not null | gen_random_uuid()
user_id | uuid | | |
token | uuid | | not null | gen_random_uuid()
expires_at | timestamp without time zone | | not null |
created_at | timestamp(6) without time zone | | not null |
updated_at | timestamp(6) without time zone | | not null |
If you scroll to the right on that code block, you'll see that both id
and token
have default values of gen_random_uuid()
. This is a function in Postgres that can produce a random (v4) UUID -- as a random, unique identifier it makes a great id
and a great magic sign in token
.
Here is the accompanying ActiveRecord model, though not a lot to see.
class MagicLink < ApplicationRecord
belongs_to :user
end
And lastly, here is what I get when I create a new MagicLink
record.
> magic_link = MagicLink.create(expires_at: 5.minutes.from_now, user: User.last)
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
TRANSACTION (0.1ms) BEGIN
MagicLink Create (0.4ms) INSERT INTO "magic_links" ("user_id", "expires_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["user_id", "f683477e-c284-4677-8804-3852fa82b799"], ["expires_at", "2021-01-19 00:54:01.490924"], ["created_at", "2021-01-19 00:49:01.492309"], ["updated_at", "2021-01-19 00:49:01.492309"]]
TRANSACTION (0.2ms) COMMIT
=> #<MagicLink:0x00007fcd0691eff8
id: "9fe2c468-fa11-4df3-9683-d11eb45aff06",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: nil,
expires_at: Tue, 19 Jan 2021 00:54:01.490924000 UTC +00:00,
created_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
used_at: nil>
The #create
call inserts a new record into the magic_links
table and produces an in-memory representation of that record which is an instance of MagicLink
. There are two things I want to note about the in-memory (ActiveRecord) object.
The
id
has been set to"9fe2c468-fa11-4df3-9683-d11eb45aff06"
. Notice in theinsert
statement right about it that no value was sent to the Postgres server forid
. Because of that, Postgres uses the default value. It invokesgen_random_uuid()
to produce the record'sid
.The
token
value appears to benil
. No value was passed fortoken
as part of theinsert
statement. Because of that, I can assure you that Postgres invokedgen_random_uuid()
for this field as well.
So, why is token
showing up as nil
in that MagicLink
object?
To answer that, let's look a little closer at the insert
statement that was produced by the #create
call.
INSERT INTO "magic_links"
("user_id", "expires_at", "created_at", "updated_at")
VALUES ($1, $2, $3, $4)
RETURNING "id"
[...]
That statement has a returning
clause which specifies exactly "id"
and nothing else. The result of ActiveRecord running that query will be:
- Inserting a new record into
magic_links
with the provided values and any necessary defaults. - Returning just the newly produced
id
value for that record and nothing else.
When ActiveRecord gets back the response from the Postgres server for that insert, it will take the new information (just the id
) and combine that with the info already embedded in the MagicLink
object. The token
value was not returned by the insert
statement, so ActiveRecord doesn't know what it is. That's why token
is nil
.
So, what can we do about this?
A Variety of Solutions
There are several ways of getting at these default values. They all have tradeoffs, so you'll want to weigh those against your particular scenario.
1. Reload
Perhaps the first and most straightforward solution is to reload the record.
> magic_link.reload
MagicLink Load (0.4ms) SELECT "magic_links".* FROM "magic_links" WHERE "magic_links"."id" = $1 LIMIT $2 [["id", "9fe2c468-fa11-4df3-9683-d11eb45aff06"], ["LIMIT", 1]]
=> #<MagicLink:0x00007fcd0691eff8
id: "9fe2c468-fa11-4df3-9683-d11eb45aff06",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: "b90427cb-ab54-4e08-97dd-f8e02aa2820c",
expires_at: Tue, 19 Jan 2021 00:54:01.490924000 UTC +00:00,
created_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
used_at: nil>
This will make a (wasteful?) roundtrip to the database, in addition to the initial insert
to fetch all the up-to-date version of this object, all defaults included.
2. Produce the Default Value in Rails-Land
Most default values can be produced on the Rails side of things at the time of the create call.
> magic_link = MagicLink.create(
expires_at: 5.minutes.from_now,
user: User.last,
token: SecureRandom.uuid
)
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
TRANSACTION (0.1ms) BEGIN
MagicLink Create (0.4ms) INSERT INTO "magic_links" ("user_id", "token", "expires_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["user_id", "f683477e-c284-4677-8804-3852fa82b799"], ["token", "33fc275e-4235-4c59-9e8c-69c5ba622a3c"], ["expires_at", "2021-01-19 01:16:09.026531"], ["created_at", "2021-01-19 01:11:09.029372"], ["updated_at", "2021-01-19 01:11:09.029372"]]
TRANSACTION (0.3ms) COMMIT
=> #<MagicLink:0x00007fcd069aeba8
id: "43c4956c-87f0-40f3-933c-9a382057d464",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: "33fc275e-4235-4c59-9e8c-69c5ba622a3c",
expires_at: Tue, 19 Jan 2021 01:16:09.026531000 UTC +00:00,
created_at: Tue, 19 Jan 2021 01:11:09.029372000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 01:11:09.029372000 UTC +00:00,
used_at: nil>
This approach gets you back to a single round-trip with the database.
I find this solution irksome because I want to be able to rely on the database for things. Postgres provides many conveniences and affordances including the assurance of default values. I want to avoid duplicating that logic in my Rails code as much as possible.
That said, based on this particular magic link use case and my code maintenance preferences, this is my preferred solution.
3. Insert with overridden returning
ActiveRecord's #insert
accepts an optional returning
argument that allows you to override the default value of the returning
clause. Setting it as returning: '*'
adjust the insert
SQL statement to return everything in the result, not just the id
.
> result = MagicLink.insert(
{
expires_at: 5.minutes.from_now,
user_id: User.last.id,
created_at: Time.zone.now,
updated_at: Time.zone.now
}, returning: "*")
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
MagicLink Insert (0.8ms) INSERT INTO "magic_links" ("expires_at","user_id","created_at","updated_at") VALUES ('2021-01-19 01:20:00.912387', 'f683477e-c284-4677-8804-3852fa82b799', '2021-01-19 01:15:00.913344', '2021-01-19 01:15:00.913351') ON CONFLICT DO NOTHING RETURNING *
=> #<ActiveRecord::Result:0x00007fcd1722dbe8 ... >
> result.rows.first[result.columns.index("token")]
=> "92f64a62-8136-4243-9da9-f91c5786255f"
This is another single roundtrip solution. The drawbacks are:
The result of the call is an
ActiveRecord::Result
object instead of aMagicLink
object.The
#insert
call skips a bunch of the utilities of the#create
call including validations and callbacks. Notice also that I had to manually specify thecreated_at
andupdated_at
values.
4. Improved insert with overridden returning
The first drawback of the previous approach was that you get back an ActiveRecord::Result
object instead of a MagicLink
object. This can be solved by splatting and wrapping the whole thing in a MagicLink.new
call.
> MagicLink.new(
**MagicLink.insert(
{
expires_at: 5.minutes.from_now,
user_id: User.last.id,
created_at: Time.zone.now,
updated_at: Time.zone.now
}, returning: "*"
).first)
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
MagicLink Insert (0.9ms) INSERT INTO "magic_links" ("expires_at","user_id","created_at","updated_at") VALUES ('2021-01-19 01:33:27.064213', 'f683477e-c284-4677-8804-3852fa82b799', '2021-01-19 01:28:27.065227', '2021-01-19 01:28:27.065234') ON CONFLICT DO NOTHING RETURNING *
=> #<MagicLink:0x00007fcd06a25dc0
id: "bb17ad85-7e19-4cbf-9821-21f9c631243b",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: "4e99b551-60a2-4956-aba9-394ac2a1c9f0",
expires_at: Tue, 19 Jan 2021 01:33:27.064213000 UTC +00:00,
created_at: Tue, 19 Jan 2021 01:28:27.065227000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 01:28:27.065234000 UTC +00:00,
used_at: nil>
This still suffers from missing out on callbacks and validations and having to specify timestamps.
5. Improved+ insert with overridden returning
At this point I feel like we are starting to jump the shark. This solution adds a custom #create_returning
method that simulates a #create
call that gets to include a returning
override. It create a new object of the model's type with the given attributes, tries the #insert
call from previous solutions if the attributes are valid, and then returns that in-memory representation.
class ApplicationRecord < ActiveRecord::Base
self.abstract_class = true
def self.create_returning(attrs = {}, returning: "*")
record = new(**attrs)
record.attributes = insert(attrs, returning: returning).first if record.valid?
record
end
end
This approach gains back the validation check, but still leaves us without most of our callbacks (though I'd happily discourage the use of AR callbacks). I struggle to see myself using this approach because it requires abandoning standard ActiveRecord methods. That is hard enough as an active maintainer, and presents a nasty surprise for anyone new to the codebase.
Conclusion
I'd like to see Rails add an optional returning
argument to the #create
method. Until then, I'll probably opt for solution 2 or 1. If 2 isn't feasible for some reason and the second roundtrip required by 1 is untenable, I'd probably opt for some version of 3 with the details buried in a domain object or service object.
If you enjoyed this post, join my newsletter to get a direct line on more stuff like this.
Huge shoutout to Dillon Hafer who had a long conversation with me that led to this post and was the inspiration behind many of these solutions.
References:
Top comments (2)
Awesome post.
Excellent post, Josh! Thanks for clearly explaining this problem and your solutions.