I published a new gem, sidekiq-dry
aiming to tackle a variety of
common frustrations when it comes to Sidekiq jobs and their arguments.
Rationale
Sidekiq is among the most popular background job solutions. It's my
first choice for Ruby apps. The dry-rb family of gems is also
indispensable in non-trivial applications. What if we combined the two..
With sidekiq-dry
you may pass instances of Dry::Struct
as arguments
to your Sidekiq jobs. But why?
Prevent Type Ambiguity
Numerous times I've had to debug jobs which where failing due to being
enqueued with invalid arguments.
Example:
class SendInvitationEmailJob
include Sidekiq::Worker
def perform(user_id, invitee_email)
# code
end
end
SendInvitationEmailJob.perform_async(user.id, params[:invitee_email])
The problem with the above code is that if the user_id
is not an
Integer id or the invitee_email
is not a valid email String then
there's absolutely no chance that the enqueued job will complete
successfully. Of course Dry::Struct
is not to be used for validations,
there's dry-validate
for that, or ActiveModel
/ ActiveRecord
validations if you prefer. Giving more structure to your
background job arguments improves the system's robustness. Your objects
in transport through Redis
, as long as the job is enqueued, they are
guaranteed to have the expected structure when the job is performed.
The above example would be refactored to:
class SendInvitationEmailJob
include Sidekiq::Worker
def perform(params)
# code
end
end
class SendInvitationEmailJob::Params < Dry::Struct
attribute :user_id, Types::Strict::Integer
attribute :invitee_email, Types::Strict::String.constrained(format: /\A[\w+\-.]+@[a-z\d\-]+(\.[a-z]+)*\.[a-z]+\z/i)
end
job_params = SendInvitationEmailJob::Params.new(user_id: user.id, invitee_email: params[:invitee_email])
SendInvitationEmailJob.perform_async(job_params)
At this point you might ask, what if we passed in a Hash, instead of a Dry::Struct
?
Well, Hash arguments are deserialised with String keys which can lead to surprises.
Eliminate Positional Arguments
When your background job takes two or more positional arguments, it's
better to refactor it to take a single struct object with a
comprehensible name.
In the Rails world it's common to enqueue jobs with a record's id
.
There's nothing wrong with this pattern. However, in some cases, developers may define a
model blindly following the convention.
Documentation
By using Dry::Struct
arguments you'll be able to express constraints
straight in your code. Instead of documenting the types of each job argument,
which can easily become outdated, you can refer to the types of the attributes of the struct.
class Post < Dry::Struct
attribute :title, Types::Strict::String
attribute :tags, Types::Array.of(Types::Coercible::String).optional
attribute :status, Types::String.enum('draft', 'published', 'archived')
attribute :body, Types::String.constrained(min_size: 10, max_size: 10_000)
end
Arguably, in the example above, both types and constraints improve readability.
Versioning
Adding this gem does not break any existing jobs in your app.
It only works on jobs enqueued with Dry::Struct
objects.
Adding a new attribute to a parameter struct won't break already enqueued jobs.
It's trivial to version your structs using either a version
attribute:
class Coupons::ApplyCouponJob::Params < Dry::Struct
attribute :user_id, Types::Strict::Integer
attribute :coupon_code, Types::Strict::String
attribute :version, Types::Strict::String.default('1')
end
or versioned classes:
class Coupons::ApplyCouponJob::Params::V1 < Dry::Struct
attribute :user_id, Types::Strict::Integer
attribute :coupon_code, Types::Strict::String
end
Caveats
Job processing libraries compatible with Sidekiq, for example
exq, won't deserialise your Dry::Struct arguments. This is most likely an acceptable tradeoff.
The Gem
The gem is hosted on rubygems (link). It provides two Sidekiq
middlewares which serialise and deserialise instances of Dry::Struct
arguments in your jobs.
Installation
Add the gem in your Gemfile:
gem 'sidekiq-dry'
Configure Sidekiq
to use the middlewares of the gem:
# File: config/initializers/sidekiq.rb
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add Sidekiq::Dry::Client::SerializationMiddleware
end
end
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add Sidekiq::Dry::Server::DeserializationMiddleware
end
end
Further Reading
For other handy libraries and posts, subscribe to my Tefter Ruby & Rails list.
Top comments (1)
This is pretty useful, thank you for sharing! Nice to see some dry-rb fans out there :)