Today, I’d like to share my journey while working for a client on implementing an efficient mailing system.
We'll see how I enabled my client to send 100, 1,000, and even 500,000 emails without impacting the performance of my Rails application.
Let's dive in now 🎉
Table of Contents
1. The Mail
2. V1: Deliver Now (up to 300 emails)
3. V2: Deliver Later (up to 100,000 emails)
4. V3: Perform All Later (up to +500,000 emails)
5. Other possible alternatives I explored
5.1. Monkey Patching Application Mailer
5.2. Using the Mail Provider’s Native Bulk Send
5.3. Using BCC
6. Conclusion
The Mail
For the project example, we'll start with a simple mailer, adding a method to send emails that simply forwards the arguments. Here's the code:
# app/mailers/application_mailer.rb
class ApplicationMailer < ActionMailer::Base
def generic_mail(subject:, body:, to:)
mail(subject: subject, body: body, to: to)
end
end
# Call the mailer like this:
ApplicationMailer.generic_mail(
subject: "Hello", body: "<p>How are you today?</p>", to: "you@gmail.com"
).deliver_now
The goal is to expose an interface to my admin so they can send emails to our entire user base. Here's an example of what the interface might look like:
The challenge is ensuring that the emails are sent promptly, ideally within 10 seconds.
Let's explore the different iterations of sending emails!
V1: Deliver Now (up to 300 emails)
The simplest way to send emails in Rails → deliver_now
User.pluck(:email).each do |email|
ApplicationMailer.generic_mail(
subject: "Hello", body: "<p>How are you today?</p>", to: email
).deliver_now
end
Initially, we only had a Rails application and a database. Given our small user base, using deliver_now
was acceptable.
The problem with deliver_now
is that it’s a blocking call. It waits for a response from the mail provider before moving to the next line of code.
Assuming each API call takes around 100ms, sending 100 emails would mean a 10-second wait on the interface. This solution is thus only suitable for testing scenarios with a minimal database.
V2: Deliver Later (up to 100,000 emails)
The logical next step to accelerate the application is to send emails within a Job. ActionMailer offers the deliver_later
method for this.
User.pluck(:email).each do |email|
ApplicationMailer.generic_mail(
subject: "Hello", body: "<p>How are you today?</p>", to: email
).deliver_later
end
deliver_later
enqueues a MailDeliveryJob
with all the mail information.
We were satisfied with this for a while. However, as our email volume increased, Redis started to be overwhelmed with write operations.
Despite Redis being very fast, sending around 10,000 emails led to slowdowns, likely due to our $3/month instance on Heroku.
Fortunately, I found an even more efficient solution for handling higher volumes of emails!
V3: Perform All Later (up to +500,000 emails)
Rails 7.1 introduced a powerful feature called ActiveJob.perform_all_later
, enabling bulk job enqueuing. The downside is that we need to move away from mail.deliver_later
and use a custom job class. Here's a simple implementation:
class SendEmailJob < ApplicationJob
queue_as :default
def perform(subject:, body:, to:)
ApplicationMailer.generic_mail(
subject: subject, body: body, to: to
).deliver_now
end
end
jobs = []
User.pluck(:email).each do |email|
jobs << SendEmailJob.new(
subject: "Hello", body: "<p>How are you today?</p>", to: email
)
end
ApplicationJob.perform_all_later(jobs)
With this implementation, we managed to send emails to 500,000 users in approximately 15 seconds. Pretty impressive!
Based on my benchmarks, this method could potentially handle sending 1,000,000 emails in about 40 seconds. Unfortunately, Heroku imposes a page timeout after 30 seconds, so another solution will be necessary when I will need to send a million.
Other possible alternatives I explored
Monkey Patching Application Mailer
Seeing the performance of ApplicationJob.perform_all_later
led me to consider monkey-patching ApplicationMailer
to implement a deliver_all_later
method. Here is my implementation:
class ApplicationMailer < ActionMailer::Base
class << self
def deliver_all_later(messages, options = {})
jobs = messages.map do |message|
message.delivery_handler.delivery_job.new(
message.delivery_handler.name, message.instance_variable_get(:@action).to_s, 'deliver_now', args: message.instance_variable_get(:@args)
).set(options)
end
ActiveJob.perform_all_later(jobs)
end
end
end
In theory, this seemed promising. However, my benchmarks showed inefficiencies beyond 100,000 emails due to memory bloat from handling many MessageDelivery
objects. Thus, it wasn’t the best path.
Using the Mail Provider’s Native Bulk Send
Some providers like Mailjet offer native bulk sending. This could be an option in the future if our email volume continues to grow, although it requires using their SDK over ApplicationMailer
, making the code less Rails-friendly.
Using BCC
An easy idea is to use BCC, sending the same email to many recipients. This would work if the email content is identical for everyone. However, my custom mail system interprets user-specific variables, so this wasn't viable for my use case.
Conclusion
That’s it, thanks for reading me so far. If you find a better way to send more and more mails I would love to hear from you. Don’t hesitate to comment if you want more details about my searches / benchmark.
Bye 🕺
Top comments (0)