Filters in Rails with Query Object Pattern

What will you learn?

Use the Query Object Pattern to implement a sustainable filter logic for your models and controllers. Connect your filter logic to your views by building a reusable filter component.

Introduction

Filtering is the process of refining a data set by excluding data according to certain criteria. Filters are frequently used to search for information in large datasets efficiently and effectively.

A filter component is an accessibility tool. It increases the accessibility and visibility of our data by providing a set of predefined options to select information. Designwise it consists of a form with multiple input fields, each corresponding to a filter condition. Filters are used in combination with lists and tables. When a filter form is submitted, the associated list or table is updated, displaying results that match the submitted filter form condition values.

It is important to complement our lists and tables with filters to make it easier for users to navigate through them. Implementing filters in your application will be a recurring issue. Each filter will vary depending on multiple aspects (the data model you work with, the list or table attached to the filter, the user needs, etc.). Thus, it is important to implement a filter logic that is easy to reuse in multiple contexts and with different sets of conditions.

The Basics

How to filter data?

From an implementation perspective, filtering consists of performing multiple custom queries to fetch data by different filter conditions.

Rails ApplicationRecord model classes count with the ActiveRecord query interface, consisting of a set of class methods to fetch records from the database. They work as an abstraction layer to build raw SQL queries easily. We can use these interface methods to implement our custom queries.

class Document < ApplicationRecord
  # (!) Filter methods are class methods too!

  # Filter document by title
  def self.filter_by_title(title)
    where("title ILIKE ?", "%#{title}%")
  end

  # Filter document by author
  def self.filter_by_author(name)
    joins(:author).where("author.name ILIKE ?", "%#{name}%")
  end
end

# SQL query generated by the query interface for one of our filters
Document.filter_by_title("My book").to_sql
# returns: "SELECT \"documents\".* FROM \"documents\" WHERE (title ILIKE '%My book%')"

Query interface methods return a relation. Relations also have access to query interface methods. This allows chaining multiple query method invocations to assemble complex SQL queries easily. To create our filter query, which will include all our custom queries, we just need to chain all our custom queries:

# Our document filter
...
  def self.filter(title, author_name)
    filter_by_title(title).filter_by_author(author_name)
  end
...

This query will always filter documents by their title and author. However, filter conditions are usually optional: when a filter value is missing, its respective condition is not considered for filtering results (for example, if I do not pass a title to the document filter, it should only filter results by author).

We could update our custom query methods, wrapping them in an if statement, so only when the filter value is present is the relation returned. However, returning a nil value instead of a relation will break the chain of custom queries.

The solution to this problem is using the scope class method, also part of the query interface. This method defines new model class methods for retrieving and querying data, such as the custom queries we previously defined. However, scope queries are lenient to nil values, always returning a relation. We can use scopes to make our custom queries conditional while preserving their chain ability.

class Document < ApplicationRecord
  ...
  scope :by_title, ->(title) { where("title ILIKE ?", "%#{title}%") if title.present? }
  scope :by_author, ->(name) { joins(:author).where("author.name ILIKE ?", "%#{name}%") if name.present? }
  ...

  def self.filter(title, author)
    by_title(title).by_author(author)
  end
end

# This will always return a relation
Document.filter(nil, nil)
Document.filter("My document", nil)
Document.filter(nil, "Leonardo Dantes")

Even though we come up with a nice way to define our custom queries and implement our filter query with them, our current approach presents multiple design flaws that need to be addressed:

• Models' main responsibilities are exchanging data with the database and implementing parts of the business logic. Filters, on the other hand, answer an accessibility problem and thus are not related to model responsibilities. Models should not be responsible for handling our filter logic.
• Filtering is a model-agnostic logic; all models filter results similarly. Placing our filter method in our model implies that each model must define this filter logic, which is redundant.
• The definition of custom query scopes in model classes will lead to big model classes overloaded with filter queries.

The Query Object Pattern

The Query Object Pattern is a Design Pattern. It consists of defining a specialized object to encapsulate all information required to perform a specific data query logic (in our case, filter queries, but it can be for any other queries, such as sort queries, for example).

We will use this pattern for our solution, refactoring all our scopes and filter logic to a specialized object responsible for assembling our filter SQL queries.

Delegating our filter logic to a query object solves all the problems previously mentioned:

• Our models do not need to know how to filter records; the query object will do. This reduces our models' responsibilities and removes all the filter-related code from them.
• Our query objects can share a common filter logic, making our system less redundant.

In the next lines, we will explain how to implement the Query Object Pattern for a simple scenario.

The Problem

We have an application to write Documents. Documents can be organized in Projects (a Project has zero or many Documents, and a Document can belong to a Project). We want to allow users to filter both Documents and Projects by different criteria.

The Solution

Model

We already learned how we can define optional filter queries with scopes. Since all our filter custom queries should be optional, we must add a conditional clause to all of them. To avoid this redundancy, we will define our own filter_scope “scope“ method instead, which will only chain a custom query to the filter query when its value is present.

# model/concerns/filter_scopeable.rb

module FilterScopeable
  extend ActiveSupport::Concern

  def filter_scope(name, block)
    define_method(name) do |filter_value|
      return self if filter_value.blank?

      instance_exec(filter_value, &block)
    end
  end
end

This approach allows us to define a common filter logic for custom queries and share it across all our filtrable models.

Let’s start implementing our query object by moving our scopes out of the model. These methods only work in the context of their respective model class. To extract and use them, we will extend the default model class scope with our filter scopes via the extending method. This method requires a known scope (our class model) and a module with methods to extend the scope. Let’s define our first filter query object with that information:

# /models/concerns/filters/document_filter_proxy.rb
module Filters
  module DocumentFilterScopes
    extend FilterScopeable

    # We define scopes with out new method
    filter_scope :name, ->(name) { where("name ILIKE ?", "%#{name}%") }
    filter_scope :status, ->(status) { where(status:) }
  end

  class DocumentFilterProxy < FilterProxy
    def self.query_scope = Document
    def self.filter_scopes_module = Filters::DocumentFilterScopes
  end
end

Finally, let's create our FilterProxy class. It will define the logic to assemble the filter query for our DocumentFilterProxy and all other future filter proxies:

# /models/concerns/filters/filter_proxy.rb
module Filters
  class FilterProxy
    extend FilterScopeable

    class << self
      # Model Class whose scope will be extended with our filter scopes module
      def query_scope
        raise "Class #{name} does not define query_scope class method."
      end

      def filter_scopes_module
        raise "Class #{name} does not define filter_scopes_module class method."
      end

      def filter_by(**filters)
        # extend model class scope with filter methods
        extended_scope = query_scope.extending(filter_scopes_module)

        # The payload for filters will be a hash. Each key will have the
        # name of a filter scope. We will map each key value pair to its
        # respective filter scope.
        filters.each do |filter_scope, filter_value|
          if filter_value.present? && extended_scope.respond_to?(filter_scope)
            extended_scope = extended_scope.send(filter_scope, filter_value)
          end
        end

        # Final relation with all filter scopes from +filters+ payload
        extended_scope
      end
    end
  end
end

Our solution is very simple, easy to maintain, and to scale. All our query objects share a common filtering logic, and the set of filter queries is now isolated in a separate module. If in the future, you want to create filters for a new model, you can do it by simply creating a new module for your queries and a FilterProxy class with a query_scope and a filter_scopes_module.

# /models/concerns/filters/project_filter_proxy.rb
module Filters
  module ProjectFilterScopes
    extend FilterScopeable

    ...
  end

  class ProjectFilterProxy < FilterProxy
    def self.query_scope = Project
    def self.filter_scopes_module = Filters::ProjectFilterScopes
  end
end

Finally, let's integrate our proxy filter logic into our models. We will expand our models' default ActiveRecord query interface with a new filter_by class method to provide an easy way to filter records (MyModel.filter_by(...)).

Since this interface will be the same for all filterable models, we will define it in a concern:

# /models/concerns/filterable_model
module FilterableModel
  extend ActiveSupport::Concern

  def filter_proxy
    raise "
      Model #{name} including FilterableModel concern requires filter_proxy method to be defined.
      Method should return filter proxy class associated to model.
    "
  end

  delegate :filter_by, to: :filter_proxy
end

# /models/project
class Project < ApplicationRecord
  extend FilterableModel

  class << self
    def filter_proxy = Filters::ProjectFilterProxy
  end
end

# To filter projects now we can do
Project.filter_by(name: "My Personal Project")

Now each model expanding the FilterableModel concern will be able to use the filter_by method. This method will also be available in all our filterable model scopes (for example, Document.all.filter_by), which again will allow us to assemble our filter queries with other queries to create more complex queries easily:

Project.all.includes(:documents).filter_by(name: "My Blog").sort_by(name: :asc)

Controller

Controllers handle HTTP requests and return the proper output in response. Our controllers are responsible for defining the interface to filter records, which consists of defining the filter parameters. Each filterable controller will have its own set of filter parameters, but the logic to define and use them to fetch the results will be the same. Thus we can encapsulate it in a concern for better reusability:

# app/controllers/concerns/filterable_controller.rb
module FilterableController
  extend ActiveSupport::Concern

  def filter(scope, filters = filter_params)
    unless scope.respond_to?(:filter_by)
      raise "
        Controller #{self.class.name} tried to filter a scope of type #{scope.class.name}.
        Scope class does not extend FilterProxy interface.
      "
    end

    scope.filter_by(**filters)
  end

  def filter_params
    raise "FilterableModel controller #{self.class.name} does not define filter_params method."
  end

  included do
    helper_method :filter_params
  end
end

This concern defines a filter method that requires a scope with a filter_by method and a set of filters formatted as a Hash. Each key-value pair in the Hash will represent the name of the filter scope and the value to filter by, respectively.

Usually, controllers work with a unique set of filter_params. However, this is not always the case. For more flexibility using multiple filter parameters, the filter method admits an optional filters parameter.

Last, we defined our filter_params as a helper method. The idea is to make each request filter parameters accessible from our filterable controller views. Thanks to it, our filter components can fetch these parameters and update the form fields according to them. If you define your own filter parameters method, make sure also to declare it as a helper.

Now we can include this concern in our controllers and update our actions with filters.

# app/controllers/projects_controller.rb

class ProjectsController < ApplicationController
  ...
  include FilterableController

  # GET /projects
  def index
    @projects = filter(Project.all)
  end
  ...
  private
  ...
  def filter_params
    {
      name: params["name"],
      description: params["description"],
    }
  end
  ...
end

View

As mentioned at the beginning of this post, a filter component is just a form in which each input field represents a filter condition. Implementing a simple form is a straightforward task, thanks to Rails form_for helper method. There are only two implementation issues to be addressed here.

The first is loading the form with fields filled with the user request filter parameters. Thanks to the previously defined filter_params helper, we can fetch user-request filter parameters and inject them into our filter input fields.

The second is to specify the URL our form filter points to (where the form payload will be submitted). This can be done in two ways.

One is specifying the URL via named URL helpers, so we explicitly set the URL.

The other consist of using the URL helper method url_for. Filter requests attach the filter parameters as query strings to the URL, using the same URL with different parameters to fetch different results on the same page.

The url_for method returns the URL of the user request. Using this as our form URL instead of a specific route name helper, we ensure that the form will always work in all routes it is rendered.

# Solution implemented with ViewComponents & TailwindCSS

module Projects
  class IndexComponent::FilterComponent < ApplicationComponent
    # Tell the component filter_params is a helper and not a component method
    delegate :filter_params, to: :helpers

    def call
      form_with(url: url_for, method: :get, class: "px-8 py-6 inline-flex gap-6") do |form|
        concat(
          tag.div(class: "inline-flex gap-4") do
            concat(form.label(:name, "Name"))
            concat(
              form.text_field(
                :name,
                value: filter_params[:name],
                class: "border-b-2 focus:border-slate-400 focus:outline-none"
              )
            )
          end
        )
        concat(
          tag.div(class: "inline-flex gap-4") do
            concat(form.label(:description, "Description"))
            concat(
              form.text_field(
                :description,
                value: filter_params[:description],
                class: "border-b-2 focus:border-slate-400 focus:outline-none"
              )
            )
          end
        )
        concat(
          form.submit
        )
      end
    end
  end
end

Tests

We must ensure our filter scopes return database records that match the filter criteria and that FilterableControllers process the filter_parameters, when present, and return a response with the corresponding filtered results.

To test our filter scopes, we can create a TestCase per FilterProxy and test per proxy filter scope. Each test will define a filter payload, create a set of records matching the payload and invoke the filter_by method to assert the returned records match the created ones.

# test/models/concerns/project_filter_proxy_test.rb
require "test_helper"

class ProjectFilterProxyTest < ActiveSupport::TestCase
  test ".name filters documents by name" do
    filters = { name: "Test Project #{Time.now}" }

    projects = FactoryBot.create_list(:project, 3, **filters)

    results = ::Filters::ProjectFilterProxy.filter_by(filters)

    assert_equal documents.pluck(:id).sort, results.pluck(:id).sort
  end

  test ".description filters documents by status" do
    filters = { description: "Test project description #{Time.now}" }

    documents = FactoryBot.create_list(:project, 3, **filters)

    results = ::Filters::ProjectFilterProxy.filter_by(filters)

    assert_equal documents.pluck(:id).sort, results.pluck(:id).sort
  end
end

To test our FilterableControllers we will add a new test to their corresponding IntegrationTests to test that requests to actions which filter parameters return filtered results. We need to define a filter payload to pass to our controller request and the corresponding filter proxy and compare that the results they return are the same. This time we will use fixtures for efficiency.

# test/controllers/projects_controller_test.rb
require "controller_test_helper"

class ProjectsControllerTest < ControllerTestHelper
  ...
  test "GET index filters projects" do
    filter_params = { name: projects(:recipes_book).name }

    filtered_projects = Project.filter_by(filter_params)

    get projects_path

    # https://github.com/rails/rails-controller-testing
    result = assigns(:projects)

    assert_equal result.pluck(:id).sort, filtered_projects.pluck(:id).sort
  end
  ...
end

Finally, we must ensure our FilterProxy assembles filter queries as expected, combining all individual filter scope SQL clauses into a single query, ignoring all scopes with a blank value. Instead of testing arbitrarily complex filter request results, we will check the composition of the resultant SQL queries, which is more aligned with the behavior we want to test.

We already mentioned how the to_sql method returns the raw SQL associated with a scope. We will use it to check the resultant filter_by query contains all the filter scope WHERE clauses from the filter scopes present in the test payload, and that there is no trace of filter scopes with empty values.

require "test_helper"

class FilterProxyTest < ActiveSupport::TestCase

  class InvalidFilterProxy < ::Filters::FilterProxy

  end

def setup

    @proxy ||= ::Filters::DocumentFilterProxy

    @proxy_model_class = Document

    @proxy_scopes_module = "Filters::DocumentFilterScopes".constantize

    @proxy_scope = @proxy_model_class.extending(@proxy_scopes_module)

  end

test "#filter_by combines all filter scopes into a single query" do

    result = @proxy.filter_by({ name: "a name", status: :idea })

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.name("a name").to_sql))

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.status(:idea).to_sql))

  end

test "#filter_by ignores scopes with empty/blank values" do

    result = @proxy.filter_by({ name: "", status: :idea })

    assert_not result.to_sql.include?("name")

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.status(:idea).to_sql))

  end

test "#filter_by ignores undefined scopes" do

    result = @proxy.filter_by({ invalid_attribute: "invalid value", status: :idea })

    assert_not result.to_sql.include?("invalid_attribute")

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.status(:idea).to_sql))

  end

private

# Extracts filter match condition from SQL query, discarding the SELECT part

  def sql_filter_clause(query)

    query.match(/WHERE (.*)/).captures[0]

  end

end

Conclusion

In this post, we have explained how to create a sustainable filter logic via the Query Object pattern.

This pattern allows us to isolate common patterns to fetch data present in our business logic in sustainable modules that are easy to maintain and scale.

We illustrated the pattern's versatility with the case scenario of the filters, but you can find many other patterns in your business logic that can be strong candidates to be implemented with this solution (for example, sorting).

And that's it! I hope you found this article helpful. Please feel free to share any feedback or opinion in the comments, and thanks for reading it.