This post was originally published on my blog on December 31st, 2018.
Being new to Elixir and Phoenix, I spend quite some time in the projects' documentation. One thing that stood out for me recently is the first sentence of Phoenix's Plug documentation:
Plug lives at the heart of Phoenix’s HTTP layer and Phoenix puts Plug front and center.
So naturally, I felt compelled to take a deeper dive into Plug and understand it better. I hope the following article will help you out in understanding Plug.
What's Plug?
As the readme puts it, Plug is:
- A specification for composable modules between web applications
- Connection adapters for different web servers in the Erlang VM
But, what does this mean? Well, it basically states that Plug 1) defines the way you build web apps in Elixir and 2) it provides you with tools to write apps that are understood by web servers.
Let's take a dive and see what that means.
Web servers, yeehaw!
One of the most popular HTTP servers for Erlang is Cowboy. It is a small, fast and modern HTTP server for Erlang/OTP. If you were to write any web application in Elixir it will run on Cowboy, because the Elixir core team has built a Plug adapter for Cowboy, conveniently named plug_cowboy.
This means that if you include this package in your package, you will get the Elixir interface to talk to the Cowboy web server (and vice-versa). It means that you can send and receive requests and other stuff that web servers can do.
So why is this important?
Well, to understand Plug we need to understand how it works. Basically, using the adapter (plug_cowboy
), Plug can accept the connection request that comes in Cowboy and turn it into a meaningful struct, also known as Plug.Conn
.
This means that Plug uses plug_cowboy
to understand Cowboy's nitty-gritty details. By doing this Plug allows us to easily build handler functions and modules that can receive, handle and respond to requests.
Of course, the idea behind Plug is not to work only with Cowboy. If you look at this SO answer from José Valim (Elixir's BDFL) he clearly states "Plug is meant to be a generic adapter for different web servers. Currently we support just Cowboy but there is work to support others."
Enter Plug
Okay, now that we've scratched the surface of Cowboy and it's Plug adapter, let's look at Plug itself.
If you look at Plug's README, you will notice that there are two flavours of plugs, a function or a module.
The most minimal plug can be a function, it just takes a Plug.Conn
struct (that we will explore more later) and some options. The function will manipulate the struct and return it at the end. Here's the example from the README
:
def hello_world_plug(conn, _opts) do
conn
|> put_resp_content_type("text/plain")
|> send_resp(200, "Hello world")
end
Code blatantly copied from Plug's docs.
If you look at the function, it's quite simple. It receives the connection struct, puts its content type to text/plain
and returns a response with an HTTP 200 status and "Hello world"
as the body.
The second flavour is the module Plug. This means that instead of just having a function that will be invoked as part of the request lifecycle, you can define a module that takes a connection and initialized options and returns the connection:
defmodule MyPlug do
def init([]), do: false
def call(conn, _opts), do: conn
end
Code blatantly copied from Plug's docs.
Having this in mind, let's take a step further and see how we can use Plug in a tiny application.
Plugging a plug as an endpoint
So far, the most important things we covered was what's Plug and what is it used for on a high level. We also took a look at two different types of plugs.
Now, let's see how we can mount a Plug on a Cowboy server and essentially use it as an endpoint:
defmodule PlugTest do
import Plug.Conn
def init(options) do
# initialize options
options
end
def call(conn, _opts) do
conn
|> put_resp_content_type("text/plain")
|> send_resp(200, "Hello world")
end
end
What this module will do is, when mounted on a Cowboy server, will set the Content-Type
header to text/plain
and will return an HTTP 200 with a body of Hello world
.
Let's fire up IEx and test this ourselves:
› iex -S mix
Erlang/OTP 21 [erts-10.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]
Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> {:ok, _ } = Plug.Cowboy.http PlugTest, [], port: 3000
{:ok, #PID<0.202.0>}
This starts the Cowboy server as a BEAM process, listening on port 3000. If we cURL
it we'll see the response body and it's headers:
› curl -v 127.0.0.1:3000
> GET / HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: max-age=0, private, must-revalidate
< content-length: 11
< content-type: text/plain; charset=utf-8
< date: Tue, 25 Dec 2018 22:54:54 GMT
< server: Cowboy
<
* Connection #0 to host 127.0.0.1 left intact
Hello world
You see, the Content-Type
of the response is set to text/plain
and the body is Hello world
. In this example, the plug is essentially an endpoint by itself, serving plain text to our cURL
command (or to a browser). As you might be able to imagine at this point, you can plug in much more elaborate Plugs to a Cowboy server and it will serve them just fine.
To shut down the endpoint all you need to do is:
iex(2)> Plug.Cowboy.shutdown PlugTest.HTTP
:ok
What we are witnessing here is probably the tiniest web application one can write in Elixir. It's an app that takes a request and returns a valid response over HTTP with a status and a body.
So, how does this actually work? How do we accept the request and build a response here?
Diving into the Plug.Conn
To understand this, we need to zoom in the call/2
function of our module PlugTest
. I will also throw in an IO.inspect
right at the end of the function so we can inspect what this struct is:
def call(conn, _opts) do
conn
|> put_resp_content_type("text/plain")
|> send_resp(200, "Hello world")
|> IO.inspect
end
If you start the Cowboy instance again via your IEx session and you hit 127.0.0.1:3000
via cURL
(or a browser), you should see something like this in your IEx session:
%Plug.Conn{
adapter: {Plug.Cowboy.Conn, :...},
assigns: %{},
before_send: [],
body_params: %Plug.Conn.Unfetched{aspect: :body_params},
cookies: %Plug.Conn.Unfetched{aspect: :cookies},
halted: false,
host: "127.0.0.1",
method: "GET",
owner: #PID<0.316.0>,
params: %Plug.Conn.Unfetched{aspect: :params},
path_info: [],
path_params: %{},
port: 3000,
private: %{},
query_params: %Plug.Conn.Unfetched{aspect: :query_params},
query_string: "",
remote_ip: {127, 0, 0, 1},
req_cookies: %Plug.Conn.Unfetched{aspect: :cookies},
req_headers: [
{"accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"},
{"accept-encoding", "gzip, deflate, br"},
{"accept-language", "en-US,en;q=0.9"},
{"connection", "keep-alive"},
{"host", "127.0.0.1:3000"},
{"upgrade-insecure-requests", "1"},
{"user-agent",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"}
],
request_path: "/",
resp_body: nil,
resp_cookies: %{},
resp_headers: [
{"cache-control", "max-age=0, private, must-revalidate"},
{"content-type", "text/plain; charset=utf-8"}
],
scheme: :http,
script_name: [],
secret_key_base: nil,
state: :sent,
status: 200
}
What are we actually looking at? Well, it's actually the Plug representation of a connection. This is a direct interface to the underlying web server and the request that the Cowboy server has received.
Some of the attributes of the struct are pretty self-explanatory, like scheme
, method
, host
, request_path
, etc. If you would like to go into detail what each of these fields is, I suggest taking a look at Plug.Conn
's documentation.
But, to understand better the Plug.Conn
struct, we need to understand the connection lifecycle of each connection struct.
Connection lifecycle
Just like any map in Elixir Plug.Conn
allows us to pattern match on it. Let's modify the little endpoint we created before and try to add some extra IO.inspect
function calls:
defmodule PlugTest do
import Plug.Conn
def init(options) do
# initialize options
options
end
def call(conn, _opts) do
conn
|> inspect_state
|> put_resp_content_type("text/plain")
|> inspect_state
|> put_private(:foo, :bar)
|> inspect_state
|> resp(200, "Hello world")
|> inspect_state
|> send_resp()
|> inspect_state
end
defp inspect_state(conn = %{state: state}) do
IO.inspect state
conn
end
end
Because Plug.Conn
allows pattern matching, we can get the state
of the connection, print it out and return the connection itself so the pipeline in the call/2
function would continue working as expected.
Let's mount this plug on a Cowboy instance and hit it with a simple cURL
request:
iex(6)> Plug.Cowboy.http PlugTest, [], port: 3000
{:ok, #PID<0.453.0>}
# curl 127.0.0.1:3000
iex(21)> :unset
:unset
:unset
:set
:sent
You see, when the connection enters the plug it's state changes from :unset
to :set
to finally :sent
. This means that once the plug is invoked the state of the connection is :unset
. Then we do multiple actions, or in other words, we invoke multiple functions on the Plug.Conn
which add more information to the connection. Obviously, since all variables in Elixir are immutable, each of these function returns a new Plug.Conn
instance, instead of mutating the existing one.
Once the body and the status of the connection are set, then the state changes to :set
. Up until that moment, the state is fixed as :unset
. Once we send the response back to the client the state is changed to :sent
.
What we need to understand here is that whether we have one or more plugs in a pipeline, they will all receive a Plug.Conn
, call functions on it, whether to extract or add data to it and then the connection will be passed on to the next plug. Eventually, in the pipeline, there will be a plug (in the form of an endpoint or a Phoenix controller) that will set the body and the response status and send the response back to the client.
There are a bit more details to this, but this is just enough to wrap our minds around Plug
and Plug.Conn
in general.
Next-level Plug
ging using Plug.Router
Now that we understand how Plug.Conn
works and how plugs can change the connection by invoking functions defined in the Plug.Conn
module, let's look at a more advanced feature of plugs - turning a plug into a router.
In our first example, we saw the simplest of the Elixir web apps - a simple plug that takes the request and returns a simple response with a text body and an HTTP 200. But, what if we want to handle different routes or HTTP methods? What if we want to gracefully handle any request to an unknown route with an HTTP 404?
One nicety that Plug
comes with is a module called Plug.Router
, you can see its documentation here. The router module contains a DSL that allows us to define a routing algorithm for incoming requests and writing handlers (powered by Plug) for the routes. If you are coming from Ruby land, while Plug
is basically Rack, this DSL is Sinatra.rb.
Let's create a tiny router using Plug.Router
, add some plugs to its pipeline and some endpoints.
Quick aside: What is a pipeline?
Although it has the same name as the pipeline operator (|>
), a pipeline in Plug's context is a list of plugs executed one after another. That's really it. The last plug in that pipeline is usually an endpoint that will set the body and the status of the response and return the response to the client.
Now, back to our router:
defmodule MyRouter do
use Plug.Router
plug :match
plug :dispatch
get "/hello" do
send_resp(conn, 200, "world")
end
match _ do
send_resp(conn, 404, "oops")
end
end
Code blatantly copied from Plug.Router
's docs.
The first thing that you will notice here is that all routers are modules as well. By use
ing the Plug.Router
module, we include some functions that make our lives easier, like get
or match
.
If you notice at the top of the module we have two lines:
plug :match
plug :dispatch
This is the router's pipeline. All of the requests coming to the router will pass through these two plugs: match
and dispatch
. The first one does the matching of the route that we define (e.g. /hello
), while the other one will invoke the function defined for a particular route. This means that if we would like to add other plugs, most of the time they will be invoked between the two mandatory ones (match
and dispatch
).
Let's mount our router on a Cowboy server and see it's behaviour:
iex(29)> Plug.Cowboy.http MyRouter, [], port: 3000
{:ok, #PID<0.1500.0>}
When we hit 127.0.0.1:3000/hello
, we will get the following:
› curl -v 127.0.0.1:3000/hello
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /hello HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: max-age=0, private, must-revalidate
< content-length: 5
< date: Thu, 27 Dec 2018 22:50:47 GMT
< server: Cowboy
<
* Connection #0 to host 127.0.0.1 left intact
world
As you can see, we received world
as the response body and an HTTP 200. But if we hit any other URL, the router will match the other route:
› curl -v 127.0.0.1:3000/foo
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /foo HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< cache-control: max-age=0, private, must-revalidate
< content-length: 4
< date: Thu, 27 Dec 2018 22:51:56 GMT
< server: Cowboy
<
* Connection #0 to host 127.0.0.1 left intact
oops
As you can see, because the /hello
route didn't match we defaulted to the other route, also known as "catch all" route, which returned oops
as the response body and an HTTP 404 status.
If you would like to learn more about Plug.Router
and its route matching macros you can read more in its documentation. We still need to cover some more distance with Plug.
Built-in Plugs
In the previous section, we mentioned the plugs match
and dispatch
, and plug pipelines. We also mentioned that we can plug in other plugs in the pipeline so we can inspect or change the Plug.Conn
of each request.
What is very exciting here is that Plug
also comes with already built-in plugs. That means that there's a list of plugs that you can plug-in in any Plug-based application:
Plug.CSRFProtection
Plug.Head
Plug.Logger
Plug.MethodOverride
Plug.Parsers
Plug.RequestId
Plug.SSL
Plug.Session
Plug.Static
Let's try to understand how a couple of them work and how we can plug them in our MyRouter
router module.
Plug.Head
This is a rather simple plug. It's so simple, I will add all of its code here:
defmodule Plug.Head do
@behaviour Plug
alias Plug.Conn
def init([]), do: []
def call(%Conn{method: "HEAD"} = conn, []), do: %{conn | method: "GET"}
def call(conn, []), do: conn
end
What this plug does is it turns any HTTP HEAD
request into a GET
request. That's all. Its call
function receives a Plug.Conn
, matches only the ones that have a method: "HEAD"
and returns a new Plug.Conn
with the method
changed to "GET"
.
If you've been wondering what the HEAD
method is for, this is from RFC 2616:
The HEAD method is identical to GET except that the server MUST NOT return a
message-body in the response. The metainformation contained in the HTTP headers
in response to a HEAD request SHOULD be identical to the information sent in
response to a GET request. This method can be used for obtaining
metainformation about the entity implied by the request without transferring
the entity-body itself. This method is often used for testing hypertext links
for validity, accessibility, and recent modification.
Let's plug this plug in our Plug.Router
(pun totally intended):
defmodule MyRouter do
use Plug.Router
plug Plug.Head
plug :match
plug :dispatch
get "/hello" do
send_resp(conn, 200, "world")
end
match _ do
send_resp(conn, 404, "oops")
end
end
Once we cURL
the routes we would get the following behaviour:
› curl -I 127.0.0.1:3000/hello
HTTP/1.1 200 OK
cache-control: max-age=0, private, must-revalidate
content-length: 5
date: Thu, 27 Dec 2018 23:25:13 GMT
server: Cowboy
› curl -I 127.0.0.1:3000/foo
HTTP/1.1 404 Not Found
cache-control: max-age=0, private, must-revalidate
content-length: 4
date: Thu, 27 Dec 2018 23:25:17 GMT
server: Cowboy
As you can see, although we didn't explicitly match the HEAD
routes using the head
macro, the Plug.Head
plug remapped the HEAD
requests to GET
and our handlers still kept on working as expected (the first one returned an HTTP 200, and the second one an HTTP 404).
Plug.Logger
This one is a bit more complicated so we cannot inline all of its code in this article. Basically, if we would plug this plug in our router, it will log all of the incoming requests and response statuses, like so:
GET /index.html
Sent 200 in 572ms
This plug uses Elixir's Logger
(docs under the hood, which supports four different logging levels:
-
:debug
- for debug-related messages -
:info
- for information of any kind (default level) -
:warn
- for warnings -
:error
- for errors
If we would look at the source of its call/2
function, we would notice two logical units. The first one is:
def call(conn, level) do
Logger.log(level, fn ->
[conn.method, ?\s, conn.request_path]
end)
# Snipped...
end
This one will take Elixir's Logger
and using the logging level
will log the information to the backend (by default it's console
). The information that is logged is the method of the request (e.g. GET
, POST
, etc) and the request path (e.g. /foo/bar
). This results in the first line of the log:
GET /index.html
The second logical unit is a bit more elaborate:
def call(conn, level) do
# Snipped...
start = System.monotonic_time()
Conn.register_before_send(conn, fn conn ->
Logger.log(level, fn ->
stop = System.monotonic_time()
diff = System.convert_time_unit(stop - start, :native, :microsecond)
status = Integer.to_string(conn.status)
[connection_type(conn), ?\s, status, " in ", formatted_diff(diff)]
end)
conn
end)
end
In short: this section records the time between the start
and the stop
(end) of the request and prints out the diff
erence between the two (or in other words - the amount of time the response took). Also, it prints out the HTTP status of the response.
To do this it uses Plug.Conn.register_before_send/2
(docs) which is a utility function that registers callbacks to be invoked before the response is sent. This means that the function which will calculate the diff
and log it to the Logger
with the response status will be invoked by Plug.Conn
right before the response is sent to the client.
Wrapping up with Plug
You actually made it this far - I applaud you. I hope that this was a nice journey for you in Plug and it's related modules/functions and that you learned something new.
We looked at quite a bit of details in and around Plug
. For some of the modules that we spoke about we barely scratched the surface. For example, Plug.Conn
has quite a bit of more useful functions. Or Plug.Router
has more functions in its DSL where you can write more elaborate and thoughtful APIs or web apps. In line with this, Plug
also offers more built-in plugs. It even has a plug which can serve static files with ease, and plugging it in your Plug-based apps is a breeze.
But, aside from all the things that we skipped in this article, I hope that you understood how powerful the Plug model is and how much power it provides us with such simplicity and unobtrusiveness.
In future posts, we will look at even more details about other plugs in Plug
, but until then please shoot me a comment or a message if you've found this article helpful (or not).
Top comments (0)