When we work in a REST API one of the first things to do is validate incomming data.
Suddenly I must to rewrite a REST API from Javascript to Elixir and find that some of the existing library does not feed my basic needs of simplicity and less code possible.
All of us who have a Node-Express background, have used Celebrate and Joi in order to validate the presence of key, their data type and their stored value.
In this article I will show you how to implement in Elixir a schema validator, trying to mimic a (minimal) functionality of Celebrate Joi.
Table of Contents
- 1-Plugs and Midlewares
- 2-Route-Schema Validation
- 3-Function Clause Matching
- 4-Json Schema Validation
1) Plugs and Midlewares
There is a common architecture between Node-Express and Elixir-Cowboy, Express have middlewares and Cowboy have Plugs.
In both cases these are used for routing, body parsing, user authentication verification, cross-origin resource sharing, data logging, data validation, metrics request, etc..
How the documentation say :"An Express application is essentially a series of middleware function calls." Using middlewars
For example: A classic Node-Express midlewares chain
//file: app_router.js
// import boiler plate ...
app.use(cors(cors_configuration)); //CORS
app.use(express.json());//Json body parsing
app.use(isAuth);//user jwt authentication verification
app.use(function (err, req, res, next) { // error handling
if (err.name === "UnauthorizedError") {
res.status(401).send("invalid token...");
} else {
next(err);
}
});
app.use(attachUserId);// add extra information about the current user
app.use((req, res, next) => { // some debug
console.log("Method",req.method);
console.log("Url", req.originalUrl);
console.log("Body:",req.body);
return next();
});
The same happens in Elixir-Cowboy, where plugs are chained one after other, like this:
# file: app_router.ex
# use/import boiler plate ...
plug Corsica, cors_configuration # CORS
plug :match
plug Plug.Telemetry, telemetry_configuration # metrics request
# body parsing
plug Plug.Parsers, parsers: [:json], pass: ["application/json"], json_decoder: Jason
# Json validation
plug Plug.RouterValidatorschema, on_error: &__MODULE__.on_error_fn/2
# user jwt authentication verification
plug Plug.Authenticator, on_error: &__MODULE__.on_error_fn/2
# add extra information about the current user
plug Plug.AttachUserId, on_error: &__MODULE__.on_error_fn/2
# some debug
plug Plug.PrintUrlMethod
plug :dispatch
Celebrate is a middleware, the equivalent in Elixir is a Plug, let start writing a plug:
The next link is the started point to understand and learn about how to write a plug, I recommend read it and then come here: The Plug Specification
There are two ways to define it: function plugs and module plugs, I will choose the module plugs way.
2) Route-Schema Validation
Has programers we want to write the minimal possible lines of code. Bassically reduce all to a schema definition and then apply in our route.
Schema:
photo_book_schema = [
qty: {:number, 1, 200},
hardcover: {:boolean},
colors: {:options, ["bw", "color", "sepia"]},
client_name: {:string, 6, 20},
client_phone: {:string, 8, 25, ~r/^[\d\-()\s]+$/}
]
Route:
plug(:match)
plug(Plug.RouterValidatorschema, on_error: &__MODULE__.on_error_fn/2)
plug(:dispatch)
post "/create/", private: %{validate: photo_book_schema} do
status = Orders.create(conn.body_params)
{:ok, str} = Jason.encode(%{status: status})
send_resp(conn |> put_resp_content_type("application/json"), str)
end
def on_error_fn(conn, errors, status \\ 422) do
IO.inspect(errors, label: "errors")
{:ok, str} = Jason.encode(%{status: "fail", errors: errors})
send_resp(conn |> put_resp_content_type("application/json"), str) |> halt()
end
Anxious for the code ?
Before to go deeper in the code, we learn a bite about Function Clause Matching.
2) Function Clause Matching
One of the powerfull tools of Elixir is pattern matching. It is applied in many places, thanks to that, we write less lines of code compared with another languages.
In this case is applied in function clauses, Elixir let us to write multiple definitions for the same function, every definitions is called a clause.
See the next example:
defmodule Discount do
def calculate(type, price) do
case type do
:normal -> price * 1
:high -> price * 0.8
:low -> price * 0.9
end
end
end
Instead we can write:
defmodule Discount do
def calculate(:normal, price), do: price
def calculate(:high, price), do: price * 0.8
def calculate(:low, price), do: price * 0.9
end
Where the "case" statement is moved outside of the function and the virtual machine take the decision of which must to apply.All definition refer to the same function "calculate" with arity 2.
2.1) Tips.
Some tips when use function clause matching.
Group:
Always group clauses of the same function together,
Don't mix with others:
defmodule FooBar do
def foo(:a), do: 'a'
def foo(:b), do: 'b'
def bar(arg), do: IO.puts arg
def foo(:c), do: 'c'
def foo(arg), do: IO.puts arg
end
Will receive a compiler warning:
warning: clauses with the same name and arity (number of arguments) should be grouped together, "def foo/1" was previously defined.
Correct way:
defmodule FooBar do
def foo(:a), do: 'a'
def foo(:b), do: 'b'
def foo(:c), do: 'c'
def foo(arg), do: IO.puts arg
def bar(arg), do: IO.puts arg
end
Order:
The declaration order really matter:
Don't do:
defmodule FooBar do
def foo(arg), do: arg
def foo(:b), do: 'b'
def foo(:c), do: 'c'
end
Will receive a compiler warning:
warning: this clause for foo/1 cannot match because a previous clause always matches
Correct way, sort for selective weight:
defmodule FooBar do
def foo(:b), do: 'b'
def foo(:c), do: 'c'
def foo(arg), do: arg
end
Match all:
Alway write a final match all clause.
Sometimes we receive a inexpectable argument value and then a error is triggered:
(FunctionClauseError) no function clause matching
Don't do:
defmodule FooBar do
def foo(:b), do: 'b'
def foo(:c), do: 'c'
end
Correct way:
defmodule FooBar do
def foo(:b), do: 'b'
def foo(:c), do: 'c'
def foo(_), do: "Something happen"
end
4) Schema Validation
The Plug.Router provide a way to passing data between routes and plugs
The Plug.conn struct
A analogy to better undertand the Plug.conn and the plug chain is think in a train locomotive with wagons, when every plug is a station where we load or unload data and the wagons are the fields of the struct.
The Plug.conn struct have a field named private, it will be seted with a key and a value.
post "/create/" , private: %{validate: photo_book_schema} do
.....
end
When the plug find the validate key inside the private field, the validation process start, calling the function validate_schema()
defmodule Plug.RouterValidatorSchema do
def init(opts), do: opts
def call(conn, opts) do
case conn.private[:validate] do
nil -> conn
schema -> validate_schema(conn,schema, opts[:on_error])
end
end
The validate_schema() walk over the schema list and validate the keys inside conn.body_params
If all validations are successful returns an empty map.
Otherwise when validation fail, call the given on_error callback with a list of the collected errors.
The errors list have the shape [ %{"key1"=> "error"}, %{"key2"=> "error"}]
defp validate_schema(conn, schema, on_error) do
errors = Enum.reduce(schema, [], validate_key(conn.body_params))
if Enum.empty?(errors) do
conn
else
on_error.(conn, %{errors: errors})
end
end
defp validate_key(data) do
fn {key, test}, errors ->
skey = Atom.to_string(key)
cond do
not Map.has_key?(data, skey) -> [%{skey => "missing key"} | errors]
is_nil(data[skey]) -> [%{skey => "value is nil"} | errors] # or just return errors if value can accept nil
true ->
case is_t(test, data[skey]) do
:ok -> errors
{:invalid, reason} -> [%{skey => reason} | errors]
end
end
end
end
The validate_key(data) return a parameterized function that be in charge of apply every test to the key associated data.
The is_t(test,value) will perform the test and here is where we take advantage of function clause matching, the function test for boolean, number, string, string patterns, options, and is very easy add a new test for any desired data type.
defp is_t({:boolean}, value) do
if (not is_boolean(value)) do
{:invalid, "not is boolean"}
else
:ok
end
end
defp is_t({:number}, value) do
if (not is_number(value)) do
{:invalid, "not is number"}
else
:ok
end
end
defp is_t({:number, vmin, vmax}, value) do
cond do
not is_number(value) -> {:invalid, "not is number"}
value <= vmin -> {:invalid, "min value"}
value > vmax -> {:invalid, "max value"}
true ->:ok
end
end
defp is_t({:string}, value) do
if is_bitstring(value) do
:ok
else
{:invalid, "not is string"}
end
end
defp is_t({:string, lmin, lmax}, value) do
cond do
not is_bitstring(value) -> {:invalid, "not is string"}
String.length(value) <= lmin -> {:invalid, "min length"}
String.length(value) > lmax -> {:invalid, "max length"}
true ->:ok
end
end
defp is_t({:string, lmin, lmax, reg}, value) do
cond do
not is_bitstring(value) -> {:invalid, "not is string"}
String.length(value) <= lmin -> {:invalid, "min length"}
String.length(value) > lmax -> {:invalid, "max length"}
not Regex.match?(reg,value) -> {:invalid, "regex not match"}
true ->:ok
end
end
defp is_t({:options, alloweds}, value) do
cond do
value not in alloweds -> {:invalid, "not allowed"}
true ->:ok
end
end
# ---------------------------
# Final match :
# ---------------------------
defp is_t(_value,_test) do
{:invalid, "test match fail"}
end
Top comments (0)