In the previous post of the series we started to explore the Effect data type in details.
The Effect type is itself used to construct a variety of data-types that are thought to cover many of the common patterns of software development.
This time we will explore Managed and Layer both constructed using Effect, the first uses to safely allocate and release resources and the second used to construct environments.
In the code snippets we will simulate data stores by making usage of a third data-type built on Effect the Ref data-type, Ref<T> encodes a mutable Reference to an immutable value T and can be used to keep track of State through a computation.
Bracket
Let's start by taking a look at a classic pattern we encounter many times: bracket
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
// connect to the database
export const connectDb = pipe(
// make a reference to an empty map
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
// get the current map and lookup k
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
// updates the reference to an updated Map containing k->v
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
// clean up (for simulation purposes)
clear: ref.set(Map.empty)
})
)
)
)
// write a program that use the database
export const program = pipe(
// acquire the database connection
connectDb,
T.bracket(
// use the database connection
(_) =>
pipe(
T.do,
T.tap(() => _.put("ka", "a")),
T.tap(() => _.put("kb", "b")),
T.tap(() => _.put("kc", "c")),
T.bind("a", () => _.get("ka")),
T.bind("b", () => _.get("kb")),
T.bind("c", () => _.get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`)
),
// release the database connection
(_) => _.clear
)
)
// run the program and print the output
pipe(
program,
T.chain((s) =>
T.effectTotal(() => {
console.log(s)
})
),
T.runMain
)
Bracket is used to safely acquire Resources like DbConnection to be used and safely released post usage.
Managed
The idea behind Managed is to generalize the behaviour of bracket by isolating the declaration of acquire and release into a specific data-type.
The inner details of how this data-type works are out of the scope of this post but the design is a very interesting (more advanced) topic to explore. It is based on the implementation of ZManaged in ZIO that is itself based on the ResourceT designed by Michael Snoyman in haskell.
Let's take a look at the same code when isolated into a Managed:
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
// connect to the database
export const managedDb = pipe(
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
clear: ref.set(Map.empty)
})
)
),
// release the connection via managed
M.make((_) => _.clear)
)
// write a program that use the database
export const program = pipe(
// use the managed DbConnection
managedDb,
M.use((_) =>
pipe(
T.do,
T.tap(() => _.put("ka", "a")),
T.tap(() => _.put("kb", "b")),
T.tap(() => _.put("kc", "c")),
T.bind("a", () => _.get("ka")),
T.bind("b", () => _.get("kb")),
T.bind("c", () => _.get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`)
)
)
)
// run the program and print the output
pipe(
program,
T.chain((s) =>
T.effectTotal(() => {
console.log(s)
})
),
T.runMain
)
We can see how we moved the release step from our program into the data-type of managedDb, now we can use the Resource without having to track its closing state.
Composing Managed Resources
One other advantage that we gained is we can now easily compose multiple Resources together, let's take a look at how we might do that:
import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
// simulate a connection to a message broker
export interface BrokerConnection {
readonly send: (message: string) => T.UIO<void>
readonly clear: T.UIO<void>
}
// connect to the database
export const managedDb = pipe(
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
clear: ref.set(Map.empty)
})
)
),
// release the connection via managed
M.make((_) => _.clear)
)
// connect to the database
export const managedBroker = pipe(
Ref.makeRef(<Array.Array<string>>Array.empty),
T.chain((ref) =>
T.effectTotal(
(): BrokerConnection => ({
send: (message) =>
pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
clear: pipe(
ref.get,
T.chain((messages) =>
T.effectTotal(() => {
console.log(`Flush:`)
messages.forEach((message) => {
console.log("- " + message)
})
})
)
)
})
)
),
// release the connection via managed
M.make((_) => _.clear)
)
// write a program that use the database
export const program = pipe(
// use the managed DbConnection
managedDb,
M.zip(managedBroker),
M.use(([{ get, put }, { send }]) =>
pipe(
T.do,
T.tap(() => put("ka", "a")),
T.tap(() => put("kb", "b")),
T.tap(() => put("kc", "c")),
T.bind("a", () => get("ka")),
T.bind("b", () => get("kb")),
T.bind("c", () => get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`),
T.tap(send)
)
)
)
// run the program and print the output
pipe(
program,
T.chain((s) =>
T.effectTotal(() => {
console.log(`Done: ${s}`)
})
),
T.runMain
)
We have added a second Managed called managedBroker to simulate a connection to a message broker and we composed the 2 using zip.
The result will be a Managed that acquires both Resources and release both Resources as you would expect.
There are many combinators in the Managed module, for example a simple change from zip to zipPar would change the behaviour of the composed Managed and will acquire and release both Resources in parallel.
Layer
The Layer data-type is closely related to Managed, in fact the idea for the design of Layer has its roots in a pattern commonly discovered in the early days of ZIO by early adopters.
We have looked at the Effect data-type already and we notice how the environment R can be used to embed services in order to gain great testability and code organization.
The question now is, how do I construct those environments?
It seems simple in principle but when you get to write an app this embedding of services becomes so pervasive that is completely normal to find yourself embedding hundreds of services in your "main" program.
Many of the services are going to be dependent on database connections and stuff like that, and as we saw before Managed is a good use case for those.
The community has started to construct their environments by using Managed so that services are constructed at bootstrap time and the result is used to provide the environment.
This pattern progressively evolved to an independent data-type that is based on Managed tailored to specifically to construct environments.
Let's start by simply transforming our code to use Layers, it will be straightforward given we have already designed it using Managed:
import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
// simulate a connection to a message broker
export interface BrokerConnection {
readonly send: (message: string) => T.UIO<void>
readonly clear: T.UIO<void>
}
// connect to the database
export const DbLive = pipe(
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
clear: ref.set(Map.empty)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromRawManaged
)
// connect to the database
export const BrokerLive = pipe(
Ref.makeRef(<Array.Array<string>>Array.empty),
T.chain((ref) =>
T.effectTotal(
(): BrokerConnection => ({
send: (message) =>
pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
clear: pipe(
ref.get,
T.chain((messages) =>
T.effectTotal(() => {
console.log(`Flush:`)
messages.forEach((message) => {
console.log("- " + message)
})
})
)
)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromRawManaged
)
export const ProgramLive = L.all(DbLive, BrokerLive)
// write a program that use the database
export const program = pipe(
// access Db from environment
T.access(({ get, put }: DbConnection) => ({ get, put })),
// access Broker from environment
T.zip(T.access(({ send }: BrokerConnection) => ({ send }))),
// use both
T.chain(([{ get, put }, { send }]) =>
pipe(
T.do,
T.tap(() => put("ka", "a")),
T.tap(() => put("kb", "b")),
T.tap(() => put("kc", "c")),
T.bind("a", () => get("ka")),
T.bind("b", () => get("kb")),
T.bind("c", () => get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`),
T.tap(send)
)
)
)
// run the program and print the output
pipe(
program,
T.chain((s) =>
T.effectTotal(() => {
console.log(`Done: ${s}`)
})
),
// provide the layer to program
T.provideSomeLayer(ProgramLive),
T.runMain
)
What we did is we turned our Managed into Layers by using the fromRawManaged constructor and we then plugged the results into a single ProgramLive layer.
We then transformed our program to consume the DbConnection and BrokerConnection services from enviroment.
At the very end we provided the ProgramLive to our program by using provideSomeLayer.
The behaviour itself remained the same but we have now fully separated program definition from resources and services location.
The Has & Tag utilities
In the last article of the series we scratched the surface of the Has & Tag apis, we are going to see now a bit more details about the inner working and how it integrates into Layers.
Up to now we haven't noticed issues because we took care of keeping function names like send and get separated but in reality it is not a good idea to intersect casual services into a big object because conflicts might easily arise.
One conflict we had without even noticing (because it's used only locally inside managed) is the "clear" effect of both DbConnection and BrokerConnection, that will get shadowed at runtime in the intersected result.
In order to solve this issue, and in order to improve the amount of types you end up writing in your app we designed two data-types that solve the issue.
We took inspiration from the work done in ZIO where the Has type is explicit and the Tag type is implicit in typescript we had to come up with an explicit solution.
The trick works like that:
import { tag} from "@effect-ts/core/Classic/Has"
/**
* Any interface or type alias
*/
interface Anything {
a: string
}
/**
* Tag<Anything>
*/
const Anything = tag<Anything>()
/**
* (r: Has<Anything>) => Anything
*/
const readFromEnv = Anything.read
/**
* (_: Anything) => Has<Anything>
*/
const createEnv = Anything.of
const hasAnything = createEnv({ a: "foo" })
/**
* Has<Anything> is fake, in reality we have:
*
* { [Symbol()]: { a: 'foo' } }
*/
console.log(hasAnything)
/**
* The [Symbol()] is:
*/
console.log((hasAnything as any)[Anything.key])
/**
* The same as:
*/
console.log(readFromEnv(hasAnything))
/**
* In order to take ownership of the symbol used we can do:
*/
const mySymbol = Symbol()
const Anything_ = tag<Anything>().setKey(mySymbol)
console.log((Anything_.of({ a: "bar" }) as any)[mySymbol])
Fundamentally Tag<T> encodes the capability of reading T from a type Has<T> and encodes the capability of producing a value of type Has<T> given a T.
With the guarantee that given T0 and T1 the result Has<T0> & Has<T1> is always discriminated.
Let's see it practice by rewriting our code:
import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
import { tag } from "@effect-ts/system/Has"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
export const DbConnection = tag<DbConnection>()
// simulate a connection to a message broker
export interface BrokerConnection {
readonly send: (message: string) => T.UIO<void>
readonly clear: T.UIO<void>
}
export const BrokerConnection = tag<BrokerConnection>()
// connect to the database
export const DbLive = pipe(
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
clear: ref.set(Map.empty)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromManaged(DbConnection)
)
// connect to the database
export const BrokerLive = pipe(
Ref.makeRef(<Array.Array<string>>Array.empty),
T.chain((ref) =>
T.effectTotal(
(): BrokerConnection => ({
send: (message) =>
pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
clear: pipe(
ref.get,
T.chain((messages) =>
T.effectTotal(() => {
console.log(`Flush:`)
messages.forEach((message) => {
console.log("- " + message)
})
})
)
)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromManaged(BrokerConnection)
)
export const ProgramLive = L.all(DbLive, BrokerLive)
// write a program that use the database
export const program = pipe(
// access Db from environment
T.accessService(DbConnection)(({ get, put }) => ({ get, put })),
// access Broker from environment
T.zip(T.accessService(BrokerConnection)(({ send }) => ({ send }))),
// use both
T.chain(([{ get, put }, { send }]) =>
pipe(
T.do,
T.tap(() => put("ka", "a")),
T.tap(() => put("kb", "b")),
T.tap(() => put("kc", "c")),
T.bind("a", () => get("ka")),
T.bind("b", () => get("kb")),
T.bind("c", () => get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`),
T.tap(send)
)
)
)
// run the program and print the output
pipe(
program,
T.chain((s) =>
T.effectTotal(() => {
console.log(`Done: ${s}`)
})
),
// provide the layer to program
T.provideSomeLayer(ProgramLive),
T.runMain
)
By this very simple change we are now safe, and as we can already see, instead of manually annotate the type DbConnection to access the service we can now simply use the Tag value.
We can go a bit further and derive some functions that we would normally like to write, the idea is:
T.accessService(DbConnection)(({ get, put }) => ({ get, put }))
You don't want that in code.
To avoid repeating this you end-up writing commodity functions like:
export function get(k: string) {
return T.accessServiceM(DbConnection)((_) => _.get(k))
}
and use this instead of direct access.
We can automatically derive such implementations in many of the common function types except the case of functions containing generics.
Let's see it in practice:
import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
import { tag } from "@effect-ts/system/Has"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
export const DbConnection = tag<DbConnection>()
// simulate a connection to a message broker
export interface BrokerConnection {
readonly send: (message: string) => T.UIO<void>
readonly clear: T.UIO<void>
}
export const BrokerConnection = tag<BrokerConnection>()
// connect to the database
export const DbLive = pipe(
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
clear: ref.set(Map.empty)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromManaged(DbConnection)
)
// connect to the database
export const BrokerLive = pipe(
Ref.makeRef(<Array.Array<string>>Array.empty),
T.chain((ref) =>
T.effectTotal(
(): BrokerConnection => ({
send: (message) =>
pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
clear: pipe(
ref.get,
T.chain((messages) =>
T.effectTotal(() => {
console.log(`Flush:`)
messages.forEach((message) => {
console.log("- " + message)
})
})
)
)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromManaged(BrokerConnection)
)
export const ProgramLive = L.all(DbLive, BrokerLive)
export const { get, put } = T.deriveLifted(DbConnection)(["get", "put"], [], [])
export const { send } = T.deriveLifted(BrokerConnection)(["send"], [], [])
// write a program that use the database
export const program = pipe(
T.do,
T.tap(() => put("ka", "a")),
T.tap(() => put("kb", "b")),
T.tap(() => put("kc", "c")),
T.bind("a", () => get("ka")),
T.bind("b", () => get("kb")),
T.bind("c", () => get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`),
T.tap(send)
)
// run the program and print the output
pipe(
program,
T.chain((s) =>
T.effectTotal(() => {
console.log(`Done: ${s}`)
})
),
// provide the layer to program
T.provideSomeLayer(ProgramLive),
T.runMain
)
The other 2 arrays of deriveLifted are used to generate functions to access Constant Effects and Constants as Effects.
Like deriveLifted other 2 functions can be used to derive commonly used utilities and those are deriveAccess and deriveAccessM.
It is left as an exercise to the reader to try out those combinations and look at the result function types in order to better understand the context of each derivation.
Another way to smooth the usage of services without the need to actually write any of the utility functions is to effectively write your main program as a service and leverage the layer constructor utilities to automate the environment wiring.
Let's see how we might do that:
import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
import { tag } from "@effect-ts/system/Has"
// simulate a database connection to a key-value store
export interface DbConnection {
readonly put: (k: string, v: string) => T.UIO<void>
readonly get: (k: string) => T.IO<NoSuchElementException, string>
readonly clear: T.UIO<void>
}
export const DbConnection = tag<DbConnection>()
// simulate a connection to a message broker
export interface BrokerConnection {
readonly send: (message: string) => T.UIO<void>
readonly clear: T.UIO<void>
}
export const BrokerConnection = tag<BrokerConnection>()
// connect to the database
export const DbLive = pipe(
Ref.makeRef(<Map.Map<string, string>>Map.empty),
T.chain((ref) =>
T.effectTotal(
(): DbConnection => ({
get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
clear: ref.set(Map.empty)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromManaged(DbConnection)
)
// connect to the database
export const BrokerLive = pipe(
Ref.makeRef(<Array.Array<string>>Array.empty),
T.chain((ref) =>
T.effectTotal(
(): BrokerConnection => ({
send: (message) =>
pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
clear: pipe(
ref.get,
T.chain((messages) =>
T.effectTotal(() => {
console.log(`Flush:`)
messages.forEach((message) => {
console.log("- " + message)
})
})
)
)
})
)
),
// release the connection via managed
M.make((_) => _.clear),
// construct the layer
L.fromManaged(BrokerConnection)
)
export function makeProgram({ get, put }: DbConnection, { send }: BrokerConnection) {
return {
main: pipe(
T.do,
T.tap(() => put("ka", "a")),
T.tap(() => put("kb", "b")),
T.tap(() => put("kc", "c")),
T.bind("a", () => get("ka")),
T.bind("b", () => get("kb")),
T.bind("c", () => get("kc")),
T.map(({ a, b, c }) => `${a}-${b}-${c}`),
T.tap(send)
)
}
}
export interface Program extends ReturnType<typeof makeProgram> {}
export const Program = tag<Program>()
export const ProgramLive = L.fromConstructor(Program)(makeProgram)(
DbConnection,
BrokerConnection
)
export const MainLive = pipe(ProgramLive, L.using(L.all(DbLive, BrokerLive)))
export const { main } = T.deriveLifted(Program)([], ["main"], [])
// run the program and print the output
pipe(
main,
T.chain((s) =>
T.effectTotal(() => {
console.log(`Done: ${s}`)
})
),
// provide the layer to program
T.provideSomeLayer(MainLive),
T.runMain
)
We basically introduced a new service Program as a constructor makeProgram that simply takes dependencies as arguments and we used the Layer constructor fromConstructor to glue everything.
We had to add the ProgramLive layer to our final MainLive cake and in the main function we called the effect from the program service (that can be easily derived as before).
Stay Tuned
In the next post of the series we will be discussing about generators and how to get back to an imperative paradigm by using generators to simulate a monadic do that will look very similar to the dear old async/await but with much better typing.
Top comments (4)
Looking forward to see how you implemtented monad comprehensions using generators!
As an exercise I have tried to implement them for a plain Reader monad. It is fully typesafe, using the yield* trick, that is also popular in the redux saga ecosystem.
gist.github.com/kasperpeulen/47242...
Similar in some way :)
"Michael Snoyberg" -> Michael Snoyman
Thanks :) I was too quick in writing and used his twitter name :)