DEV Community

Giulio Canti
Giulio Canti

Posted on • Edited on

Introduction to property based testing

In the last posts about Eq, Ord, Semigroup and Monoid we saw that instances must comply with some laws.

So how can we ensure that our instances are lawful?

Property based testing

Property based testing is another way to test your code which is complementary to classical unit-test methods.

It tries to discover inputs causing a property to be falsy by testing it against multiple generated random entries. In case of failure, a property based testing framework provides both a counterexample and the seed causing the generation.

Let's apply property based testing to the Semigroup law:

Associativity : concat(concat(x, y), z) = concat(x, concat(y, z))

I'm going to use fast-check, a property based testing framework written in TypeScript.

Testing a Semigroup instance

We need three ingredients

  1. a Semigroup<A> instance for the type A
  2. a property that encodes the associativity law
  3. a way to generate random values of type A

Instance

As instance I'm going to use the following

import { Semigroup } from 'fp-ts/Semigroup'

const S: Semigroup<string> = {
  concat: (x, y) => x + ' ' + y
}
Enter fullscreen mode Exit fullscreen mode

Property

A property is just a predicate, i.e a function that returns a boolean. We say that the property holds if the predicate returns true.

So in our case we can define the associativity property as

const associativity = (x: string, y: string, z: string) =>
  S.concat(S.concat(x, y), z) === S.concat(x, S.concat(y, z))
Enter fullscreen mode Exit fullscreen mode

Arbitrary<A>

An Arbitrary<A> is responsible to generate random values of type A. We need an Arbitrary<string>, fortunately fast-check provides many built-in arbitraries

import * as fc from 'fast-check'

const arb: fc.Arbitrary<string> = fc.string()
Enter fullscreen mode Exit fullscreen mode

Let's wrap all together

it('my semigroup instance should be lawful', () => {
  fc.assert(fc.property(arb, arb, arb, associativity))
})
Enter fullscreen mode Exit fullscreen mode

If fast-check doesn't raise any error we can be more confident that our instance is well defined.

Testing a Monoid instance

Let's see what happens when an instance is lawless!

As instance I'm going to use the following

import { Monoid } from 'fp-ts/Monoid'

const M: Monoid<string> = {
  ...S,
  empty: ''
}
Enter fullscreen mode Exit fullscreen mode

We must encode the Monoid laws as properties:

  • Right identity : concat(x, empty) = x
  • Left identity : concat(empty, x) = x
const rightIdentity = (x: string) => M.concat(x, M.empty) === x

const leftIdentity = (x: string) => M.concat(M.empty, x) === x
Enter fullscreen mode Exit fullscreen mode

and finally write a test

it('my monoid instance should be lawful', () => {
  fc.assert(fc.property(arb, rightIdentity))
  fc.assert(fc.property(arb, leftIdentity))
})
Enter fullscreen mode Exit fullscreen mode

When we run the test we get

Error: Property failed after 1 tests
{ seed: -2056884750, path: "0:0", endOnFailure: true }
Counterexample: [""]
Enter fullscreen mode Exit fullscreen mode

That's great, fast-check even gives us a counterexample: ""

M.concat('', M.empty) = ' ' // should be ''
Enter fullscreen mode Exit fullscreen mode

Resources

For a library that makes easy to test type classes laws, check out fp-ts-laws

Top comments (4)

Collapse
 
tevescastro profile image
Vítor Castro

I was playing a bit with this and wondering if/how it would make sense to test a monoid for tasks. Any ideas? Particularly on generating a setoid for those types?

Collapse
 
jvanbruegge profile image
Jan van Brügge

I think you have a typo in your Monoid instance. The string should not be empty for the test to fail

Collapse
 
gcanti profile image
Giulio Canti

concat(x, empty) is equal to x + ' ' + empty by definition of concat. If x = '' then x + ' ' + empty is equal to '' + ' ' + '' which is equal to ' ' so concat(x, empty) !== x

Collapse
 
jvanbruegge profile image
Jan van Brügge

Ah, yeah, I missed the extra space in the Semigroup instance