In the last post I did about the Vegetables testing framework I developed, I talked about how much of a success it had been. Now I’ve cranked it up to 11. I’ve now got what is effectively QuickCheck level functionality implemented in a Fortran testing framework.
For anyone not familiar with QuickCheck, it is effectively a library for generating random values to use as examples for your tests. But the really useful part is that if it finds an example that causes your test to fail, it then attempts to find the simplest example that causes your test to fail, and reports that to you.
The simplest example I have is just that an integer should equal itself.
function checkPassForSameInteger(input) result(result_)
use Vegetables_m, only: Result_t, assertEquals, assertThat, fail
class(*), intent(in) :: input
type(Result_t) :: result_
type(Result_t) :: example_result
select type (input)
type is (integer)
example_result = assertEquals(input, input)
result_ = assertThat( &
example_result%passed(),
example_result%verboseDescription(.false.))
class default
result_ = fail("Expected to get an integer")
end select
end function checkPassForSameInteger
Running just this test now gives the following result.
$ test_build/vegetable_driver -q -v -f "passes with the same integer"
Running Tests
A total of 1 test cases
All Passed
Test that
assertEquals with integers
passes with the same integer
Passed after 100 examples
A total of 1 test cases containg a total of 1 assertions
Then, if I change the assert to assertEquals(input, input+1)
, I get the following.
$ test_build/vegetable_driver -q -v -f "passes with the same integer"
Running Tests
A total of 1 test cases
Failed
Test that
assertEquals with integers
passes with the same integer
Fails with the simplest possible example
Expected to be true
User Message:
[Expected
[0]
but got
[1]]
1 of 1 cases failed
2 of 2 assertions failed
Or, if I change the assert to assertEquals(input, input/2)
, I get the following.
$ test_build/vegetable_driver -q -v -f "passes with the same integer"
Running Tests
A total of 1 test cases
Failed
Test that
assertEquals with integers
passes with the same integer
Found simplest example causing failure
Expected to be true
User Message:
[Expected
[1]
but got
[0]]
1 of 1 cases failed
2 of 2 assertions failed
In order to use the QuickCheck functionality you need to create a derived type extended from the Generator_t
type. This requires you to implement two functions for it. You need generate
, which takes your generator type as an argument, and produces a random value wrapped in a Generated_t
type. Then you need shrink
, which takes one of your generated values as a class(*)
, and returns a “simpler” value than the one it was given, wrapped in either a ShrunkValue_t
or a SimplestValue_t
to indicate the value can’t be made any simpler.
I’ve implemented a couple of basic Generators and some helper functions for generating some random values, but for most things, you’ll probably need to create your own. But if you can, and you can specify some properties of your code that should hold for any random value, you can really stress test your code.
I think at this point I have one of the most feature-full testing frameworks I’ve ever used. And I think in fewer lines of code than any other testing framework I’ve ever seen. Functional programming principles for the win!
Top comments (0)