How to iteratively test in Haskell using QuickCheck?

Question

How do you generate a random instance in Haskell QuickCheck and then run further tests on this very instance?

Let me demonstrate by giving an example. Within the following code snippet I generate an Int and then want to run further tests on this very Int:

{-# OPTIONS -Wall #-}

import           Test.Tasty
import           Test.Tasty.QuickCheck as QC
import qualified Debug.Trace as Trace

main :: IO()
main = defaultMain tests

test1 :: Int -> Property
test1 i = withMaxSuccess 2 (test2 i)

test2 :: Int -> (Int, Int) -> Property
test2 i (a, b) =
  Trace.trace ("(n,a,b) = " ++ show (i, a, b)) $
  property True

tests :: TestTree
tests =
  testGroup
  "Test suite"
  [
    QC.testProperty "Test" (withMaxSuccess 3 test1)
  ]

As output I want to have something like:

(n,a,b) = (0,0,0)
(n,a,b) = (0,1,2)
(n,a,b) = (-1,-1,-1)
(n,a,b) = (-1,-2,-3)
(n,a,b) = (2,0,-2)
(n,a,b) = (2,1,-1)

But instead I get:

(n,a,b) = (0,0,0)
(n,a,b) = (-1,-1,-1)
(n,a,b) = (2,0,-2)

I found this post (How can QuickCheck test all properties for each sample), but it didn't really help me.

Mark Seemann Mark Seemann · Accepted Answer · 2020-01-16T10:44:33

Consider the withMaxSuccess function:

withMaxSuccess :: Testable prop => Int -> prop -> Property

This is a function that takes any Testable and turns it into a Property. The resulting Property has the behaviour that it'll run for a maximum number as specified by the Int.

The test1 function is a Testable instance. When you call withMaxSuccess 3 test1 you change the behaviour of test1, so that when it runs it runs for a maximum of three times.

While test1 returns a Property with the behaviour that it runs only two times (withMaxSuccess 2), it gets overridden by withMaxSuccess 3.

The withMaxSuccess function doesn't run the test, it just changes the definition of a property. withMaxSuccess 3 is the latest modification of the property, so that's why you see it running three times.

How to iteratively test in Haskell using QuickCheck?

1 Answers