Exhaustive properties in Haskell QuickCheck?

Question

Consider the following QuickCheck program in Haskell

{-# LANGUAGE TemplateHaskell #-}
import Test.QuickCheck
import Test.QuickCheck.All

prop_trivial :: Bool
prop_trivial = 42 == (6 * 7)

-- Wacky boilerplate to make all tests run.
return []
runTests = $quickCheckAll
main = do
  runTests

This works with ghc version 7.8.3 and QuickCheck 2.7.6. Problem is that it repeats the test 100 times. I look around for a mitigation and find exhaustive in the QuickCheck docs here. Groovy! I change my prop_trivial to the following:

prop_trivial = exhaustive $ property $ 42 == (6 * 7)

which type-checks and compiles, but fails:

=== prop_trivial from /blahblahblah/FooTest.hs:6 ===
*** Failed! Falsifiable (after 1 test): 
False

I'm a bit stuck on how to understand and debug this result; the docs are a bit too thin for me to figure out what's going on.

Answer 1

It seems that you can use once to modify a property to run only once.

Some example code:

{-# LANGUAGE TemplateHaskell #-}
import Test.QuickCheck
import Test.QuickCheck.All

prop_trivial :: Int -> Bool
prop_trivial x = x == x
prop_trivial2 = once prop_trivial
prop_true = True
prop_true2 = once True

-- Wacky boilerplate to make all tests run.
return []
runTests = $quickCheckAll
main = do
  putStrLn $ "exhaustive prop_trivial  = " ++ show (exhaustive prop_trivial)
  putStrLn $ "exhaustive prop_trivial2 = " ++ show (exhaustive prop_trivial2)
  putStrLn $ "exhaustive prop_true     = " ++ show (exhaustive prop_true)
  putStrLn $ "exhaustive prop_true2    = " ++ show (exhaustive prop_true2)
  runTests

Output:

exhaustive prop_trivial  = False
exhaustive prop_trivial2 = False
exhaustive prop_true     = True
exhaustive prop_true2    = False
=== prop_trivial from qc2.hs:5 ===
+++ OK, passed 100 tests.

=== prop_trivial2 from qc2.hs:7 ===
+++ OK, passed 1 tests.

=== prop_true from qc2.hs:8 ===
+++ OK, passed 100 tests.

=== prop_true2 from qc2.hs:9 ===
+++ OK, passed 1 tests.

exhaustive prop only returns True if testing of prop is known to be exhaustive - note the difference between exhaustive True and exhaustive $ once True.

Answer 2

You might like smallcheck. It is not as developed as QuickCheck, but quite nice for exhaustive testing. Where QuickCheck generates random inputs, smallcheck generates its inputs systematically, running your properties on all inputs up to a given (by you) size. For types with finitely many inhabitants, this makes it pretty trivial to set up exhaustive tests, and because we're not trying to generate random stuff, it also makes it easy to avoid repeating tests. Here's a simple ghci example:

Test.SmallCheck> smallCheck 10 (6*7 == 42)
Completed 1 tests without failure.

The 10 is a maximum depth used to choose when to stop generating inputs; since there are no inputs, it's ignored in this example. For an example where the size is used, consider:

Test.SmallCheck> smallCheck 10 (\x -> x <= (10 :: Integer))
Completed 21 tests without failure.
Test.SmallCheck> smallCheck 20 (\x -> x <= (10 :: Integer))
Failed test no. 22.
there exists 11 such that
  condition is false

Answer 3

The source is really straight-forward. The default definition for exhaustive is used for the Property instance and the default is exhaustive _ = False. I think you probably want exhaustive $ 42 == (6 * 7)