Haskell Testing

Question

I am new to haskell and working on unit testing, but I think the ecosystem is very confusing. I am confused regarding the relationship between HTF and HUnit.

In some examples, I see that you set up test cases, export them to the test list, and then run in ghci using runTestsTT (for example, this HUnit example ).

In other examples, you create a test runner attached to a bonded file that uses some preprocessor magic to find your tests, as in this git example . It also seems that HTF tests should have a prefix with test_ or they do not run? It was difficult for me to find documentation on this subject, I just noticed a template that everyone had.

In any case, can someone help to figure this out? What is considered the standard way to do things in Haskell? What are the best practices? What is the easiest to configure and maintain?

Answer 1

Typically, any significant Haskell project is started using cabal. It takes care of construction, distribution, documentation (using haddock) and testing.

The standard approach is to put your tests in the test directory and then set up the test suite in your .cabal file. This is described in detail in the user manual . Here is a test suite for one of my projects that looks like

 Test-Suite test-melody type: exitcode-stdio-1.0 main-is: Main.hs hs-source-dirs: test build-depends: base >=4.6 && <4.7, test-framework, test-framework-hunit, HUnit, containers == 0.5.* 

Then in the file test/Main.hs

 import Test.HUnit import Test.Framework import Test.Framework.Providers.HUnit import Data.Monoid import Control.Monad import Utils pushTest :: Assertion pushTest = [NumLit 1] ^? push (NumLit 1) pushPopTest :: Assertion pushPopTest = [] ^? (push (NumLit 0) >> void pop) main :: IO () main = defaultMainWithOpts [testCase "push" pushTest ,testCase "push-pop" pushPopTest] mempty 

Where Utils defines nicer interfaces over HUnit .

For easier weight testing, I highly recommend using QuickCheck . This allows you to write short properties and test them against a series of random inputs. For example:

  -- Tests.hs import Test.QuickCheck prop_reverseReverse :: [Int] -> Bool prop_reverseReverse xs = reverse (reverse xs) == xs 

And then

  $ ghci Tests.hs > import Test.QuickCheck > quickCheck prop_reverseReverse .... Passed Tests (100/100) 

Answer 2

I am also a newcomer to haskeller, and I found this introduction very useful: " Getting started with HUnit ." To summarize, I will give here a simple example of testing the use of HUnit without a project .cabal file:

Suppose we have the SafePrelude.hs module:

 module SafePrelude where safeHead :: [a] -> Maybe a safeHead [] = Nothing safeHead (x:_) = Just x 

we can put the tests in TestSafePrelude.hs as follows:

 module TestSafePrelude where import Test.HUnit import SafePrelude testSafeHeadForEmptyList :: Test testSafeHeadForEmptyList = TestCase $ assertEqual "Should return Nothing for empty list" Nothing (safeHead ([]::[Int])) testSafeHeadForNonEmptyList :: Test testSafeHeadForNonEmptyList = TestCase $ assertEqual "Should return (Just head) for non empty list" (Just 1) (safeHead ([1]::[Int])) main :: IO Counts main = runTestTT $ TestList [testSafeHeadForEmptyList, testSafeHeadForNonEmptyList] 

Now it's easy to run tests with ghc :

 runghc TestSafePrelude.hs 

or hugs - in this case TestSafePrelude.hs needs to be renamed to Main.hs (as far as I am familiar with hugs) (do not forget to also change the module title):

 runhugs Main.hs 

or any other haskell compiler; -)

Of course, there is more to HUnit , so I really recommend reading the proposed tutorial and library User Guide .

Answer 3

You had the answers to most of your questions, but you also asked about HTF and how it works.

HTF is a structure designed for unit testing - it is backward compatible with HUnit (it combines and wraps it provides additional functions) - and property-based testing - it integrates with quickcheck. It uses a preprocessor to search for tests, so you do not need to manually create a list. A preprocessor is added to your test source files using a pragma:

 {-# OPTIONS_GHC -F -pgmF htfpp #-} 

(alternatively, I think you could add the same parameters to the ghc-options property in your cabal file, but I never tried this, so I don’t know if it is useful or not).

The preprocessor scans your module for top-level functions named test_xxxx or prop_xxxx and adds them to the list of tests for the module. You can use this list directly by placing the main function in the module and running them ( main = htfMain htf_thisModuleTests ) or exporting them from the module and having the main test program for several modules that imports the modules using tests and runs all of them:

 import {-@ HTF_TESTS @-} ModuleA import {-@ HTF_TESTS @-} ModuleB main :: IO () main = htfMain htf_importedTests 

This program can be integrated with cabal using the technique described by @jozefg, or loaded into ghci and launched interactively (although not on Windows - see https://github.com/skogsbaer/HTF/issues/60 for more details information).

Tasty is another alternative that provides a way to integrate different types of tests. It does not have a preprocessor such as HTF, but it has a module that performs similar functions using the Haskell template . Like HTF, it also relies on a naming convention to identify your tests (in this case case_xxxx , not test_xxxx ). In addition to the HUnit and QuickCheck tests, it also has modules for processing several other types of tests.