What is unit Testing?
A test-oriented methodology for software development is most effective whent tests are easy to create, change, and execute. The JUnit tool pioneerded for test-first development in Java. OUnit is an adaptation of JUnit to OCaml.
With OUnit, as with JUnit, you can easily create tests, name them, group them into suites, and execute them, with the framework checking the results automatically.
Getting Started
The basic principle of a test suite is to have a file test.ml which will contain the tests, and an OCaml module under test, named foo.ml.
File foo.ml:
(* The functions we wish to test *)
let unity x = x;;
let funix ()= 0;;
let fgeneric () = failwith "Not implemented";;
The main point of a test is to check that the function under test has the
expected behavior. You check the behavior using assert functions. The most
simple one is OUnit.assert_equal
. This function compares the result of the
function with an expected result.
The most useful functions are:
OUnit.assert_equal
the basic assert functionOUnit.(>:::)
to define a list of tests OUnit.(>::)
to name a testOUnit.run_test_tt_main
to run the test suite you defineOUnit.bracket
that can help to set and clean environment, it is especially
useful if you deal with temporary filename.open OUnit;;
let test1 () = assert_equal "x" (Foo.unity "x");;
let test2 () = assert_equal 100 (Foo.unity 100);;
(* Name the test cases and group them together *)
let suite =
"suite">:::
["test1">:: test1;
"test2">:: test2]
;;
let _ =
run_test_tt_main suite
;;
And compile the module
$ ocamlfind ocamlc -o test -package oUnit -linkpkg -g foo.ml test.ml
A executable named "test" will be created. When run it produces the following output.
$ ./tests
..
Ran: 2 tests in: 0.00 Seconds
OK
When using OUnit.run_test_tt_main
, a non zero exit code signals that the
test suite was not successful.
Advanced usage
The topics, cover here, are only for advanced users who wish to unravel the power of OUnit.
OUnit |
Unit test building blocks
|
OUnitDiff |
Unit tests for collection of elements
|
Error reporting
The error reporting part of OUnit is quite important. If you want to identify the failure, you should tune the display of the value and the test.
Here is a list of thing you can display:
~msg
parameter: it allows you to define say which assert has failed in your
test. When you have more than one assert in a test, you should provide a
~msg
to be able to make the difference~printer
parameter: OUnit.assert_equal
allows you to define a printer for
compared values. A message "abcd" is not equal to "defg"
is better than not
equal
open OUnit;;
let _ =
"mytest">::
(fun () ->
assert_equal
~msg:"int value"
~printer:string_of_int
1
(Foo.unity 1))
;;
Command line arguments
OUnit.run_test_tt_main
already provides a set of command line argument to
help user to run only the test he wants:
-only-test
: skip all the tests except this one, you can use this flag
several time to select more than one test to run -list-test
: list all the available tests and exit-verbose
: rather than displaying dots while running the test, be more
verbose-help
: display help message and exit
open OUnit;;
let my_program_to_test =
ref None
;;
let test1 () =
match !my_program_to_test with
| Some prg ->
assert_command prg []
| None ->
skip_if true "My program is not defined"
;;
let _ =
run_test_tt_main
~arg_specs:["-my-program",
Arg.String (fun fn -> my_program_to_test := Some fn),
"fn Program to test"]
("test1" >:: test1)
;;
Skip and todo tests
Tests are not always meaningful and can even fail because something is missing in the environment. In order to manage this, you can define a skip condition that will skip the test.
If you start by defining your tests rather than implementing the functions under test, you know that some tests will just fail. You can mark these tests as to do tests, this way they will be reported differently in your test suite.
open OUnit;;
let _ =
"allfuns" >:::
[
"funix">::
(fun () ->
skip_if (Sys.os_type = "Win32") "Don't work on Windows";
assert_equal
0
(Foo.funix ()));
"fgeneric">::
(fun () ->
todo "fgeneric not implemented";
assert_equal
0
(Foo.fgeneric ()));
]
;;
Effective OUnit
This section is about general tips about unit testing and OUnit. It is the result of some years using OUnit in real world applications.
-long
and skip the tests that are too long in
your test suite according to it. When you do a release, you should use run
your long test suite. List.map
and
OUnit.(>:::)
are your friends. For example:open OUnit;;
let _ =
"Family">:::
(List.map
(fun (arg,res) ->
let title =
Printf.sprintf "%s->%s" arg res
in
title >::
(fun () ->
assert_equal res (Foo.unity arg)))
["abcd", "abcd";
"defg", "defg";
"wxyz", "wxyz"])
;;
OUnit.assert_equal
and never encounter any errors, just because the assert_equal
is not called.
In this case, if you test errors as well, you will have a missing errors as
well.open OUnit;;
let _ =
(* We need to call a function in a particular directory *)
"change-dir-and-run">::
bracket
(fun () ->
let pwd = Sys.getcwd () in
Sys.chdir "test";
pwd)
(fun _ ->
assert_command "ls" [])
(fun pwd ->
Sys.chdir pwd)
;;
In term of line of codes, a test suite can represent from 10% to 150% of the
code under test. With time, your test suite will grow faster than your
program/library. A good ratio is 33%.
Author(s): Maas-Maarten Zeeman, Sylvain Le Gall