4 After a successful build, and before installing, the libraries should be tested.
11 **Warning:** you MUST run the tests from an unprivileged account
12 (or disable your privileges temporarily if your platform allows it).
14 If some tests fail, take a look at the section Test Failures below.
19 If some tests fail, look at the output. There may be reasons for the failure
20 that isn't a problem in OpenSSL itself (like an OS malfunction or a Perl issue).
21 You may want increased verbosity, that can be accomplished like this:
23 Full verbosity, showing full output of all successful and failed test cases
24 (`make` macro `VERBOSE` or `V`):
26 $ make V=1 test # Unix
27 $ mms /macro=(V=1) test ! OpenVMS
28 $ nmake V=1 test # Windows
30 Verbosity on failed (sub-)tests only
31 (`VERBOSE_FAILURE` or `VF` or `REPORT_FAILURES`):
35 Verbosity on failed (sub-)tests, in addition progress on succeeded (sub-)tests
36 (`VERBOSE_FAILURE_PROGRESS` or `VFP` or `REPORT_FAILURES_PROGRESS`):
40 If you want to run just one or a few specific tests, you can use
41 the make variable TESTS to specify them, like this:
43 $ make TESTS='test_rsa test_dsa' test # Unix
44 $ mms/macro="TESTS=test_rsa test_dsa" test ! OpenVMS
45 $ nmake TESTS='test_rsa test_dsa' test # Windows
47 And of course, you can combine (Unix examples shown):
49 $ make test TESTS='test_rsa test_dsa' VF=1
50 $ make test TESTS="test_cmp_*" VFP=1
52 You can find the list of available tests like this:
54 $ make list-tests # Unix
55 $ mms list-tests ! OpenVMS
56 $ nmake list-tests # Windows
58 Have a look at the manual for the perl module Test::Harness to
59 see what other HARNESS_* variables there are.
61 To report a bug please open an issue on GitHub, at
62 <https://github.com/openssl/openssl/issues>.
64 For more details on how the `make` variables `TESTS` can be used,
65 see section Running Selected Tests below.
67 Running Selected Tests
68 ----------------------
70 The `make` variable `TESTS` supports a versatile set of space separated tokens
71 with which you can specify a set of tests to be performed. With a "current
72 set of tests" in mind, initially being empty, here are the possible tokens:
74 alltests The current set of tests becomes the whole set of available
75 tests (as listed when you do 'make list-tests' or similar).
77 xxx Adds the test 'xxx' to the current set of tests.
79 -xxx Removes 'xxx' from the current set of tests. If this is the
80 first token in the list, the current set of tests is first
81 assigned the whole set of available tests, effectively making
82 this token equivalent to TESTS="alltests -xxx".
84 nn Adds the test group 'nn' (which is a number) to the current
87 -nn Removes the test group 'nn' from the current set of tests.
88 If this is the first token in the list, the current set of
89 tests is first assigned the whole set of available tests,
90 effectively making this token equivalent to
91 TESTS="alltests -xxx".
93 Also, all tokens except for "alltests" may have wildcards, such as *.
94 (on Unix and Windows, BSD style wildcards are supported, while on VMS,
95 it's VMS style wildcards)
99 Run all tests except for the fuzz tests:
101 $ make TESTS=-test_fuzz test
103 or, if you want to be explicit:
105 $ make TESTS='alltests -test_fuzz' test
107 Run all tests that have a name starting with "test_ssl" but not those
108 starting with "test_ssl_":
110 $ make TESTS='test_ssl* -test_ssl_*' test
112 Run only test group 10:
114 $ make TESTS='10' test
116 Run all tests except the slow group (group 99):
118 $ make TESTS='-99' test
120 Run all tests in test groups 80 to 99 except for tests in group 90:
122 $ make TESTS='[89]? -90' test
124 To run specific fuzz tests you can use for instance:
126 $ make test TESTS=test_fuzz FUZZ_TESTS="cmp cms"
128 To stochastically verify that the algorithm that produces uniformly distributed
129 random numbers is operating correctly (with a false positive rate of 0.01%):
131 $ ./util/wrap.sh test/bntest -stochastic
133 Running Tests in Parallel
134 -------------------------
136 By default the test harness will execute the selected tests sequentially.
137 Depending on the platform characteristics, running more than one test job in
138 parallel may speed up test execution.
139 This can be requested by setting the `HARNESS_JOBS` environment variable to a
140 positive integer value. This specifies the maximum number of test jobs to run in
143 Depending on the Perl version different strategies could be adopted to select
144 which test recipes can be run in parallel. In recent versions of Perl, unless
145 specified otherwise, any task can be run in parallel. Consult the documentation
146 for `TAP::Harness` to know more.
148 To run up to four tests in parallel at any given time:
150 $ make HARNESS_JOBS=4 test