Run tests in parallel

The environment variable `HARNESS_JOBS` can be used to control how many
jobs to run in parallel.  The default is still to run jobs sequentially.

This commit does not define custom `rules`, and different versions of
`TAP::Harness` come with different strategies regarding the default
`rules` that define which test recipes can be run in parallel.
In recent versions of Perl, unless specified otherwise any task can be
run in parallel.

Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/12326)

diff --git a/test/README.md b/test/README.md
index 4db26bd047d824ba25a93028b31d553b4d684894..5c80d64762e4537059aab8210b342cb9f5b1d514 100644 (file)
--- a/test/README.md
+++ b/test/README.md
@@ -128,3 +128,22 @@ To stochastically verify that the algorithm that produces uniformly distributed
 random numbers is operating correctly (with a false positive rate of 0.01%):
 
     $ ./util/wrap.sh test/bntest -stochastic
+
+Running Tests in Parallel
+-------------------------
+
+By default the test harness will execute the selected tests sequentially.
+Depending on the platform characteristics, running more than one test job in
+parallel may speed up test execution.
+This can be requested by setting the `HARNESS_JOBS` environment variable to a
+positive integer value. This specifies the maximum number of test jobs to run in
+parallel.
+
+Depending on the Perl version different strategies could be adopted to select
+which test recipes can be run in parallel.  In recent versions of Perl, unless
+specified otherwise, any task can be run in parallel. Consult the documentation
+for `TAP::Harness` to know more.
+
+To run up to four tests in parallel at any given time:
+
+    $ make HARNESS_JOBS=4 test

diff --git a/test/run_tests.pl b/test/run_tests.pl
index 8306c6018ae0a349d6631ebfb80d6f34c3b5ad81..5eddaf846884032f1b521566bc52cf65943e9d1a 100644 (file)
--- a/test/run_tests.pl
+++ b/test/run_tests.pl
@@ -30,6 +30,7 @@ my $srctop = $ENV{SRCTOP} || $ENV{TOP};
 my $bldtop = $ENV{BLDTOP} || $ENV{TOP};
 my $recipesdir = catdir($srctop, "test", "recipes");
 my $libdir = rel2abs(catdir($srctop, "util", "perl"));
+my $jobs = $ENV{HARNESS_JOBS};
 
 $ENV{OPENSSL_CONF} = rel2abs(catdir($srctop, "apps", "openssl.cnf"));
 $ENV{OPENSSL_CONF_INCLUDE} = rel2abs(catdir($bldtop, "providers"));
@@ -44,6 +45,8 @@ my %tapargs =
       merge             => 1,
     );
 
+$tapargs{jobs} = $jobs if defined $jobs;
+
 # Additional OpenSSL special TAP arguments.  Because we can't pass them via
 # TAP::Harness->new(), they will be accessed directly, see the
 # TAP::Parser::OpenSSL implementation further down