From: Richard Levitte <levitte@openssl.org>
Date: Thu, 3 Sep 2015 10:00:28 +0000 (+0200)
Subject: Add documentation for the new testing framework
X-Git-Tag: OpenSSL_1_1_0-pre1~654
X-Git-Url: https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff_plain;h=5ab4f893ce6d10e9286fd746acebe382c36cd32d

Add documentation for the new testing framework

Reviewed-by: Rich Salz <rsalz@openssl.org>
---

diff --git a/CHANGES b/CHANGES
index 384abf882f..a7dab6cd12 100644
--- a/CHANGES
+++ b/CHANGES
@@ -4,6 +4,21 @@
 
  Changes between 1.0.2 and 1.1.0  [xx XXX xxxx]
 
+  *) New testing framework
+     The testing framework has been largely rewritten and is now using
+     perl and the perl modules Test::Harness and an extended variant of
+     Test::More called OpenSSL::Test to do its work.  All test scripts in
+     test/ have been rewritten into test recipes, and all direct calls to
+     executables in test/Makefile have become individual recipes using the
+     simplified testing OpenSSL::Test::Simple.
+
+     For documentation on our testing modules, do:
+
+        perldoc test/testlib/OpenSSL/Test/Simple.pm
+        perldoc test/testlib/OpenSSL/Test.pm
+
+     [Richard Levitte]
+
   *) In DSA_generate_parameters_ex, if the provided seed is too short,
      return an error
      [Rich Salz and Ismo Puustinen <ismo.puustinen@intel.com>]
diff --git a/test/README b/test/README
new file mode 100644
index 0000000000..8df35fe46a
--- /dev/null
+++ b/test/README
@@ -0,0 +1,107 @@
+How to add recipes
+==================
+
+For any test that you want to perform, you write a script located in
+test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
+{name} is a unique name of your choice.
+
+Please note that if a test involves a new testing executable, you will need to
+do some additions in test/Makefile.  More on this later.
+
+
+Naming convetions
+=================
+
+A test executable is named test/{name}test.c
+
+A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
+digit number and {name} is a unique name of your choice.
+
+The number {nn} is (somewhat loosely) grouped as follows:
+
+05  individual symmetric cipher algorithms
+10  math (bignum)
+15  individual asymmetric cipher algorithms
+20  openssl enc
+25  certificate forms, generation and verification
+30  engine and evp
+70  PACKET layer
+80  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
+90  misc
+
+
+A recipe that just runs a test executable
+=========================================
+
+A script that just runs a program looks like this:
+
+    #! /usr/bin/perl
+    
+    use OpenSSL::Test::Simple;
+    
+    simple_test("test_{name}", "{name}test", "{name}");
+
+{name} is the unique name you have chosen for your test.
+
+The second argument to `simple_test' is the test executable, and `simple_test'
+expects it to be located in test/
+
+For documentation on OpenSSL::Test::Simple, do
+`perldoc test/testlib/OpenSSL/Test/Simple.pm'.
+
+
+A recipe that runs a more complex test
+======================================
+
+For more complex tests, you will need to read up on Test::More and
+OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
+documentation.  For OpenSSL::Test, do `perldoc test/testlib/OpenSSL/Test.pm'.
+
+A script to start from could be this:
+
+    #! /usr/bin/perl
+    
+    use strict;
+    use warnings;
+    use OpenSSL::Test;
+    
+    setup("test_{name}");
+    
+    plan tests => 2;                # The number of tests being performed
+    
+    ok(test1, "test1");
+    ok(test2, "test1");
+    
+    sub test1
+    {
+        # test feature 1
+    }
+    
+    sub test2
+    {
+        # test feature 2
+    }
+    
+
+Changes to test/Makefile
+========================
+
+Whenever a new test involves a new test executable you need to do the
+following (at all times, replace {NAME} and {name} with the name of your
+test):
+
+* among the variables for test executables at the beginning, add a line like
+  this:
+
+    {NAME}TEST= {name}test
+
+* add `$({NAME}TEST)$(EXE_EXT)' to the assignment of EXE:
+
+* add `$({NAME}TEST).o' to the assignment of OBJ:
+
+* add `$({NAME}TEST).c' to the assignment of SRC:
+
+* add the following lines for building the executable:
+
+    $({NAME}TEST)$(EXE_EXT): $({NAME}TEST).o $(DLIBCRYPTO)
+           @target=$({NAME}TEST); $(BUILD_CMD)