X-Git-Url: https://git.openssl.org/?p=openssl.git;a=blobdiff_plain;f=test%2FREADME;h=37722e79f3934b8d63a2ca1aaa38414bac9d59e3;hp=8df35fe46a80f300b82c6042d5bceb9d604b2a92;hb=ab9c0d28610ce4ed4185bae325b03e2da5dd76fe;hpb=5ab4f893ce6d10e9286fd746acebe382c36cd32d

diff --git a/test/README b/test/README
index 8df35fe46a..37722e79f3 100644
--- a/test/README
+++ b/test/README
@@ -9,7 +9,7 @@ Please note that if a test involves a new testing executable, you will need to
 do some additions in test/Makefile.  More on this later.
 
 
-Naming convetions
+Naming conventions
 =================
 
 A test executable is named test/{name}test.c
@@ -19,15 +19,18 @@ digit number and {name} is a unique name of your choice.
 
 The number {nn} is (somewhat loosely) grouped as follows:
 
-05  individual symmetric cipher algorithms
-10  math (bignum)
-15  individual asymmetric cipher algorithms
-20  openssl enc
-25  certificate forms, generation and verification
-30  engine and evp
-70  PACKET layer
-80  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
-90  misc
+00-04  sanity, internal and essential API tests
+05-09  individual symmetric cipher algorithms
+10-14  math (bignum)
+15-19  individual asymmetric cipher algorithms
+20-24  openssl commands (some otherwise not tested)
+25-29  certificate forms, generation and verification
+30-35  engine and evp
+60-79  APIs
+   70  PACKET layer
+80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
+90-98  misc
+99     most time consuming tests [such as test_fuzz]
 
 
 A recipe that just runs a test executable
@@ -36,9 +39,9 @@ A recipe that just runs a test executable
 A script that just runs a program looks like this:
 
     #! /usr/bin/perl
-    
+
     use OpenSSL::Test::Simple;
-    
+
     simple_test("test_{name}", "{name}test", "{name}");
 
 {name} is the unique name you have chosen for your test.
@@ -47,7 +50,7 @@ The second argument to `simple_test' is the test executable, and `simple_test'
 expects it to be located in test/
 
 For documentation on OpenSSL::Test::Simple, do
-`perldoc test/testlib/OpenSSL/Test/Simple.pm'.
+`perldoc util/perl/OpenSSL/Test/Simple.pm'.
 
 
 A recipe that runs a more complex test
@@ -55,53 +58,92 @@ A recipe that runs a more complex test
 
 For more complex tests, you will need to read up on Test::More and
 OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
-documentation.  For OpenSSL::Test, do `perldoc test/testlib/OpenSSL/Test.pm'.
+documentation.  For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'.
 
 A script to start from could be this:
 
     #! /usr/bin/perl
-    
+
     use strict;
     use warnings;
     use OpenSSL::Test;
-    
+
     setup("test_{name}");
-    
+
     plan tests => 2;                # The number of tests being performed
-    
+
     ok(test1, "test1");
     ok(test2, "test1");
-    
+
     sub test1
     {
         # test feature 1
     }
-    
+
     sub test2
     {
         # test feature 2
     }
-    
 
-Changes to test/Makefile
-========================
+
+Changes to test/build.info
+==========================
 
 Whenever a new test involves a new test executable you need to do the
 following (at all times, replace {NAME} and {name} with the name of your
 test):
 
-* among the variables for test executables at the beginning, add a line like
-  this:
+* add {name} to the list of programs under PROGRAMS_NO_INST
+
+* create a three line description of how to build the test, you will have
+to modify the include paths and source files if you don't want to use the
+basic test framework:
+
+    SOURCE[{name}]={name}.c
+    INCLUDE[{name}]=.. ../include
+    DEPEND[{name}]=../libcrypto libtestutil.a
+
+Generic form of C test executables
+==================================
+
+    #include "testutil.h"
+
+    static int my_test(void)
+    {
+        int testresult = 0;                 /* Assume the test will fail    */
+        int observed;
+
+        observed = function();              /* Call the code under test     */
+        if (!TEST_int_equal(observed, 2))   /* Check the result is correct  */
+            goto end;                       /* Exit on failure - optional   */
+
+        testresult = 1;                     /* Mark the test case a success */
+    end:
+        cleanup();                          /* Any cleanup you require      */
+        return testresult;
+    }
+
+    int setup_tests(void)
+    {
+        ADD_TEST(my_test);                  /* Add each test separately     */
+        return 1;                           /* Indicate success             */
+    }
 
-    {NAME}TEST= {name}test
+You should use the TEST_xxx macros provided by testutil.h to test all failure
+conditions.  These macros produce an error message in a standard format if the
+condition is not met (and nothing if the condition is met).  Additional
+information can be presented with the TEST_info macro that takes a printf
+format string and arguments.  TEST_error is useful for complicated conditions,
+it also takes a printf format string and argument.  In all cases the TEST_xxx
+macros are guaranteed to evaluate their arguments exactly once.  This means
+that expressions with side effects are allowed as parameters.  Thus,
 
-* add `$({NAME}TEST)$(EXE_EXT)' to the assignment of EXE:
+    if (!TEST_ptr(ptr = OPENSSL_malloc(..)))
 
-* add `$({NAME}TEST).o' to the assignment of OBJ:
+works fine and can be used in place of:
 
-* add `$({NAME}TEST).c' to the assignment of SRC:
+    ptr = OPENSSL_malloc(..);
+    if (!TEST_ptr(ptr))
 
-* add the following lines for building the executable:
+The former produces a more meaningful message on failure than the latter.
 
-    $({NAME}TEST)$(EXE_EXT): $({NAME}TEST).o $(DLIBCRYPTO)
-           @target=$({NAME}TEST); $(BUILD_CMD)