-Running the BoringSSL test suite with OpenSSL
-=============================================
+Running external test suites with OpenSSL
+=========================================
-It is possible to integrate external test suites into OpenSSL's "make test". At
-the current time the only supported external suite is the one used by
-BoringSSL.
-
-This capability is considered a developer option and may not work on all
+It is possible to integrate external test suites into OpenSSL's "make test".
+This capability is considered a developer option and does not work on all
platforms.
+
+
+The BoringSSL test suite
+========================
+
In order to run the BoringSSL tests with OpenSSL, first checkout the BoringSSL
-source code into an appropriate directory:
+source code into an appropriate directory. This can be done in two ways:
+
+1) Separately from the OpenSSL checkout using:
+
+ $ git clone https://boringssl.googlesource.com/boringssl boringssl
-$ git clone https://boringssl.googlesource.com/boringssl boringssl
+ The BoringSSL tests are only confirmed to work at a specific commit in the
+ BoringSSL repository. Later commits may or may not pass the test suite:
-The BoringSSL tests are only confirmed to work at a specific commit in the
-BoringSSL repository. Later commits may or may not pass the test suite:
+ $ cd boringssl
+ $ git checkout 490469f850e
-$ cd boringssl
-$ git checkout 490469f850e
+2) Using the already configured submodule settings in OpenSSL:
-From the OpenSSL source code configure to use the external tests:
+ $ git submodule update --init
+
+Configure the OpenSSL source code to enable the external tests:
$ cd ../openssl
$ ./config enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers \
enable-external-tests
-Note that using other config option than those given above may cause the tests
+Note that using other config options than those given above may cause the tests
to fail.
Run the OpenSSL tests by providing the path to the BoringSSL test runner in the
verbose option:
$ VERBOSE=1 BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make \
- TESTS="test_external" test
+ TESTS="test_external_boringssl" test
Test failures and suppressions
-==============================
+------------------------------
A large number of the BoringSSL tests are known to fail. A test could fail
because of many possible reasons. For example:
The community is encouraged to contribute patches which reduce the number of
suppressions that are currently present.
+
+Python PYCA/Cryptography test suite
+===================================
+
+This python test suite runs cryptographic tests with a local OpenSSL build as
+the implementation.
+
+First checkout the PYCA/Cryptography module into ./pyca-cryptography using:
+
+$ git submodule update --init
+
+Then configure/build OpenSSL compatible with the python module:
+
+$ ./config shared enable-external-tests
+$ make
+
+The tests will run in a python virtual environment which requires virtualenv
+to be installed.
+
+$ make test VERBOSE=1 TESTS=test_external_pyca
+
+Test failures and suppressions
+------------------------------
+
+Some tests target older (<=1.0.2) versions so will not run. Other tests target
+other crypto implementations so are not relevant. Currently no tests fail.
+
+
+krb5 test suite
+===============
+
+Much like the PYCA/Cryptography test suite, this builds and runs the krb5
+tests against the local OpenSSL build.
+
+You will need a git checkout of krb5 at the top level:
+
+$ git clone https://github.com/krb5/krb5
+
+krb5's master has to pass this same CI, but a known-good version is
+krb5-1.15.1-final if you want to be sure.
+
+$ cd krb5
+$ git checkout krb5-1.15.1-final
+$ cd ..
+
+OpenSSL must be built with external tests enabled:
+
+$ ./config enable-external-tests
+$ make
+
+krb5's tests will then be run as part of the rest of the suite, or can be
+explicitly run (with more debugging):
+
+$ VERBOSE=1 make TESTS=test_external_krb5 test
+
+Test-failures suppressions
+--------------------------
+
+krb5 will automatically adapt its test suite to account for the configuration
+of your system. Certain tests may require more installed packages to run. No
+tests are expected to fail.
+
+
+Updating test suites
+====================
+
+To update the commit for any of the above test suites:
+
+- Make sure the submodules are cloned locally:
+
+ $ git submodule update --init --recursive
+
+- Enter subdirectory and pull from the repository (use a specific branch/tag if required):
+
+ $ cd <submodule-dir>
+ $ git pull origin master
+
+- Go to root directory, there should be a new git status:
+
+ $ cd ../
+ $ git status
+ ...
+ # modified: <submodule-dir> (new commits)
+ ...
+
+- Add/commit/push the update
+
+ git add <submodule-dir>
+ git commit -m "Updated <submodule> to latest commit"
+ git push
+
projects / openssl.git / blobdiff