Prepare for 1.1.0e release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove an OPENSSL_assert() and replace with a soft assert and check

Following on from CVE-2017-3733, this removes the OPENSSL_assert() check
that failed and replaces it with a soft assert, and an explicit check of
value with an error return if it fails.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't change the state of the ETM flags until CCS processing

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

The problem is caused by changing the flag indicating whether to use ETM
or not immediately on negotiation of ETM, rather than at CCS. Therefore,
during a renegotiation, if the ETM state is changing (usually due to a
change of ciphersuite), then an error/crash will occur.

Due to the fact that there are separate CCS messages for read and write
we actually now need two flags to determine whether to use ETM or not.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Provide a test for the Encrypt-Then-Mac renegotiation crash

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

This commit provides a test for the issue.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Properly zero cipher_data for ChaCha20-Poly1305 on cleanup

Fix a typo. Probably this has not been found because EVP_CIPHER_CTX is
smaller than EVP_CHACHA_AEAD_CTX and heap overflow does not occur.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2294)
(cherry picked from commit a8f957686675194d786b41f6e1f7c48bb85723ec)

crypto/armcap.c: short-circuit processor capability probe in iOS builds.

Capability probing by catching SIGILL appears to be problematic
on iOS. But since Apple universe is "monocultural", it's actually
possible to simply set pre-defined processor capability mask.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit 8653e78f4319b23d60239f9557d8c1e1d23be1a5)

ARMv4 assembly pack: harmonize Thumb-ification of iOS build.

Three modules were left behind in a285992763f3961f69a8d86bf7dfff020a08cef9.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit c93f06c12f10c07cea935abd78a07a037e27f155)

Rework error handling of custom_ext_meth_add towards strong exception safety.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2636)
(cherry picked from commit ed874fac6399d5064d6eb8fe2022b918aeaf75af)

Fix a few typos

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2571)
(cherry picked from commit 7e12cdb52e3f4beff050caeecf3634870bb9a7c4)

Remove obsolete comment

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1613)
(cherry picked from commit 7c120357e5ef434c8a7d1d1c3ba4f2a33266374e)

Use TLSEXT_KEYNAME_LENGTH in tls_decrypt_ticket.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2618)
(cherry picked from commit 57b0d651f052ed86528da916397acbcce035fb21)

Prevent allocations of size 0 in sh_init.

which are not possible with the default OPENSSL_zalloc, but are possible if
the user has installed their own allocator using CRYPTO_set_mem_functions. If
the 0-allocations succeeds, the secure heap code will later access
(at least) the first byte of that space, which is technically an OOB
access. This could lead to problems with some custom allocators that only
return a valid pointer for subsequent free()-ing, and do not expect that
the pointer is actually dereferenced.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2605)
(cherry picked from commit 7f07149d25f8d7e00e9350ff2f064a4d25c1a13d)

Make -xcert work again.

When a certificate is prepended update the list pointer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2628)
(cherry picked from commit 52f4840cb237cc37cad5eac8328828cf3d3e1049)

Add no-ec build

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2626)
(cherry picked from commit b4568b04c7cd425103ac8f1603682e8da2044238)

mem leak on error path and error propagation fix

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2559)
(cherry picked from commit e0670973d5c0b837eb5a9f1670e47107f466fbc7)

Further improvements to ASYNC_WAIT_CTX_clear_fd

Remove call to cleanup function
Use only one loop to find previous element

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit 219aa86cb04e1bfc9c156fab18da2f767502afb2)

Remove fd from the list when the engine clears the wait context before pause

This fixes the num of fds added/removed returned by ASYNC_WAIT_CTX_get_changed_fds

Previously, the numbers were not consistent with the fds actually written in
the buffers since the fds that have been both added and removed are explicitly
ignored in the loop.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f89dd6738a0ec2b6cfb05a3cc5fa38843dc27d2f)

Add test to show wrong behavior of ASYNC_WAIT_CTX

This happens when a fd is added and then immediately removed from the
ASYNC_WAIT_CTX before pausing the job.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f44e63644d29e5908be52b7896d5031a5cf460eb)

DES keys are not 7 days long.

CLA: trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2604)

(cherry picked from commit 4fd7b54dc224930a0ce6dd67b35c598c5072857c)

test_rehash does nothing, have it do something

test/recipes/40-test_rehash.t uses test files from certs/demo, which
doesn't exist any longer.  Have it use PEM files from test/ instead.

Because rehash wants only one certificate or CRL per file, we must
also filter those PEM files to produce test files with a single object
each.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2594)
(cherry picked from commit 4bbd8a5daaa810c487f684971c0339a1d7c15da9)

Restore EVP_CIPH_FLAG_LENGTH_BITS working properly

EVP_CIPH_FLAG_LENGTH_BITS flag for CFB1 has been broken with the
introduction of the is_partially_overlapping() check that did not take
it into the account (treating number of bits passed as bytes). This
remedies that and allows this flag to work as intended.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1942)
(cherry picked from commit 64846096b18340b9a39ddd29a7a0e23c56f22959)

Don't read uninitialised data for short session IDs.

While it's always safe to read |SSL_MAX_SSL_SESSION_ID_LENGTH| bytes
from an |SSL_SESSION|'s |session_id| array, the hash function would do
so with without considering if all those bytes had been written to.

This change checks |session_id_length| before possibly reading
uninitialised memory. Since the result of the hash function was already
attacker controlled, and since a lookup of a short session ID will
always fail, it doesn't appear that this is anything more than a clean
up.

In particular, |ssl_get_prev_session| uses a stack-allocated placeholder
|SSL_SESSION| as a lookup key, so the |session_id| array may be
uninitialised.

This was originally found with libFuzzer and MSan in
https://boringssl.googlesource.com/boringssl/+/e976e4349d693b4bbb97e1694f45be5a1b22c8c7,
then by Robert Swiecki with honggfuzz and MSan here. Thanks to both.

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2583)
(cherry picked from commit bd5d27c1c6d3f83464ddf5124f18a2cac2cbb37f)

Fix a typo in the X509_get0_subject_key_id() documentation

Fixes a copy&paste error

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2568)
(cherry picked from commit fbc9eeaaa32ba1416d6cb2794201f440bbaeb629)

Centralize documentation about config file location

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2569)
(cherry picked from commit e9681f8314c64c6802b11997c471bd763de38c8c)

Remove unused variable

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2566)
(cherry picked from commit a19a6c8179faa3da0dedaaf2effae385cf7dd65d)

Fix a crash in EVP_CIPHER_CTX_cleanup due to cipher_data may be NULL
or EVP_CTRL_INIT/EVP_CTRL_COPY was not called or failed.
If that happens in EVP_CipherInit_ex/EVP_CIPHER_CTX_copy set cipher = NULL,
aes_gcm_cleanup should check that gctx != NULL before calling OPENSSL_cleanse.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2562)
(cherry picked from commit 273a0218e65f1737cdbb0ef65a5ddebd601e6bef)

Fix parsing of serial# in req

Reported by Jakub Wilk.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2556)
(cherry picked from commit be4c82aa767998ce2a5717fc895482052373f1b1)

Doc fix

Reported by Alexander Köppe

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2557)
(cherry picked from commit bb6c5e7f6956c5cd1049136d79e631ca8338fc7b)

Fix a crash with malformed user notice policy numbers

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2352)
(cherry picked from commit fe4075f25962dbd302d856c11ac47adb84edc9ca)

Combined patch against OpenSSL_1_1_0-stable branch for the following issues:

Fixed a memory leak in ASN1_digest and ASN1_item_digest.
Reworked error handling in asn1_item_embed_new.
Fixed error handling in int_ctx_new and EVP_PKEY_CTX_dup.
Fixed a memory leak in CRYPTO_free_ex_data.
Reworked error handing in x509_name_ex_d2i, x509_name_encode and x509_name_canon.
Check for null pointer in tls_process_cert_verify.

Fixes #2103 #2104 #2105 #2109 #2111 #2115

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2163)

Majority rules, use session_ctx vs initial_ctx

session_ctx and initial_ctx are aliases of each other, and with the
opaque data structures, there's no need to keep both around. Since
there were more references of session_ctx, replace all instances of
initial_ctx with session_ctx.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2340)

Grouped data declarations [skip ci]
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit bc1dba209533f2033a4de0d93380fc0f485e6f7e)

Removed tab spaces.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 5d5eed4456ebc035893eedbcc4e32a9d065cecb3)

Corrections according the review comments.

Updated indentations according project rules, renamed file-local define to the shorter version - USE_RWLOCK, fixed declaration after the if statement in CRYPTO_THREAD_lock_new().
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit ec93a2921f6128ac931466ae171fba92a0eab45d)

Compile fix on platforms with missing pthread_rwlock_t.

Fix compilation on platforms with missing pthread_rwlock_t implementation by replacing it with pthread_mutex_t. An example of such platform can be Android OS 2.0 - 2.1, API level 5 (Eclair), Android NDK platform - android-5 where pthread_rwlock_t is not implemented and is missing in pthread.h.

In case of missing pthread_rwlock_t implementation CRYPTO_RWLOCK will work as exclusive lock in write-only mode of pthread_rwlock_t lock.

The implementation based on pthread_mutex_t must be using PTHREAD_MUTEX_RECURSIVE mode to be compatible with recursive behavior of pthread_rwlock_rdlock.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 2accf3f7e013c3d02312afc27cc2edbd1f149db3)

remove test/.rnd on make clean

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2344)
(cherry picked from commit 122fa088524571a3b60ebf301873f69afdac8f7a)

bn: fix occurance of negative zero in BN_rshift1()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 0a2dcb6990dacc94337f746f4f4a6dfac1fbeac4)

bn: fix occurances of negative zero

The BIGNUM behaviour is supposed to be "consistent" when going into and
out of APIs, where "consistent" means 'top' is set minimally and that
'neg' (negative) is not set if the BIGNUM is zero (which is iff 'top' is
zero, due to the previous point).

The BN_DEBUG testing (make test) caught the cases that this patch
corrects.

Note, bn_correct_top() could have been used instead, but that is intended
for where 'top' is expected to (sometimes) require adjustment after direct
word-array manipulation, and so is heavier-weight. Here, we are just
catching the negative-zero case, so we test and correct for that
explicitly, in-place.

Change-Id: Iddefbd3c28a13d935648932beebcc765d5b85ae7
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 38d1b3cc0271008b8bd130a2c4b442775b028a08)

bn: catch negative zero as an error

Change-Id: I5ab72ad0aae9069b47d5b7b7b9e25bd1b7afa251
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 2fc9b36a96ccd77cbd9ecfb3a3cdaa7ad2ca305e)

bn: fix BN_DEBUG + BN_DEBUG_RAND support

Couple of updates to make this code work properly again;
* use OPENSSL_assert() instead of assert() (and #include <assert.h>)
* the circular-dependency-avoidance uses RAND_bytes() (not pseudo)

Change-Id: Iefb5a9dd73f71fd81c1268495c54a64378955354
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 0b50ac1a0fe907f4effcf3f2f36dac32523938c5)

Fix faulty free

On error, i2o_SCT_signature() and i2o_SCT() free a pointer that may
have wandered off from the start of the allocated block (not currently
true for i2o_SCT_signature(), but has that potential as the code may
change.  To avoid this, save away the start of the allocated block and
free that instead.

Thanks to Guido Vranken for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2323)
(cherry picked from commit d85d3c993e322d3e4c3f00be2910faa8c55b40e3)

test/evp_test.c: If no algorithm was specified, don't try to check for DES

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2319)
(cherry picked from commit 31b69e9a26c5b127ce273bc5834b9e13e5e25556)

Add a couple of test to check CRL fingerprint

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 929860d0e6112f5c7766d9ea036c3f8bd8d3d719)

Document what EXFLAG_SET is for in x509v3.h

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 2d60c923141e7853c268364f26195343a5e995bf)

X509_CRL_digest() - ensure precomputed sha1 hash before returning it

X509_CRL_digest() didn't check if the precomputed sha1 hash was actually
present.  This also makes sure there's an appropriate flag to check.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 6195848b2eea627c47f74b63eb2ba3dc3d5b6436)

Correct pointer to be freed

The pointer that was freed in the SSLv2 section of ssl_bytes_to_cipher_list
may have stepped up from its allocated position.  Use a pointer that is
guaranteed to point at the start of the allocated block instead.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2312)
(cherry picked from commit 63414e64e66e376654e993ac966e3b2f9d849d3b)

Travis: run on Trusty with clang 3.9 (1.1.0 branch)

See https://github.com/travis-ci/travis-ci/issues/6460 for context on the changes to wine install.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add server signature algorithm bug test.

Add a client authentication signature algorithm to simple
ssl test and a server signature algorithm. Since we don't
do client auth this should have no effect. However if we
use client auth signature algorithms by mistake this will
abort the handshake with a no shared signature algorithms
error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2297)

Use correct signature algorithm list when sending or checking.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2297)

Prepare for 1.1.0e-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0d release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

bn/asm/x86_64-mont5.pl: fix carry bug in bn_sqr8x_internal.

CVE-2017-3732

Reviewed-by: Rich Salz <rsalz@openssl.org>

Document DH_check_params()

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Better check of DH parameters in TLS data

When the client reads DH parameters from the TLS stream, we only
checked that they all are non-zero.  This change updates the check to
use DH_check_params()

DH_check_params() is a new function for light weight checking of the p
and g parameters:

    check that p is odd
    check that 1 < g < p - 1

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

crypto/evp: harden AEAD ciphers.

Originally a crash in 32-bit build was reported CHACHA20-POLY1305
cipher. The crash is triggered by truncated packet and is result
of excessive hashing to the edge of accessible memory. Since hash
operation is read-only it is not considered to be exploitable
beyond a DoS condition. Other ciphers were hardened.

Thanks to Robert Święcki for report.

CVE-2017-3731

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/evp: harden RC4_MD5 cipher.

Originally a crash in 32-bit build was reported CHACHA20-POLY1305
cipher. The crash is triggered by truncated packet and is result
of excessive hashing to the edge of accessible memory (or bogus
MAC value is produced if x86 MD5 assembly module is involved). Since
hash operation is read-only it is not considered to be exploitable
beyond a DoS condition.

Thanks to Robert Święcki for report.

CVE-2017-3731

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove assert from is_partially_overlapping()

This function is used to validate application supplied parameters. An
assert should be used to check for an error that is internal to OpenSSL.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit b153f0921bea38127de0b9440b0487db3004330d)

Adjust in and in_len instead of donelen

Don't use the temp variable donelen in the non-aad fragmented code path.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit ef055ec536a3c6e698dd9f45b41d57a32366b4fa)

Fix the overlapping check for fragmented "Update" operations

When doing in place encryption the overlapping buffer check can fail
incorrectly where we have done a partial block "Update" operation. This
fixes things to take account of any pending partial blocks.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 7141ba31969d0b378d08104a51f8f99b9187b9d5)

Update evp_test to make sure passing partial block to "Update" is ok

The previous commit fixed a bug where a partial block had been passed to
an "Update" function and it wasn't properly handled. We should catch this
type of error in evp_test.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 0b96d77a62d8ac9a45ac1dda47560ced676b5b8d)

Properly handle a partial block in OCB mode

If we have previously been passed a partial block in an "Update" call then
make sure we properly increment the output buffer when we use it.

Fixes #2273

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 7c12c7b61c5b37c9dff930ccc68421fb7de00271)

Don't use magic numbers in aes_ocb_cipher()

Lots of references to 16 replaced by AES_BLOCK_SIZE. Also a few other style
tweaks in that function

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 0ba5a9eaa0a6ae7fc25ee70eefc1f4fbdaf09483)

s_client: Better response success check for CONNECT

Instead of looking for "200" and "established" (and failing all other
2xx responses or "Established"), let's look for a line that's not a
header (i.e. doesn't contain a ':') and where the first space is
followed by a '2'.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1664)
(cherry picked from commit ec2a0e60652c0e61e90dde367756c5d92cd882d3)

Replace div-spoiler hack with simpler code, GH#1027,2253.

This is 1.1.0-specific 8f77fab82486c19ab48eee07718e190f76e6ea9a redux.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Cleanup EVP_CIPH/EP_CTRL duplicate defines

Remove duplicate defines from EVP source files.
Most of them were in evp.h, which is always included.
Add new ones evp_int.h
EVP_CIPH_FLAG_TLS1_1_MULTIBLOCK is now always defined in evp.h, so
remove conditionals on it

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2201)
(cherry picked from commit 9d6fcd4295fef7ebc4232aab85718a99d36cc50a)

Fix error handling in compute_key, BN_CTX_get can return NULL

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2132)
(cherry picked from commit 7928ee4d685b727619555bc1ec0aee805f6fc8c4)

Do not overallocate for tmp.ciphers_raw

Well, not as much, at least.

Commit 07afdf3c3ac97af4f2b4eec22a97f7230f8227e0 changed things so
that for SSLv2 format ClientHellos we store the cipher list in the
TLS format, i.e., with two bytes per cipher, to be consistent with
historical behavior.

However, the space allocated for the array still performed the computation
with three bytes per cipher, a needless over-allocation (though a relatively
small one, all things considered).

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2281)
(cherry picked from commit f1429b85c5821e55224e5878da9d0fa420a41f71)

Fix SSL_get0_raw_cipherlist()

SSL_get0_raw_cipherlist() was a little too "raw" in the case of an SSLv2
compat ClientHello. In 1.0.2 and below, during version negotiation, if
we received an SSLv2 compat ClientHello but actually wanted to do SSLv3+
then we would construct a "fake" SSLv3+ ClientHello. This "fake" ClientHello
would have its ciphersuite list converted to the SSLv3+ format. It was
this "fake" raw list that got saved away to later be returned by a call to
SSL_get0_raw_cipherlist().

In 1.1.0+ version negotiation works differently and we process an SSLv2
compat ClientHello directly without the need for an intermediary "fake"
ClientHello. This meant that the raw ciphersuite list being saved was in
the SSLv2 format. Any caller of this function would not expect that and
potentially overread the returned buffer by one byte.

Fixes #2189

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2280)
(cherry picked from commit 07afdf3c3ac97af4f2b4eec22a97f7230f8227e0)

Add missing flags for EVP_chacha20()

ChaCha20 code uses its own custom cipher_data. Add EVP_CIPH_CUSTOM_IV
and EVP_CIPH_ALWAYS_CALL_INIT so that the key and the iv can be set by
different calls of EVP_CipherInit_ex().

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2156)
(cherry picked from commit c83680a04aac7a15e6ac48ed732b8322c4c9d49a)

Fix a ssl session leak due to OOM in lh_SSL_SESSION_insert

- s == NULL can mean c is a new session *or* lh_insert was
  unable to create a hash entry.
- use lh_SSL_SESSION_retrieve to check for this error condition.
- If it happens simply remove the extra reference again.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2138)
(cherry picked from commit 38088ce9934a90d4aea486edbff864f3935342e6)

RAND_egd_bytes: No need to check RAND_status on connection error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1886)
(cherry picked from commit c2114afc1622ff0113974b3696e557ea8bf7ffb4)

Fixed the return code for RAND_egd_bytes.

According to the documentation, the return code should be -1 when
RAND_status does not return 1.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1886)
(cherry picked from commit 3ed93c863374125296954edcbc595599adbd07ea)

Fixed the return code of RAND_query_egd_bytes when connect fails.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1886)
(cherry picked from commit 1381684daf8800487b48a70cd634f433b1d6366f)

Fix use before assignment

 it was getting the SerialNumber of a previous cert.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2272)
(cherry picked from commit 0db1fb3fc13c4b1a2b916efbb374f40579b1398f)

Extend the test_ssl_new renegotiation tests to include client auth

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

Stop server from expecting Certificate message when not requested

In a non client-auth renegotiation where the original handshake *was*
client auth, then the server will expect the client to send a Certificate
message anyway resulting in a connection failure.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

Stop client from sending Certificate message when not requested

In a non client-auth renegotiation where the original handshake *was*
client auth, then the client will send a Certificate message anyway
resulting in a connection failure.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

Fix SSL_VERIFY_CLIENT_ONCE

The flag SSL_VERIFY_CLIENT_ONCE is documented as follows:

  B<Server mode:> only request a client certificate on the initial TLS/SSL
  handshake. Do not ask for a client certificate again in case of a
  renegotiation. This flag must be used together with SSL_VERIFY_PEER.

  B<Client mode:> ignored

But the implementation actually did nothing. After the server sends its
ServerKeyExchange message, the code was checking s->session->peer to see if
it is NULL. If it was set then it did not ask for another client
certificate. However s->session->peer will only be set in the event of a
resumption, but a ServerKeyExchange message is only sent in the event of a
full handshake (i.e. no resumption).

The documentation suggests that the original intention was for this to
have an effect on renegotiation, and resumption doesn't come into it.

The fix is to properly check for renegotiation, not whether there is already
a client certificate in the session.

As far as I can tell this has been broken for a *long* time.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

Add a test to check messsages sent are the ones we expect

Repeat for various handshake types

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

Support renegotiation in TLSProxy

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

Fix a bug in TLSProxy where zero length messages were not being recorded

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1983)

fix a memory leak in ssl3_generate_key_block fix the error handling in ssl3_change_cipher_state

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2164)
(cherry picked from commit a6fd7c1dbef2c3da3c87f1582ae48e4c29aa303c)

Check the exit code from the server process

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2205)
(cherry picked from commit 57a19206b5aad7ca67f0d5ebae05af2799609819)

Fix DSA parameter generation control error

When setting the digest parameter for DSA parameter generation, the
signature MD was set instead of the parameter generation one.
Fortunately, that's also the one that was used for parameter
generation, but it ultimately meant the parameter generator MD and the
signature MD would always be the same.

Fixes github issue #2016

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2250)
(cherry picked from commit 8a05c4d3b5a1bfb9193ea24e71735e11de7168d2)

correct 3 mistakes

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2257)

(cherry picked from commit 424baabdf5af540bda4a69122d274b071d804390)

Skip ECDH tests for SSLv3

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1597)
(cherry picked from commit 4f326dd8998d5b33f72b11432f110e0f2cfcb92b)

If client doesn't send curves list, don't assume all.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1597)
(cherry picked from commit 3e37351834c203421b7f492dd83d5e5872e17778)

Clean dead macros and defines

... mostly related to some old discarded modules .

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1906)
(cherry picked from commit 31a51151fc163a7f5f4d07dff9478be50e4b5707)

Clean one unused variable, plus an useless one.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1906)
(cherry picked from commit 2191dc846a85ce82925cb06b4dd8649da7fc403c)

Correct reference to SSL_get_peer_cert_chain().

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
CLA: trivial
(Merged from https://github.com/openssl/openssl/pull/2247)
(cherry picked from commit 1f164c6fadd0eef0aa0997c51639a11276bf9072)

Fix list -disabled for blake2 alg

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2241)
(cherry picked from commit 487d3a726a1970e84853434561d88cb4ac212d15)

Fix man3 reference to CRYPTO_secure_used

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2238)

(cherry picked from commit ef3f621ed572fefe768f26989c16264496d24f69)

Add missing braces.

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #2234
(cherry picked from commit c4a60150914fc260c3fc2854e13372c870bdde76)

Fix undefined behaviour when printing the X509 and CRL version

Found by oss-fuzz

Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #2231
(cherry picked from commit c2ce477f1f3c0a98802fb087b0cf4b0a99ea2b1d)

Fix VC warnings about unary minus to an unsigned type.

Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #2230
(cherry picked from commit 68d4bcfd0651c7ea5d37ca52abc0d2e6e6b3bd20)

Fix undefined behaviour when printing the X509 serial

Found by afl

Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #2230
(cherry picked from commit 244d7b288f2b9ab7f6a2dbf068eccd6e20d9eef6)

Clarify what RUN_ONCE returns

RUN_ONCE really just returns 0 on failure or whatever the init
function returned.  By convention, however, the init function must
return 0 on failure and 1 on success.  This needed to be clarified.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2225)
(cherry picked from commit 37b8f1e21c56996644afa38816f575b05eb483ae)

Fix no-ocsp

The use of EXFLAG_SET requires the inclusion of openssl/x509v3.h.
openssl/ocsp.h does that, except when OCSP is disabled.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2227)
(cherry picked from commit d62210af2e169d0818c65ff9f20ab5276693b2bf)