Make the Boring tests pass

The boring tests are currently failing because they send a PSK extension
which isn't in the last place. This is not allowed in the latest TLS1.3
specs. However the Boring tests we have are based on an old commit that
pre-date when that rule first appeared.

The proper solution is to update the tests to a later commit. But for now
to get travis to go green we disable the failing tests.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2928)

Add the presence of ARIA to the change log.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2926)

Added a test case for RSA_padding_add_PKCS1_PSS_mgf1.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2801)

Fix a crash or unbounded allocation in RSA_padding_add_PKCS1_PSS_mgf1
and RSA_verify_PKCS1_PSS_mgf1 with 512-bit RSA vs. sha-512.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2801)

Better way to recognise mingw64 in config script

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2922)

poly1305/asm/poly1305-x86_64.pl: add poly1305_blocks_vpmadd52_4x.

As hinted by its name new subroutine processes 4 input blocks in
parallel. It still operates on 256-bit registers and is just
another step toward full-blown AVX512IFMA procedure.

Reviewed-by: Rich Salz <rsalz@openssl.org>

poly1305/asm/poly1305-armv8.pl: ilp32-specific poly1305_init fix.

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/x86*cpuid.pl: move extended feature detection.

Exteneded feature flags were not pulled on AMD processors, as result
a number of extensions were effectively masked on Ryzen. Original fix
for x86_64cpuid.pl addressed this problem, but messed up processor
vendor detection. This fix moves extended feature detection past
basic feature detection where it belongs. 32-bit counterpart is
harmonized too.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Document in CHANGES that config now recognises 64-bit mingw

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2917)

Recognise mingw64 in config script

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2914)

Fix some doc nits

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2909)

Fix out-of-memory condition in conf

conf has the ability to expand variables in config files. Repeatedly doing
this can lead to an exponential increase in the amount of memory required.
This places a limit on the length of a value that can result from an
expansion.

Credit to OSS-Fuzz for finding this problem.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2894)

Handle find-doc-nits script rename

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix many doc L<> errors

Add 2017 copyright year
Add missing typedef to NAME
Remove ec(7) and bn(7) doc links
Remove .pod link errors, bogus links, make a few typo corrections
Fix some typo's in links and some missing items.
Don't link to C runtime functions (See OPENSSL_malloc for example/precedent)
Document ASN1_tag2str(), add a few typedef's that were missing from NAME
Update doc-nits target; addresses
    https://github.com/openssl/openssl/pull/1900#issuecomment-259943891,
Merge check-doc-links into find-doc-nits; if run regularly, would have found
    https://github.com/openssl/openssl/pull/2825

Reviewed-by: Richard Levitte <levitte@openssl.org>
    (Merged from https://github.com/openssl/openssl/pull/2862)

UI docs: Rephrase the UI method function return value description

It seems the =item isn't supposed to have pure numbers, or so tells me
perldoc.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2906)

Fix UI_get0_action_string()

It shouldn't try to return an action description for UIT_PROMPT type
UI strings.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2904)

Document UI_METHOD and UI_STRING, both useful for UI_METHOD creators

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2903)

Use the callbacks from the SSL object instead of the SSL_CTX object

... in functions dealing with the SSL object rather than the context.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2870)

Encourage having external tests in multiple test recipes

This will make the individual external tests more easily selectable /
deselectable through the usual test selection mechanism.

This also moves external tests to group 95.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2902)

Avoid questionable use of the value of a pointer

that refers to space
deallocated by a call to the free function in tls_decrypt_ticket.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2897)
(cherry picked from commit 13ed1afa923f4ffb553e389de08f26e9ce84e8a2)

Split test/recipes/03_test_internal.t into individual tests

This allows a finer granularity when selecting which tests to run, and
makes the tests more vidible.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2901)

Document how to select / deselect test group numbers

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2892)

Add HelloRetryRequest text to s_client/s_server

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2900)

Fix some RSA documentation

RSA_private_encrypt(), RSA_public_decrypt(), RSA_public_encrypt() and
RSA_private_decrypt() are declared with a "const" from parameter, but
this is not reflected in the docs.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2899)

Use the new TLSv1.3 certificate_required alert where appropriate

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2898)

Add a test to check that if a PSK extension is not last then we fail

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2896)

Check that the PSK extension is last

We need to check that the PSK extension in a ClientHello is the last one.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2896)

Fixup previous merge.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2703)

Make it possible to select or deselect test groups by number

Examples of possible expressions (adapt to your platform):

    make test TESTS=-99
    make test TESTS=10
    make test TESTS=-9?
    make test TESTS=-[89]0
    make test TESTS=[89]0

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2887)

Make the output of enc -ciphers identical even if run several times in a session.

This amounts to moving the column counter so it isn't a function local static variable and reinitialising it each time.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2884)

Exit the loop on failure

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2805)

Add some TLS13 values to s_client/s_server

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2882)

Limit the output of the enc -ciphers command to just the ciphers enc can
process.  This means no AEAD ciphers and no XTS mode.

Update the test script that uses this output to test cipher suites to not
filter out the now missing cipher modes.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2876)

this change will prevent undefined behavior when src and dst are equal (memcpy), effectively allowing setting length only in both functions.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2750)

Fix no-comp

The value of SSL3_RT_MAX_ENCRYPTED_LENGTH normally includes the compression
overhead (even if no compression is negotiated for a connection). Except in
a build where no-comp is used the value of SSL3_RT_MAX_ENCRYPTED_LENGTH does
not include the compression overhead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2872)

Fix the number of tests to skip if TLSv1.3 is disabled

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2875)

Add a test for records not on the record boundary

Test that we check that key change messages appear on a record boundary.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2875)

Check TLSv1.3 ServerHello, Finished and KeyUpdates are on record boundary

In TLSv1.3 the above messages signal a key change. The spec requires that
the end of these messages must align with a record boundary. We can detect
this by checking for decrypted but as yet unread record data sitting in
OpenSSL buffers at the point where we process the messages.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2875)

Provide a function to test whether we have unread records pending

Also updates SSL_has_pending() to use it. This actually fixes a bug in
SSL_has_pending() which is supposed to return 1 if we have any processed
or unprocessed data sitting in OpenSSL buffers. However it failed to return
1 if we had processed non-application data pending.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2875)

Remove doc reference to non-existant GCM example

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2866)

Get pointer type right in BIO_ssl_shutdown()

Also, restore 1.0.2 behavior of looping over all BIO's in the chain.
Thanks to Joseph Bester for finding this and suggesting a fix to the
crash.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2651)

crypto/x86_64cpuid.pl: move extended feature detection upwards.

Exteneded feature flags were not pulled on AMD processors, as result a
number of extensions were effectively masked on Ryzen. It should have
been reported for Excavator since it implements AVX2 extension, but
apparently nobody noticed or cared...

Reviewed-by: Rich Salz <rsalz@openssl.org>

test: add chacha_internal_test.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Increase the password buffer size to APP_PASS_LEN.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2868)

Unix Makefile: Have manual generation use the same perl script as Windows and VMS

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2863)

util/process_docs.pl: make it possible to add a suffix to man docs

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2863)

Add a test to check that we correctly handle record overflows

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2861)

Tweak the TLSv1.3 record overflow limits

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2861)

Fix double free in cookie generation.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #2850

Remove some duplicate manpage entries

[skip ci]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2860)

Add documentation on platform specific checks

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2851)

Add a platform specific configuration checker

For each platform, we may need to perform some basic checks to see
that available tools perform as we expect them.

For the moment, the added checkers test that Perl gives the expected
path format.  This should help MingW users to see if they run an
appropriate Perl implementation, for example.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2851)

Fix an endless loop in rsa_builtin_keygen.

And add a test case.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2757)

Fix no-ec

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2858)

Fix a test failure with no-tls1_1

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2857)

Fix no-psk

Fixes #2847

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2856)

Don't call memcmp with a NULL pointer

If early data is sent to a server, but ALPN is not used then memcmp is
called with a NULL pointer which is undefined behaviour.

Fixes #2841

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2845)

Add a test for TLSv1.3 cookies

We just check that if we insert a cookie into an HRR it gets echoed back
in the subsequent ClientHello.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2839)

Add basic TLSv1.3 cookie support

We do not allow the generation of TLSv1.3 cookies. But if we receive one
in an HRR we will echo it back in the ClientHello.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2839)

appveyor.yml: call upon cmd to redirect stderr.

If stderr is not redirected an uncatchable exception is thrown.

Reviewed-by: Rich Salz <rsalz@openssl.org>

apps/s_client.c: harden ldap_ExtendedResponse_parse.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a memory leak on an error path

A leak of an SSL_SESSION object can occur when decoding a psk extension on
an error path when using TLSv1.3

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2843)

Ensure we don't call memcpy with a NULL pointer

Commit d5aa14dd simplified the bn_expand_internal() and BN_copy() functions.
Unfortunately it also removed some checks which are still required,
otherwise we call memcpy passing in NULL which is not allowed.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2836)

Update and add test

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2840)

Set specific error is we have no valid signature algorithms set

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2840)

Signature algorithm enhancement.

Change tls12_sigalg_allowed() so it is passed a SIGALG_LOOKUP parameter,
this avoids multiple lookups.

When we copy signature algorithms return an error if no valid TLS message
signing algorithm is present. For TLS 1.3 this means we need at least one
signature algorithm other than RSA PKCS#1 or SHA1 both of which can only be
used to sign certificates and not TLS messages.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2840)

Disallow zero length signature algorithms

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2840)

Don't allow DSA for TLS 1.3

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2840)

use OSSLzu instead of lu format for size_t display

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2660)

Reset executable bits on files where not needed.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2835)

Update the cipher(1) documentation to explicitly state that the RSA cipher
string means the same a kRSA.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2821)

sh_malloc & sh_free prototype change to match POSIX

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2823)

Silence some more clang warnings

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2832)

Silence some clang warnings

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2831)

More early data documentation updates following feedback

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Update the API documentation for the latest early data changes

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Update early data API for writing to unauthenticated clients

Change the early data API so that the server must use
SSL_write_early_data() to write to an unauthenticated client.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Rename SSL_write_early() to SSL_write_early_data()

This is for consistency with the rest of the API where all the functions
are called *early_data*.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Rename SSL_read_early() to SSL_read_early_data()

This is for consistency with the rest of the API where all the functions
are called *early_data*.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Updates to the early data documentation

Following on from the latest API changes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Updates to s_server and s_client for the latest early_data API changes

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Make SSL_write_early_finish() an internal only function

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Add early_data tests

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Various fixes required to allow SSL_write/SSL_read during early data

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Enable the server to call SSL_write() without stopping the ability to call SSL_read_early()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Enable the client to call SSL_read() without stopping the ability to call SSL_write_early()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Introduce a new early_data state in the state machine

Also simplifies the state machine a bit.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Improve the early data sanity check in SSL_do_handshake()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Add documentation for the new s_client and s_server early_data options

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Tighten sanity checks when calling early data functions

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Add documentation for the early data functions

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Make SSL_get_early_data_status() take a const

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Make SSL_get_max_early_data() and SSL_CTX_get_max_early_data() take a const

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Add a SSL_SESSION_get_max_early_data() function

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Don't attempt to write more early_data than we know the server will accept

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Only accept early_data if the negotiated ALPN is the same

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Skip early_data if appropriate after a HelloRetryRequest

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Don't accept early_data if we are going to issue a HelloRetryRequest

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Add extra validation parsing the server-to-client early_data extension

Check that we actually resumed the session, and that we selected the first
identity.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)

Remove some TLSv1.3 TODOs that are no longer relevant

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2737)