Don't export the submodules 'boringssl', 'krb5' and 'pyca-cryptography'

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7696)

(cherry picked from commit 76bc401cc63219a462224884cb4af787e17725ed)

Don't export util/mktar.sh

When creating a tarball, it's pointless to include scripts that assume
a git workspace.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7696)

(cherry picked from commit b9a694717902af796639e1dff641ba620703303b)

Document the removed 'dist' target

Also adds missing copyright boilerplate to util/mktar.sh

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7696)

(cherry picked from commit b42922ea2f605fd6c42faad1743fb27be5f7f1f3)

VMS config: Typo fix, as -> AS

This typo prevented ia64 assembler to be compiled on VMS

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7700)

(cherry picked from commit 04375857cf4971460c473fa44bef86351f2177fe)

VMS: fix collected error strings

It turns out that on VMS, strerror() returns messages with added
spaces at the end.

We wouldn't had noticed if it wasn't for perl trimming those spaces
off for its own sake and thereby having test/recipes/02-test_errstr.t
fail on VMS.

The safe fix is to do the same trimming ourselves.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7701)

(cherry picked from commit 9f15e5b911ba6053e09578f190354568e01c07d7)

Honour mandatory digest on private key in has_usable_cert()

If the private key says it can only support one specific digest, then
don't ask it to perform a different one.

Fixes: #7348
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 2d263a4a73f852005b16359873475d48755999ad)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7609)

Stop marking default digest for EC keys as mandatory

ASN1_PKEY_CTRL_DEFAULT_MD_NID is documented to return 2 for a mandatory
digest algorithm, when the key can't support any others. That isn't true
here, so return 1 instead.

Partially fixes #7348

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit eb7eb1378cd15c4652884b3701d4c0ef27b5b8a6)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7609)

Remove all 'make dist' artifacts

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7692)

(cherry picked from commit 8d9535ec3e317641b8e551973c8cfe2ee1c89296)

Change tarball making procedure

Since recently, OpenSSL tarballs are produced with 'make tar' rather
than 'make dist', as the latter has turned out to be more troublesome
than useful.

The next step to look at is why we would need to configure at all to
produce a Makefile just to produce a tarball.  After all, the tarball
should now only contain source files that are present even without
configuring.

Furthermore, the current method for producing tarballs is a bit
complex, and can be greatly simplified with the right tools.  Since we
have everything versioned with git, we might as well use the tool that
comes with it.

Added: util/mktar.sh, a simple script to produce OpenSSL tarballs.  It
takes the options --name to modify the prefix of the distribution, and
--tarfile tp modify the tarball file name specifically.

This also adds a few entries in .gitattributes to specify files that
should never end up in a distribution tarball.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7692)

(cherry picked from commit 8c209eeef426ded66ce99048f535f35d08b88462)

Add an error message test recipes for system error messages

This ensures we collected them properly and and as completely as can
be tested safely.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7681)

(cherry picked from commit 4b801fdcf4c25f44374eb18cb18f36d904975edd)

Smarter build of system error text database

We stored copies of the system error texts in a fixed line size array,
which is a huge waste.  Instead, use a static memory pool and pack all
the string in there.  The wasted space at the end, if any, gives us
some leeway for longer strings than we have measured so far.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7681)

(cherry picked from commit 2c5b6bbb6797242f43b5a986e1c018943e5c1305)

rsa/rsa_ossl.c: cache MONT_CTX for public modulus earlier.

Blinding is performed more efficiently and securely if MONT_CTX for public
modulus is available by the time blinding parameter are instantiated. So
make sure it's the case.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7586)

(cherry picked from commit 2cc3f68cde77af23c61fbad65470602ee86f2575)

openssl s_server: don't use sendto() with connected UDP socket

Fixes #7675

On macOS, if you call `connect()` on a UDP socket you cannot then
call `sendto()` with a destination, otherwise it fails with Err#56
('socket is already connected').

By calling `BIO_ctrl_set_connected()` on the wbio we can tell it
that the socket has been connected and make it call `send()` rather
than `sendto()`.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7676)

(cherry picked from commit b92678f4e94eeec468f194333f874906a6fff7f1)

Fix wrong return value in ssl3_ctx_ctrl

This fixes issue #7677

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7678)

Travis CI: Use flake8 to find Python syntax errors or undefined names

CLA: trivial

In Travis CI, add a Python linting step that runs flake8 tests in Travis CI
to find syntax errors and undefined names. (http://flake8.pycqa.org)

__E901,E999,F821,F822,F823__ are the "_showstopper_" flake8 issues that can halt
the runtime with a SyntaxError, NameError, etc. Most other flake8 issues are
merely "style violations" -- useful for readability but they do not effect
runtime safety.

* F821: undefined name `name`
* F822: undefined name `name` in `__all__`
* F823: local variable name referenced before assignment
* E901: SyntaxError or IndentationError
* E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7410)

(cherry picked from commit 2a6f57bc098cdfa6320189f2725337c7f74a052e)

Prepare for 1.1.1b-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.1a release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7669)

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7664)

Configuration: only include shared_sources in dirinfo in shared config

Without this precaution, we end up having directory targets depend on
shlib object files for which there are no rules.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7645)

Unix build: for mingw and cygwin, create the right location for DLLs

Mingw and Cygwin builds install the DLLs in the application directory,
not the library directory, so ensure that one is created for them when
installing the DLLs.

Fixes #7653

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7654)

(cherry picked from commit 9694ebf753e571a55935a63b4df8016e7bd3248d)

sha/asm/sha512p8-ppc.pl: optimize epilogue.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7643)

(cherry picked from commit 6b956fe77b8aeb899ef7bdfa147a00bda51b804a)

sha/asm/sha512p8-ppc.pl: fix typo in prologue.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7643)

(cherry picked from commit 79d7fb990cd28bbe2888ed8f9051ab54c9f986b0)

test/siphash_internal_test.c: ensure the SIPHASH structure is zeroed

Fixes #7641

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7642)

(cherry picked from commit c7af8b0a267981c25cc42643493289a01ffe1bbd)

DSA: Check for sanity of input parameters

dsa_builtin_paramgen2 expects the L parameter to be greater than N,
otherwise the generation will get stuck in an infinite loop.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(cherry picked from commit 3afd38b277a806b901e039c6ad281c5e5c97ef67)

(Merged from https://github.com/openssl/openssl/pull/7493)

Fix no-ec and no-tls1_2

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7620)

(cherry picked from commit 65d2c16cbe0da8efed2f285f59930297326fb435)

Fix typo in util/perl/OpenSSL/Test.pm

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7633)

(cherry picked from commit 2dc37bc2b4c678462a24d2904604e58c0c5ac1cb)

test/recipes/90-test_shlibload.t needs $target{shared_extension}

We therefore must add defaults.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7626)

(cherry picked from commit 45cdae1c7c93fe7ef2a981da4c36c3b8cb09e855)

Fix rpath-related Linux "test_shlibload" failure.

When libssl and libcrypto are compiled on Linux with "-rpath", but
not "--enable-new-dtags", the RPATH takes precedence over
LD_LIBRARY_PATH, and we end up running with the wrong libraries.
This is resolved by using full (or at least relative, rather than
just the filename to be found on LD_LIBRARY_PATH) paths to the
shared objects.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7626)

(cherry picked from commit 18289399743da6c3db462f37fc8797738e8acf7c)

Configuration: make sure the shared_sources table doesn't contain empty elements

Fixes #7634

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7635)

Added missing signature algorithm reflection functions

    SSL_get_signature_nid()      -- local signature algorithm
    SSL_get_signature_type_nid() -- local signature algorithm key type
    SSL_get_peer_tmp_key()       -- Peer key-exchange public key
    SSL_get_tmp_key              -- local key exchange public key

Aliased pre-existing SSL_get_server_tmp_key(), which was formerly
just for clients, to SSL_get_peer_tmp_key().  Changed internal
calls to use the new name.

Reviewed-by: Matt Caswell <matt@openssl.org>

Merge the CA list documentation for clarity

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7503)

(cherry picked from commit 6e68dae85a8f91944370125561c7ec0d5da46c20)

Add a test for SSL_CTX_set0_CA_list()/SSL_CTX_set_client_CA_list()

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7503)

(cherry picked from commit fb8c83599e869516552f7c27bdc4dd26947fe657)

Separate ca_names handling for client and server

SSL(_CTX)?_set_client_CA_list() was a server side only function in 1.1.0.
If it was called on the client side then it was ignored. In 1.1.1 it now
makes sense to have a CA list defined for both client and server (the
client now sends it the the TLSv1.3 certificate_authorities extension).
Unfortunately some applications were using the same SSL_CTX for both
clients and servers and this resulted in some client ClientHellos being
excessively large due to the number of certificate authorities being sent.

This commit seperates out the CA list updated by
SSL(_CTX)?_set_client_CA_list() and the more generic
SSL(_CTX)?_set0_CA_list(). This means that SSL(_CTX)?_set_client_CA_list()
still has no effect on the client side. If both CA lists are set then
SSL(_CTX)?_set_client_CA_list() takes priority.

Fixes #7411

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7503)

(cherry picked from commit 98732979001dbb59320803713c4c91ba40234250)

Configure: ensure empty arrays aren't created inadvertently

Just refering to a hash table element as an array reference will
automatically create that element.  Avoid that by defaulting to
a separate empty array reference.

Fixes #7543

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7544)

(cherry picked from commit 3bed01a09071fb289484dfd265f0a8a991537282)

Test use of a brainpool ECDSA certificate

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7442)

(cherry picked from commit 24ae00388fb9e25af8f94d36b7c191ae90061586)

Add some test brainpool certificates

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7442)

(cherry picked from commit 83c81eebed52aa84b6b34d26e984c859158ca1c0)

Don't negotiate TLSv1.3 if our EC cert isn't TLSv1.3 capable

TLSv1.3 is more restrictive about the curve used. There must be a matching
sig alg defined for that curve. Therefore if we are using some other curve
in our certificate then we should not negotiate TLSv1.3.

Fixes #7435

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7442)

(cherry picked from commit de4dc598024fd0a9c2b7a466fd5323755d369522)

Fix SipHash init order.

Setting the SipHash hash size and setting its key is done with two
independent functions...  and yet, the internals depend on both.

Unfortunately, the function to change the size wasn't adapted for the
possibility that the key was set first, with a different hash size.

This changes the hash setting function to fix the internal values
(which is easy, fortunately) according to the hash size.

evpmac.txt value for digestsize:8 is also corrected.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7613)

(cherry picked from commit 425036130dfb3cfbef5937772f7526ce60133264)

Unbreak SECLEVEL 3 regression causing it to not accept any ciphers.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #7391
(cherry picked from commit 75b68c9e4e8591a4ebe083cb207aeb121baf549f)

Check return value of EVP_PKEY_new

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7427)

(cherry picked from commit d896b79b0994a35ecfd1c8e729d348d67236150e)

[crypto/bn] swap BN_FLG_FIXED_TOP too

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7599)

(cherry picked from commit dd41956d80686638d74fd203bd67060f90966280)

Fix issues with do_rand_init/rand_cleanup_int

Fixes #7022

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7588)

(cherry picked from commit e2d227bb4a25bb75354a40816439630a8162f073)

VMS build: colon after target must be separated with a space

... otherwise, it's taken to be part of a device name.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7602)

(cherry picked from commit e9994901f835420764d020968d4588fc09ec74c3)

Have install targets depend on more precise build targets

We only had the main 'install' target depend on 'all'.  This changes
the dependencies so targets like install_dev, install_runtime_libs,
install_engines and install_programs depend on build targets that are
correspond to them more specifically.  This increases the parallel
possibilities.

Fixes #7466

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7583)

(cherry picked from commit e8d01a608705e4320082a11a3870aa7e19c7290f)

Allow parallel install

When trying 'make -j{n} install', you may occasionally run into
trouble because to sub-targets (install_dev and install_runtime) try
to install the same shared libraries.  That makes parallel install
difficult.

This is solved by dividing install_runtime into two parts, one for
libraries and one for programs, and have install_dev depend on
install_runtime_libs instead of installing the shared runtime
libraries itself.

Fixes #7466

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7583)

(cherry picked from commit c1123d9f7efb005a109aeccaba82c40bf9bd4c1d)

Fix a doc-nit in EVP_PKEY_CTX_ctrl.pod

[skip-ci]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7546)

(cherry picked from commit e5a8712d03334c4b7cb9f29d6d1daee399c1223e)

rand_unix.c: open random devices on first use only

Commit c7504aeb640a (pr #6432) fixed a regression for applications in
chroot environments, which compensated the fact that the new OpenSSL CSPRNG
(based on the NIST DRBG) now reseeds periodically, which the previous
one didn't. Now the reseeding could fail in the chroot environment if the
DEVRANDOM devices were not present anymore and no other entropy source
(e.g. getrandom()) was available.

The solution was to keep the file handles for the DEVRANDOM devices open
by default. In fact, the fix did more than this, it opened the DEVRANDOM
devices early and unconditionally in rand_pool_init(), which had the
unwanted side effect that the devices were opened (and kept open) even
in cases when they were not used at all, for example when the getrandom()
system call was available. Due  to a bug (issue #7419) this even happened
when the feature was disabled by the application.

This commit removes the unconditional opening of all DEVRANDOM devices.
They will now only be opened (and kept open) on first use. In particular,
if getrandom() is available, the handles will not be opened unnecessarily.

This change does not introduce a regression for applications compiled for
libcrypto 1.1.0, because the SSLEAY RNG also seeds on first use. So in the
above constellation the CSPRNG will only be properly seeded if it is happens
before the forking and chrooting.

Fixes #7419

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7437)

(cherry picked from commit 8cfc19716c22dac737ec8cfc5f7d085e7c37f4d8)

Test: enable internal tests for shared Windows builds

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7462)

(cherry picked from commit 1901516a4ba909fff12e0e7815aa2d499f4d6d67)

Test: link drbgtest statically against libcrypto

and remove duplicate rand_drbg_seedlen() implementation again.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7462)

(cherry picked from commit 1c615e4ce97715ae3af9255bc57be32a49687966)

Give a better error if an attempt is made to set a zero length groups list

Previously we indicated this as a malloc failure which isn't very
helpful.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7479)

(cherry picked from commit 680bd131b69d57e891888ab70d300176a5a16617)

Ignore disabled ciphers when deciding if we are using ECC

use_ecc() was always returning 1 because there are default (TLSv1.3)
ciphersuites that use ECC - even if those ciphersuites are disabled by
other options.

Fixes #7471

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7479)

(cherry picked from commit 589b6227a85ea0133fe91d744b16dd72edee929a)

Add missing RAND initialisation call.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7587)

(cherry picked from commit ac765685d4b08a48cefffc71c434760045154dad)

Rename the rand_drbg_st data member "pool" to "seed_pool"

... to make the intended use more clear and differentiate
it from the data member "adin_pool".

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7575)

(cherry picked from commit 31f32abb8eb2a3ebd8500e6e0460b4a6791e5ed7)

Initialize reseed_gen_counter to 1, like it is done in master

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7532)

util/add-depends.pl: go through shared_sources too

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7545)

(cherry picked from commit 3866b2247fb7904a4e660593a16365147f479415)

Fix a race condition in drbgtest.c

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7531)

(cherry picked from commit 2bb1b5ddd12c23bbfa7fb60ee3296612ca943fef)

Avoid two memory allocations in each RAND_DRBG_bytes

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7518)

Fix error handling in RAND_DRBG_uninstantiate

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7519)

Fix error handling in drbgtest.c

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7519)

Fix error handling in rand_drbg_new

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7519)

Fix error handling in RAND_DRBG_set

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7519)

Fix return formatting.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7564)

(cherry picked from commit 2087028612027368e9508e1b253aab715a5a35d6)

Cleanse the key log buffer.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7564)

(cherry picked from commit e931f370aa38d8645b35fb8d6260cb44d37b6b61)

Restore sensible "sess_accept" counter tracking

Commit 9ef9088c1585e13b9727796f15f77da64dbbe623 switched the SSL/SSL_CTX
statistics counters to using Thread-Sanitizer-friendly primitives.
However, it erroneously converted an addition of -1
(for s->session_ctx->stats.sess_accept) to an addition of +1, since that
is the only counter API provided by the internal tsan_assist.h header
until the previous commit.  This means that for each accepted (initial)
connection, the session_ctx's counter would get doubly incremented, and the
(switched) ctx's counter would also get incremented.

Restore the counter decrement so that each accepted connection increments
exactly one counter exactly once (in net effect).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7464)

(cherry picked from commit 2aaa0b146b967397a6e61fa8df969e7847f82086)

Add tsan_decr() API, counterpart of tsan_counter()

The existing tsan_counter() API increments a reference counter.
Provide a new API, tsan_decr(), to decrement such a reference counter.
This can be used, for example, when a reference is added to the session_ctx's
sess_accept stats but should more properly be tracked in the regular ctx's
statistics.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7464)

(cherry picked from commit 95658c32436017aeeef3d8598957071baf6769a9)

crypto/engine/eng_devcrypto.c: ensure we don't leak resources

If engine building fails for some reason, we must make sure to close
the /dev/crypto handle.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

(cherry picked from commit 681e8cacdbdc44ac00af29b6656fc52745a9baa2)

crypto/engine/eng_devcrypto.c: open /dev/crypto only once

We opened /dev/crypto once for each session, which is quite unnecessary.
With this change, we open /dev/crypto once at engine init, and close
it on unload.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

(cherry picked from commit 458c7dad9e3f59490fb0908c1a27ca39e4cf38dd)

crypto/engine/eng_devcrypto.c: new compilers are strict on prototypes

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

(cherry picked from commit 28ac1bd9a97d9725273956e26d89ccfa5b4de67b)

crypto/engine/eng_devcrypto.c: add digest copy

Copying an EVP_MD_CTX, including the implementation local bits, is a
necessary operation.  In this case, though, it's the same as
initializing the local bits to be "copied to".

Fixes #7495

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

(cherry picked from commit 36af124bfb209b49cb92a5fb9fab627d9cd4a44b)

apps: Stop pretending to care about Netscape keys

The documentation says some commands care, but the code says differently.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7440)

(cherry picked from commit d91d443f0d26262148d1dc9d29f9fdf025b958ca)

Add a constant time flag to one of the bignums to avoid a timing leak.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7549)

(cherry picked from commit 00496b6423605391864fbbd1693f23631a1c5239)

Configuration: when building the dirinfo structure, include shared_sources

This makes sure that any resulting directory target in the build files
also depend on object files meant for shared libraries.

As a side effect, we move the production of the dirinfo structure from
common.tmpl to Configure, to make it easier to check the result.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7452)

(cherry picked from commit b6e660754c2e799cffe4906269fcace0e07c73bc)

Configurations/15-android.conf: detect NDK llvm-ar.

This excluses user from additional PATH adjustments in case NDK has
llvm-ar.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7443)

(cherry picked from commit ddf2e8c8ccad49cff0d6a0a0b26ca892ec4ce33a)

Configurations/15-android.conf: fix implicit __ANDROID_API__ handling.

03ad7c009e16a233c733098db3169c560142ccd3 failed if one didn't pass
explicit -D__ANDROID_API__=N :-(

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7443)

(cherry picked from commit 71f2b3171e8b94b0fbdc7fab2d7ca4ea66edec2b)

Fix a race condition in drbg_add

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7523)

(cherry picked from commit 4011bab1f85d4429bad1e9388bed90a8d0da5639)

Add a client_cert_cb test

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7413)

(cherry picked from commit 6e46c065b9b97212d63ef1f321b08fb7fa6b320d)

Don't call the client_cert_cb immediately in TLSv1.3

In TLSv1.2 and below a CertificateRequest is sent after the Certificate
from the server. This means that by the time the client_cert_cb is called
on receipt of the CertificateRequest a call to SSL_get_peer_certificate()
will return the server certificate as expected. In TLSv1.3 a
CertificateRequest is sent before a Certificate message so calling
SSL_get_peer_certificate() returns NULL.

To workaround this we delay calling the client_cert_cb until after we
have processed the CertificateVerify message, when we are doing TLSv1.3.

Fixes #7384

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7413)

(cherry picked from commit e45620140fce22c3251440063bc17440289d730c)

rand_drbg.h: include <openssl/obj_mac.h>

The RAND_DRBG_TYPE preprocessor define depends on a NID, so  we have
to include obj_mac.h to make the header selfcontained.

Fixes #7521

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7524)

(cherry picked from commit cf3d6ef7af7d6c47f5ccd4ce58d822972018b21c)

Fix MSVC warning C4819

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7444)

(cherry picked from commit cf4eea12046445fc418507d2d5e14956b4353495)

test/evp_test.c: don't misuse pkey_test_ctrl() in mac_test_run()

pkey_test_ctrl() was designed for parsing values, not for using in
test runs.  Relying on its returned value when it returned 1 even for
control errors made it particularly useless for mac_test_run().

Here, it gets replaced with a MAC specific control function, that
parses values the same way but is designed for use in a _run() rather
than a _parse() function.

This uncovers a SipHash test with an invalid control that wasn't
caught properly.  After all, that stanza is supposed to test that
invalid control values do generate an error.  Now we catch that.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7500)

(cherry picked from commit ce5d64c79c4d809ece8fe28a5b62915467a1c0e1)

Add a test where we reuse the EVP_PKEY_CTX for two HKDF test runs

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7501)

(cherry picked from commit 10d5b415f9e973f44f18eeaf2713868ec813e1d7)

Reset the HKDF state between operations

Fixes #7497

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7501)

(cherry picked from commit ca55d70be031746daddd8bd0611db54ed81f1737)

Windows build: build foo.d after foo.obj

We made the build of foo.obj depend on foo.d, meaning the latter gets
built first.  Unfortunately, the way the compiler works, we are forced
to redirect all output to foo.d, meaning that if the source contains
an error, the build fails without showing those errors.

We therefore remove the dependency and force the build of foo.d to
always happen after build of foo.obj.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7469)

(cherry picked from commit ecc347f5f53a9f2edc2805d50cba07db64267e8a)

ssl/statem: Don't compare size_t with less than zero

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7194)

(cherry picked from commit 60690b5b8396d7d5234cd067206190fb8aca78d2)

VMS & cryptoerr.h: include symhacks.h

Needed to clear a clash between ERR_load_CRYPTO_strings and
ERR_load_crypto_strings

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7194)

(cherry picked from commit cc3a2e4f51679d99507a979f9a920f6068473aa2)

apps/rehash.c: Convert ISO-8859-1 to UTF-8

Believe it or not, the VMS C compiler is remarking on this

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7194)

(cherry picked from commit 4602cc85aff35b7aa5e27dc57ead89f0867637f9)

Rework and simplify resource flow in drbg_add

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7504)

(cherry picked from commit f9e43929c46b38667f67e02765fe0f1c0d3061d6)

randfile.c: fix a Coverity warning

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7510)

(cherry picked from commit 040a03470c7c5bf95fe8e6143db7bef357a22833)

Timing vulnerability in ECDSA signature generation (CVE-2018-0735)

Preallocate an extra limb for some of the big numbers to avoid a reallocation
that can potentially provide a side channel.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7486)

(cherry picked from commit 99540ec79491f59ed8b46b4edf130e17dc907f52)

Timing vulnerability in DSA signature generation (CVE-2018-0734).

Avoid a timing attack that leaks information via a side channel that
triggers when a BN is resized.  Increasing the size of the BNs
prior to doing anything with them suppresses the attack.

Thanks due to Samuel Weiser for finding and locating this.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7486)

(cherry picked from commit a9cfb8c2aa7254a4aa6a1716909e3f8cb78049b6)

DSA mod inverse fix

There is a side channel attack against the division used to calculate one of
the modulo inverses in the DSA algorithm.  This change takes advantage of the
primality of the modulo and Fermat's little theorem to calculate the inverse
without leaking information.

Thanks to Samuel Weiser for finding and reporting this.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7487)

(cherry picked from commit 415c33563528667868c3c653a612e6fc8736fd79)

drbg_lib: avoid NULL pointer dereference in drbg_add

Found by Coverity Scan

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7511)

(cherry picked from commit 59f90557dd6e35cf72ac72016609d759ac78fcb9)

iOS config targets: disable "async" by default

This also gave enough reason to collect the stuff that's common for
all iOS config targets into the template "ios-common".

Fixes #7318

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7403)

(cherry picked from commit 04e3f9a114c2c142356ef1639d68397a72e0c7f8)

Configure: Improve warning if no random seed source was configured

The new Configure summary box (41349b5e6db) now hides the warning
about the missing seed source (2805ee1e095) too much. To make it
more visible again, add warning markers.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7509)

(cherry picked from commit caa8595276c60f009c8621ad466338d2ae39fb86)

Fix data race in RAND_DRBG_generate

Fixes #7394

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7399)

(cherry picked from commit a83dc59afa2e0207180d7218efed19b20d48de95)

RAND_add()/RAND_seed(): fix failure on short input or low entropy

Commit 5b4cb385c18a (#7382) introduced a bug which had the effect
that RAND_add()/RAND_seed() failed for buffer sizes less than
32 bytes. The reason was that now the added random data was used
exlusively as entropy source for reseeding. When the random input
was too short or contained not enough entropy, the DRBG failed
without querying the available entropy sources.

This commit makes drbg_add() act smarter: it checks the entropy
requirements explicitely. If the random input fails this check,
it won't be added as entropy input, but only as additional data.
More precisely, the behaviour depends on whether an os entropy
source was configured (which is the default on most os):

- If an os entropy source is avaible then we declare the buffer
  content as additional data by setting randomness to zero and
  trigger a regular   reseeding.

- If no os entropy source is available, a reseeding will fail
  inevitably. So drbg_add() uses a trick to mix the buffer contents
  into the DRBG state without forcing a reseeding: it generates a
  dummy random byte, using the buffer content as additional data.

Related-to: #7449
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7456)

(cherry picked from commit 8817215d5c52a76f2b184b624bde4df8556dee6d)

Backport some DRBG renamings and typo fixes

In commit 8bf366519661 some renamings andd typo fixes were made
while adding back the DRBG-HMAC and DRBG-HASH implementation.
Since the commit could not be backported, a lot of unnecessary
differences between master and 1.1.1 were introduced.

These differences result in tiresome merge conflicts when
cherry-picking. To minimize these merge-conflicts, this patch
ports all 'non-feature' changes of commit 8bf366519661
(e.g., renamings of private variables, fixes of typographical
errors, comment changes) manually back to 1.1.1.

The commits a83dc59afa2e (#7399) and 8817215d5c52 (#7456)
failed to cherry-pick previously to 1.1.1, with this patch
they both cherry-pick without conflicts.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7505)

Add a test for duplicated DTLS records

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7414)

(cherry picked from commit f1358634af5b84be22cb20fff3dcb613f5f8c978)

Properly handle duplicated messages from the next epoch

Since 1fb9fdc30 we may attempt to buffer a record from the next epoch
that has already been buffered. Prior to that this never occurred.

We simply ignore a failure to buffer a duplicated record.

Fixes #6902

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7414)

(cherry picked from commit 840facc3ccab481e1a0bc2cb6e7740e362df7422)