ecdsa/ecs_ossl.c: revert blinding in ECDSA signature.

Originally suggested solution for "Return Of the Hidden Number Problem"
is arguably too expensive. While it has marginal impact on slower
curves, none to ~6%, optimized implementations suffer real penalties.
Most notably sign with P-256 went more than 2 times[!] slower. Instead,
just implement constant-time BN_mod_add_quick.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 3fc7a9b96cbed0c3da6f53c08e34d8d0c982745f)

Resolved onflicts:
crypto/ec/ecdsa_ossl.c
crypto/include/internal/bn_int.h

bn/bn_{mont|exp}.c: switch to zero-padded intermediate vectors.

Note that exported functions maintain original behaviour, so that
external callers won't observe difference. While internally we can
now perform Montogomery multiplication on fixed-length vectors, fixed
at modulus size. The new functions, bn_to_mont_fixed_top and
bn_mul_mont_fixed_top, are declared in bn_int.h, because one can use
them even outside bn, e.g. in RSA, DSA, ECDSA...

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 71883868ea5b33416ae8283bcc38dd2d97e5006b)

Resolved conflicts:
crypto/bn/bn_exp.c
crypto/bn/bn_lcl.h
crypto/bn/bn_mont.c
crypto/include/internal/bn_int.h

bn/bn_lib.c: add BN_FLG_FIXED_TOP flag.

The new flag marks vectors that were not treated with bn_correct_top,
in other words such vectors are permitted to be zero padded. For now
it's BN_DEBUG-only flag, as initial use case for zero-padded vectors
would be controlled Montgomery multiplication/exponentiation, not
general purpose. For general purpose use another type might be more
appropriate. Advantage of this suggestion is that it's possible to
back-port it...

bn/bn_div.c: fix memory sanitizer problem.
bn/bn_sqr.c: harmonize with BN_mul.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 305b68f1a2b6d4d0aa07a6ab47ac372f067a40bb)

Resolved conflicts:
crypto/bn/bn_lcl.h
crypto/bn/bn_lib.c

bn/bn_mont.c: improve readability of post-condition code.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 6c90182a5f87af1a1e462536e7123ad2afb84c43)

bn/bn_lib.c: remove bn_check_top from bn_expand2.

Trouble is that addition is postponing expansion till carry is
calculated, and if addition carries, top word can be zero, which
triggers assertion in bn_check_top.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit e42395e637c3507b80b25c7ed63236898822d2f1)

Resolved conflicts:
crypto/bn/bn_lib.c

Fix inconsistent use of bit vs bits

Reviewed-by: Tim Hudson <tjh@openssl.org>
GH: #6794
(cherry picked from commit b9e54e98066c1ff8adab5d68b6c114b14d2f74e5)

Make number of Miller-Rabin tests for a prime tests depend on the security level of the prime

The old numbers where all generated for an 80 bit security level. But
the number should depend on security level you want to reach. For bigger
primes we want a higher security level and so need to do more tests.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #6075
Fixes: #6012
(cherry picked from commit feac7a1c8be49fbcb76fcb721ec9f02fdd91030e)

Change the number of Miller-Rabin test for DSA generation to 64

This changes the security level from 100 to 128 bit.
We only have 1 define, this sets it to the highest level supported for
DSA, and needed for keys larger than 3072 bit.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #6075
(cherry picked from commit 74ee379651fb2bb12c6f7eb9fa10e70be89ac7c8)

Check for failures, to avoid memory leak

Thanks to Jiecheng Wu, Zuxing Gu for the report.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6791)

(cherry picked from commit 037241bf046be8cfc7e9216959393dd20b06fc21)
(Only the EC part)

ec/ecp_nistz256.c: fix ecp_nistz256_set_from_affine.

ecp_nistz256_set_from_affine is called when application attempts to use
custom generator, i.e. rarely. Even though it was wrong, it didn't
affect point operations, they were just not as fast as expected.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6761)

PKCS12: change safeContentsBag from a SET OF to a SEQUENCE OF

As per RFC 7292.

Fixes #6665

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/6708)

(cherry picked from commit b709babbca0498cd2b05f543b09f57f4a670298e)

Documentation typo fix in BN_bn2bin.pod

Change the description for BN_hex2bn() so that it uses the same BIGNUM argument name as its prototype.

CLA: trivial

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6712)

Don't create an invalid CertificateRequest

We should validate that the various fields we put into the
CertificateRequest are not too long. Otherwise we will construct an
invalid message.

Fixes #6609

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6629)

Zero-fill IV by default.

Fixes uninitialized memory read reported by Nick Mathewson

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6603)
(cherry picked from commit 10c3c1c1ec41ce16e51b92bb18fab92d1a42b49c)

Fix some more gcc-9 warnings [-Wstringop-truncation]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6586)

Fix a new gcc-9 warning [-Wstringop-truncation]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6581)

(cherry picked from commit dc6c374bdb4872f6d5d727e73a2ed834e972842c)

Improve the example getpass() implementation to show an error return

Also, modernize the code, so that it isn't trying to store a size_t
into an int, and then check the int's sign. :/

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6271)

(cherry picked from commit c8c250333cd254ab3f4d709ebc5ed86a7c065721)
(cherry picked from commit 50d06d1c7d2682b0042e921a76beb509d7ea68e1)

doc/crypto/pem.pod: modernise the example code

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6552)

Update documentation for PEM callback: error is now -1.

In previous versions of OpenSSL, the documentation for PEM_read_*
said:

   The callback B<must> return the number of characters in the
   passphrase or 0 if an error occurred.

But since c82c3462267afdbbaa5, 0 is now treated as a non-error
return value.  Applications that want to indicate an error need to
return -1 instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6271)

(cherry picked from commit bbbf752a3c8b5a966bcb48fc71a3dc03832e7b27)
(cherry picked from commit e4b47f7f19392e3be604e44f6999de2bc9e7ecf3)

Add blinding to a DSA signature

This extends the recently added ECDSA signature blinding to blind DSA too.

This is based on side channel attacks demonstrated by Keegan Ryan (NCC
Group) for ECDSA which are likely to be able to be applied to DSA.

Normally, as in ECDSA, during signing the signer calculates:

s:= k^-1 * (m + r * priv_key) mod order

In ECDSA, the addition operation above provides a sufficient signal for a
flush+reload attack to derive the private key given sufficient signature
operations.

As a mitigation (based on a suggestion from Keegan) we add blinding to
the operation so that:

s := k^-1 * blind^-1 (blind * m + blind * r * priv_key) mod order

Since this attack is a localhost side channel only no CVE is assigned.

This commit also tweaks the previous ECDSA blinding so that blinding is
only removed at the last possible step.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6524)

Fix no-ssl3-method in 1.0.2

Fixes #5322

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6510)

sha/asm/sha{1|256}-586.pl: harmonize clang version detection.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6499)

(cherry picked from commit b55e21b357902959ae8ec0255952402f5ccaa515)

bn/asm/rsaz-avx2.pl: harmonize clang version detection.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6499)

(cherry picked from commit 9e97f61dec312084abe03226e5c962d818c9fc2b)

Warn against nonce reuse in DSA_sign_setup() doc

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6465)

Add blinding to an ECDSA signature

Keegan Ryan (NCC Group) has demonstrated a side channel attack on an
ECDSA signature operation. During signing the signer calculates:

s:= k^-1 * (m + r * priv_key) mod order

The addition operation above provides a sufficient signal for a
flush+reload attack to derive the private key given sufficient signature
operations.

As a mitigation (based on a suggestion from Keegan) we add blinding to
the operation so that:

s := k^-1 * blind^-1 (blind * m + blind * r * priv_key) mod order

Since this attack is a localhost side channel only no CVE is assigned.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reject excessively large primes in DH key generation.

CVE-2018-0732

Signed-off-by: Guido Vranken <guidovranken@gmail.com>
(cherry picked from commit 91f7361f47b082ae61ffe1a7b17bb2adf213c7fe)

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6457)

bn/asm/sparcv9-mont.pl: iron another glitch in squaring code path.

This module is used only with odd input lengths, i.e. not used in normal
PKI cases, on contemporary processors. The problem was "illuminated" by
fuzzing tests.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6440)

(cherry picked from commit f55ef97b5c0f8559f393b72ebd4b2de32ad6d231)

Document failure return of ECDSA_SIG_new

ECDSA_SIG_new() returns NULL on error.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6398)
(cherry picked from commit 6da34cfbddede5e46f9c9183b724c99999dcfb41)

ENGINE_pkey_asn1_find_str(): don't assume an engine implements ASN1 method

Just because an engine implements algorithm methods, that doesn't mean
it also implements the ASN1 method.  Therefore, be careful when looking
for an ASN1 method among all engines, don't try to use one that doesn't
exist.

Fixes #6381

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6383)

apps: when the 'compat' nameopt has been set, leave it be

XN_FLAG_COMPAT has a unique property, its zero for value.  This means
it needs special treatment; if it has been set (which can only be
determined indirectly) and set alone (*), no other flags should be
set.

(*) if any other nameopt flag has been set by the user, compatibility
mode is blown away.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6382)

(cherry picked from commit 3190d1dca43ecfd748c06aa06752de06af3768b9)

The result of a ^ 0 mod -1 is 0 not 1

Thanks to Guido Vranken and OSSFuzz for finding this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6355)

(cherry picked from commit 4aa5b725d549b3ebc3a4f2f1c44e44a11f68752b)

Fix a bogus warning about an uninitialised var

The variable in question can never be used uninitialised, but we silence
the compiler anyway.

Fixes #6301

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6329)

Fix undefined behaviour in X509_NAME_cmp()

If the lengths of both names is 0 then don't attempt to do a memcmp.

Issue reported by Simon Friedberger, Robert Merget and Juraj Somorovsky.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6291)

(cherry picked from commit 511190b691183a1fb160e7e05e2974dc73cab0c6)

util/domd: harmonize with compiler detection in Configure.

Configure detects if compiler can generate dependency lists for any
command name, while this script didn't quite tolerate arbitrary one.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6261)

Make BN_GF2m_mod_arr more constant time

Experiments have shown that the lookup table used by BN_GF2m_mod_arr
introduces sufficient timing signal to recover the private key for an
attacker with access to cache timing information on the victim's host.
This only affects binary curves (which are less frequently used).

No CVE is considered necessary for this issue.

The fix is to replace the lookup table with an on-the-fly calculation of
the value from the table instead, which can be performed in constant time.

Thanks to Youngjoo Shin for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6270)

(cherry picked from commit b336ce57f2d5cca803a920d2a9e622b588cead3c)

When configuring 'no-comp', zlib support should be disabled too

Fixes #6241

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6248)

UI console: Restore tty settings, do not force ECHO after prompt

The Console UI method always set echo on after prompting without
echo.  However, echo might not have been on originally, so just
restore the original TTY settings.

Fixes #2373

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6158)

Fix memory leaks in CA related functions.

(cherry picked from commit aebd0e5ca12d1ba0b229a4121a54afa5ea2d8aa1)

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6238)

Don't memcpy the contents of an empty fragment

In DTLS if we have buffered a fragment for a zero length message (e.g.
ServerHelloDone) then, when we unbuffered the fragment, we were attempting
to memcpy the contents of the fragment which is zero length and a NULL
pointer. This is undefined behaviour. We should check first whether we
have a zero length fragment.

Fixes a travis issue.

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6225)

In cases where we ask PEM_def_callback for minimum 0 length, accept 0 length

Fixes #4716

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6173)

(cherry picked from commit c82c3462267afdbbaa53e11da0508ce4e03c02b3)

Fix typo 'is an error occurred' in documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6231)

X509 time: tighten validation per RFC 5280

- Reject fractional seconds
- Reject offsets
- Check that the date/time digits are in valid range.
- Add documentation for X509_cmp_time

GH issue 2620

Backported from 80770da39e

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6182)

BIO_s_mem() write: Skip early when input length is zero

When the input length is zero, just return zero early.  Otherwise,
there's a small chance that memory allocation is engaged, fails and
returns -1, which is a bit confusing when nothing should be written.

Fixes #4782 #4827

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/6175)

(cherry picked from commit 0d94212a046e87fafea6e83e8ea2b2a58db49979)

bn/asm/*-mont.pl: harmonize with BN_from_montgomery_word.

Montgomery multiplication post-conditions in some of code paths were
formally non-constant time. Cache access pattern was result-neutral,
but a little bit asymmetric, which might have produced a signal [if
processor reordered load and stores at run-time].

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6163)

(cherry picked from commit 774ff8fed67e19d4f5f0df2f59050f2737abab2a)

Resolved conflicts in ppc-mont.pl and x86_64-mont.pl.

(cherry picked from commit d067ba3bcc4944140578bdbf857affa6faf9e8c1)

v3_purp.c: add locking to x509v3_cache_extensions()

Fixes #6121

Thanks to Mingtao Yang for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6164)

Configure: fix Mac OS X builds that still require makedepend

Earlier Apple Xcode compilers, e.g. one targeting Mac OS X 10.7, don't
support dependency generation and one still has to use makedepend.  It's
unclear when it was fixed, but all clang-based Apple compilers seem to
support -M options.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6073)

Add some documentation for SSL_get_shared_ciphers()

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6115)

Fix comment in ssl.h

The ciphers field in a session contains the stack of ciphers offered by
the client.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6115)

Fix SSL_get_shared_ciphers()

The function SSL_get_shared_ciphers() is supposed to return ciphers shared
by the client and the server. However it only ever returned the client
ciphers.

Fixes #5317

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6115)

a_strex.c: prevent out of bound read in do_buf()

which is used for ASN1_STRING_print_ex*() and X509_NAME_print_ex*().

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6118)

Clarify BN_mod_exp docs

Specifically this is not supported with an even modulus and
BN_FLG_CONSTTIME.

Fixes #5082

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6138)

(cherry picked from commit 966b3df182b9c8644c154f1c3e686027d37acb05)

Fix some errors and missing info in the CMS docs

Fixes #5063

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6136)

Update version docs

Make it clear that you should not attempt to get the version before the
first handshake is complete.

Fixes #2893

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6015)

Skip the CMS test if configured 'no-cms'

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6108)

fix: BN_swap mishandles flags (1.0.2)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6102)

s_server: fix warnings unused-but-set-variable (no-dh)

This patch fixes the following two warnings when OpenSSL is built with no-dh option:

s_server.c: In function 's_server_main':
s_server.c:1105:25: warning: variable 'no_dhe' set but not used [-Wunused-but-set-variable]
     int no_tmp_rsa = 0, no_dhe = 0, no_ecdhe = 0, nocert = 0;
                         ^
s_server.c:1101:11: warning: variable 'dhfile' set but not used [-Wunused-but-set-variable]
     char *dhfile = NULL;
           ^
CLA: trivial
Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6087)

fix warning unused-but-set-variable 'alg_k' (no-dh and no-ec)

This patch fixes the following warning when OpenSSL is configured with
no-dh and no-ec:

./Configure no-ec no-dh linux-x86_64

...
s3_lib.c: In function 'ssl3_get_req_cert_type':
s3_lib.c:4234:19: warning: variable 'alg_k' set but not used [-Wunused-but-set-variable]
     unsigned long alg_k;

CLA: trivial
Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6087)

fix warning unused-but-set-variable 'nostrict' (no-dh and no-ec)

This patch fixes the following warning when OpenSSL is configured with
no-dh and no-ec:

./Configure no-ec no-dh linux-x86_64

...
s3_lib.c:4231:9: warning: variable 'nostrict' set but not used [-Wunused-but-set-variable]
     int nostrict = 1;
         ^

CLA: trivial
Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6087)

Update the *use_certificate* docs

Note that calling the *use_certificate* functions will replace any existing
certificate of the same type. The same thing applies for private keys.

Fixes #2147

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6093)

(cherry picked from commit db747425b0caae12e73c680799b57bdfcc72dbb4)

PEM_def_callback(): use same parameter names as for pem_password_cb

Add a bit more commentary to explain what's going on.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6080)

(cherry picked from commit d6d94d339756332bbabe2a1032ac511ae31b3fdc)

PEM_def_callback(): don't loop because of too short password given

That error is already caught by EVP_read_pw_string_min, and causes
this function to return -1, so the code detecting too short passwords
in this function is practically dead.

Fixes #5465

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6080)

(cherry picked from commit 4977b4e9281c981efcf6a8b31378b8bbd6a8a96f)

Fix documentation for the -showcerts s_client option

This option shows the certificates as sent by the server. It is not the
full verified chain.

Fixes #4933

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6069)

Document when a session gets removed from cache

Document the fact that if a session is not closed down cleanly then the
session gets removed from the cache and marked as non-resumable.

Fixes #4720

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6054)

(cherry picked from commit 0abb903b7618ad404c83ac9b44b2c1bf162b90de)

Fix the alert sent if no shared sig algs

We were sending illegal parameter. This isn't correct. The parameters are
legal, we just don't have an overlap. A more appropriate alert is
handshake failure.

Fixes #2919

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6011)

apps/s_socket.c: Fix do_accept

do_accept() checked that the peer IP address had a PTR record, and would
fail if not.  The retrieved named was then never used, even though passed
around.  All this is unnecessary, so we remove it.

Fixes #3407

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6018)

Don't crash if there are no trusted certs

The X509_STORE_CTX_init() docs explicitly allow a NULL parameter for the
X509_STORE. Therefore we shouldn't crash if we subsequently call
X509_verify_cert() and no X509_STORE has been set.

Fixes #2462

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6003)

Return 0 on a non-matching kdf_type

If we have a non-matching kdf_type then pkey_dh_derive silently succeeds.
It should fail. This is a "should not happen" condition anyway so the
impact is negligible.

Fixes #2440

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6000)

Fix ocsp app exit code

If we run the ocsp command line app and the responder returns a
non-successful status code then the app should exit with a failure code.

Based on an original patch by Tatsuhiro Tsujikawa.

Fixes #2387

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5999)

(cherry picked from commit 2add645dfc364cfc609e6138de1bf43c217bb557)

X509_cmp_time: only return 1, 0, -1.

The behaviour of X509_cmp_time used to be undocumented.

The new behaviour, documented in master, is to return only 0, 1, or -1.
Make the code in the other branches to adhere to this behaviour too,
to reduce confusion. There is nothing to be gained from returning
other values.

Fixes GH#4954

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4955)

(cherry picked from commit 48345917747a34feea3da2936994a265c7f2ca11)

Correct the check of RSA_FLAG_SIGN_VER

The wrong flags were being tested. It is the rsa->meth flags not the rsa
flags that should be tested.

wpa_supplicant has a bit of code that
1. Allocates and defines a RSA_METHOD structure.
2. calls RSA_new();
3. calls RSA_set_method().

In current versions of that code the rsa_sign and rsa_verify members of
the RSA_METHOD structure are not defined, thus making it compatible
with the really old versions of OpenSSL.

But should one change it use the rsa_sign method one must set the
RSA_FLAG_SIGN_VER bit of the RSA_METHOD structure to indicate that
one or both of those new methods are required.  In doing so, OpenSSL
will not call the new methods, not without this change.

CLA: trivial

Change-Id: I6e65a80f21399f25e966466ff676e3b21f85f360

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5971)

Update fingerprints.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5988)

Ignore the status_request extension in a resumption handshake

We cannot provide a certificate status on a resumption so we should
ignore this extension in that case.

Fixes #1662

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5898)

RSA key generation: ensure BN_mod_inverse and BN_mod_exp_mont both get called with BN_FLG_CONSTTIME flag set.

CVE-2018-0737

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 6939eab03a6e23d2bd2c3f5e34fe1d48e542e787)

Clarify default section in config.pod

This is a minor update which hopefully makes these particular lines
read a little easier.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5938)

(cherry picked from commit 0652e8a7fe6dd8cbdf4516b221642b10bbfc68fe)
(cherry picked from commit 9e8554fca5019b81ed7659d97f103f388ea3a2bf)

Updated to CONTRIBUTING to reflect GitHub, etc.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5889)

(cherry picked from commit 2876872ffe5dd53ec1c446656e924ff463e5d4bf)

Update the genpkey documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5884)

Pick a q size consistent with the digest for DSA param generation

There are two undocumented DSA parameter generation options available in
the genpkey command line app:
dsa_paramgen_md and dsa_paramgen_q_bits.

These can also be accessed via the EVP API but only by using
EVP_PKEY_CTX_ctrl() or EVP_PKEY_CTX_ctrl_str() directly. There are no
helper macros for these options.

dsa_paramgen_q_bits sets the length of q in bits (default 160 bits).
dsa_paramgen_md sets the digest that is used during the parameter
generation (default SHA1). In particular the output length of the digest
used must be equal to or greater than the number of bits in q because of
this code:

            if (!EVP_Digest(seed, qsize, md, NULL, evpmd, NULL))
                goto err;
            if (!EVP_Digest(buf, qsize, buf2, NULL, evpmd, NULL))
                goto err;
            for (i = 0; i < qsize; i++)
                md[i] ^= buf2[i];

            /* step 3 */
            md[0] |= 0x80;
            md[qsize - 1] |= 0x01;
            if (!BN_bin2bn(md, qsize, q))
                goto err;

qsize here is the number of bits in q and evpmd is the digest set via
dsa_paramgen_md. md and buf2 are buffers of length SHA256_DIGEST_LENGTH.
buf2 has been filled with qsize bits of random seed data, and md is
uninitialised.

If the output size of evpmd is less than qsize then the line "md[i] ^=
buf2[i]" will be xoring an uninitialised value and the random seed data
together to form the least significant bits of q (and not using the
output of the digest at all for those bits) - which is probably not what
was intended. The same seed is then used as an input to generating p. If
the uninitialised data is actually all zeros (as seems quite likely)
then the least significant bits of q will exactly match the least
significant bits of the seed.

This problem only occurs if you use these undocumented and difficult to
find options and you set the size of q to be greater than the message
digest output size. This is for parameter generation only not key
generation. This scenario is considered highly unlikely and
therefore the security risk of this is considered negligible.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5884)

Don't crash if an unrecognised digest is used with dsa_paramgen_md

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5884)

Change the "offset too large" message to more generic wording

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 1518c55a796b058eff01f3cbf177f4b726c01d7c)

Fix range checks with -offset and -length in asn1parse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 16e1eea6a67c85c9d786f3c4448182b1aca101b8)

Fix a crash in the asn1parse command

Thanks to Sem Voigtländer for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 752837e0664e990b5edf6f0b69e1b4612efadce0)

o_time.c: use gmtime_s with MSVC

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5719)

Adds multiple checks to avoid buffer over reads

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5687)

Don't write out a bad OID

If we don't have OID data for an object then we should fail if we
are asked to encode the ASN.1 for that OID.

Fixes #5723

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5725)

(cherry picked from commit 53c9818e970fc0c22d77e19fda3b3e6f6c9e759d)

Prepare for 1.0.2p-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.0.2o release

Reviewed-by: Richard Levitte <levitte@openssl.org>

make update

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for the new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Limit ASN.1 constructed types recursive definition depth

Constructed types with a recursive definition (such as can be found in
PKCS7) could eventually exceed the stack given malicious input with
excessive recursion. Therefore we limit the stack depth.

CVE-2018-0739

Credit to OSSFuzz for finding this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix dsaparam -genkey with DER outform

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5744)

(cherry picked from commit 5281435258b5d8201a00b4a9781bb724d99630f0)

Fix ecparam -genkey with point compression or DER outform

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5741)

(cherry picked from commit 4bdc25b07f007d9c383fbad159f81543f2e95965)

The default conv_form is uncompressed

Fixes #5711

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5712)

(cherry picked from commit ab0a3914a64d8f1fce22795c02269e1288df52b1)

consttime flag changed

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5170)

(cherry picked from commit 7150a4720af7913cae16f2e4eaf768b578c0b298)

used ERR set/pop mark

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5170)

(cherry picked from commit 011f82e66f4bf131c733fd41a8390039859aafb2)

Replaced variable-time GCD with consttime inversion to avoid side-channel attacks on RSA key generation

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5170)

(cherry picked from commit 9db724cfede4ba7a3668bff533973ee70145ec07)

Fix some bugs with the cfb1 bitsize handling

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5450)

Fix a memory leak in the ca application

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5445)

Allow multiple entries without a Subject even if unique_subject == yes

It is quite likely for there to be multiple certificates with empty
subjects, which are still distinct because of subjectAltName. Therefore
we allow multiple certificates with an empty Subject even if
unique_subject is set to yes.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5445)

Report a readable error on a duplicate cert in ca app

Commit 87e8fec (16 years ago!) introduced a bug where if we are
attempting to insert a cert with a duplicate subject name, and
duplicate subject names are not allowed (which is the default),
then we get an unhelpful error message back (error number 2). Prior
to that commit we got a helpful error message which displayed details
of the conflicting entry in the database.

That commit was itself attempting to fix a bug with the noemailDN option
where we were setting the subject field in the database too early
(before extensions had made any amendments to it).

This PR moves the check for a conflicting Subject name until after all
changes to the Subject have been made by extensions etc.

This also, co-incidentally Fixes the ca crashing bug described in issue
5109.

Fixes #5109

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5445)