Add a sigalg test to check we only allow sigalgs we sent

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Extend PSS signature support to TLSv1.2

TLSv1.3 introduces PSS based sigalgs. Offering these in a TLSv1.3 client
implies that the client is prepared to accept these sigalgs even in
TLSv1.2.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Fix test_sslversions to know that TLSv1.3 sets record version to TLSv1.0

This also acts as a test for the bug fixed in the previous commit.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Always use TLSv1.0 for record layer version in TLSv1.3

TLSv1.3 freezes the record layer version and ensures that it is always set
to TLSv1.0. Some implementations check this.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Add a TLS1.3 TODO for setting of sig algs

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Add some sig algs tests

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Ignore PKCS1 based sig algs in TLSv1.3

In TLSv1.3 we must use PSS based sig algs for RSA signing. Ignore any
shared sig algs which are PKCS1 based.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Verify that the sig algs extension has been sent for TLSv1.3

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Fix client application traffic secret

A misreading of the TLS1.3 spec meant we were using the handshake hashes
up to and including the Client Finished to calculate the client
application traffic secret. We should be only use up until the Server
Finished.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Temporarily ignore NewSessionTickets for TLS1.3

We can't handle these messages yet, so ignore them for now.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Use the correct size for TLSv1.3 finished keys

We need to use the length of the handshake hash for the length of the
finished key.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Ensure the record sequence number gets incremented

We were not incrementing the sequence number every time we sent/received
a record.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Remove some unneeded functions

The sigalgs work has made some old lookup tables and functions redundant
so remove them.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Use NIDs instead of the old TLSv1.2 sigalgs hash and sig ids

We had an extra layer of indirection in looking up hashes and sigs based
on sigalgs which is now no longer necessary. This removes it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Remove a redundant function

The extensions refactor made this function redundant so we can remove it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Convert Sigalgs processing to use ints

In TLSv1.2 an individual sig alg is represented by 1 byte for the hash
and 1 byte for the signature. In TLSv1.3 each sig alg is represented by
two bytes, where the two bytes together represent a single hash and
signature combination. This converts the internal representation of sigalgs
to use a single int for the pair, rather than a pair of bytes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Sign CertificateVerify messages using PSS padding

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Make CertificateVerify TLS1.3 aware

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Move Certificate Verify construction and processing into statem_lib.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Add a TODO(TLS1.3) around certificate selection

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Fix typo in Blake2 function names

Fixes GitHub issue 2169.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2207)

Print the X509 version signed, and convert to unsigned for the hex version.

Found by tis-interpreter

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1754

Only enable CRYPTO_3DES_ECB if that name is an existing macro

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2203)

Small fixes of cryptodev engine

- guard CRYPTO_3DES_CBC
- add a missing cast

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2203)

Mark a HelloRequest record as read if we ignore it

Otherwise the client will try to process it again. The second time around
it will try and move the record data into handshake fragment storage and
realise that there is no data left. At that point it marks it as read
anyway. However, it is a bug that we go around the loop a second time, so
we prevent that.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2200)

use EVP_CIPHER_CTX_ functions instead of accessing EVP_CIPHER_CTX internals

by levitte

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

fix for BSD cryptodev

by levitte

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

Remove commented-out HMAC code

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

Style the code

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

Remove unused ret variable

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

Remove non-functional CRYPTO_AES_CTR ifdef disabling AES-CTR in cryptodev

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

Add AES-ECB and 3DES-ECB to cryptodev

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

cryptodev: allow copying EVP contexts

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

cryptodev: Fix issue with signature generation

That patch also enables support for SHA2 hashes, and
removes support for hashes that were never supported by
cryptodev.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1784)

Review comments

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2052)

Use typedefs for PSK, NPN, ALPN callback functions

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2052)

Move extension data into sub-structs

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2052)

Fix build issues with no-dh, no-dsa and no-ec

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2192)

Fix a memory leak in RSA_padding_add_PKCS1_OAEP_mgf1

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #2140

Rename "verify_cb" to SSL_verify_cb

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2151)

Doc nits: callback function typedefs

Enhance find-doc-nits to be better about finding typedefs for
callback functions.  Fix all nits it now finds.  Added some new
typedef names to ssl.h some of which were documented but did not
exist

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2151)

Add server temp key type checks

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2191)

Add new ssl_test option.

Add option ExpectedTmpKeyType to test the temporary key the server
sends is of the correct type.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2191)

fix a few more style issues

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Documentation clarification and fixes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Remove unnecessary frees and style fixes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

fix typo and remove duplicate macro

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Add documentation for PSS control operations.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Use more desciptive macro name rsa_pss_restricted()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

style issues

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

free str on error

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

clarify comment

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

fix various style issues

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

add test for invalid key parameters

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

document RSA-PSS algorithm options

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

add PSS key tests

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

print errors in pkey utility

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

make errors

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

add parameter error

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Set EVP_PKEY_CTX in SignerInfo

If we aren't setting public key parameters make EVP_PKEY_CTX available
in SignerInfo so PSS mode and parameters are automatically selected.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Only allow PSS padding for PSS keys.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Decode parameters properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Return errors PKCS#7/CMS enveloped data ctrls and PSS

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Add PSS parameter restrictions.

If a key contains any PSS parameter restrictions set them during
sign or verification initialisation. Parameters now become the
default values for sign/verify. Digests are fixed and any attempt
to change them is an error. The salt length can be modified but
must not be less than the minimum value.

If the key parameters are invalid then verification or signing
initialisation returns an error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Initial parameter restrictions.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Add rsa_pss_get_param.

New function rsa_pss_get_param to extract and sanity check PSS parameters.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Don't allow PKCS#7/CMS encrypt with PSS.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Add macros to determine if key or ctx is PSS.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Support pad mode get/set for PSS keys.

Pad mode setting returns an error if the mode is anything other then PSS.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Key gen param support.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Set PSS padding mode for PSS keys.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Digest string helper function.

New function EVP_PKEY_CTX_md() which takes a string and passes a digest
to a ctrl.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Support RSA operations in PSS.

Add support for common operations in PSS by adding a new function
RSA_pkey_ctx_ctrl() which calls EVP_PKEY_CTX_ctrl if the key type
is RSA or PSS.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

PSS EVP_PKEY method

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

RSA-PSS key printing.

Print out RSA-PSS key parameters if present.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

PSS parameter encode and decode.

For RSA PSS keys encode and decode parameters when handling public
and private keys.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Split PSS parameter creation.

Split PSS parameter creation. This adds a new function rsa_pss_params_create
which creates PSS parameters from digest and salt values. This will be
used for PSS key generation.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Use method key type instead of EVP_PKEY_RSA

Make RSA method more flexible by using the key type from the
method instead of hard coding EVP_PKEY_RSA: by doing this the
same code supports both RSA and RSA-PSS.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

PSS ASN.1 method

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

add EVP_PKEY_RSA_PSS

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Add pss field to RSA structure and free it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Cache maskHash parameter

Store hash algorithm used for MGF1 masks in PSS and OAEP modes in PSS and
OAEP parameter structure: this avoids the need to decode part of the ASN.1
structure every time it is used.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Update fuzz corpora

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2182

Make client and server fuzzer reproducible

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2182

Make the bignum fuzzer reproducible

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2182

Update fuzz documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2182

Make rand_add predictable when fuzzing

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2182

server fuzzer: add support for DSA and ECDSA

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2182

Fix various style issues following feedback

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Rename the chain variable to chainidx

This variable represents the index of the cert within the chain, so give it
a name that better represents that.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Fix a double blank line style issue

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Initialise the al variable

al can be used uninitialised in an error path.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Update SSL_trace to understand TLSv1.3 Certificates

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Implement TLSv1.3 style CertificateStatus

We remove the separate CertificateStatus message for TLSv1.3, and instead
send back the response in the appropriate Certificate message extension.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Create Certificate messages in TLS1.3 format

Also updates TLSProxy to be able to understand the format and parse the
contained extensions.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Extends extension parsing to take the Certificate

Continuing from the previous commit we also need to extend the extensions
framework to supply the Certificate we just read during parsing.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Extend tls_construct_extensions() to enable passing of a certificate

The Certificate message in TLS1.3 has an extensions block for each
Certificate. Therefore we need to extend tls_construct_extensions() to pass
in the certificate we are working on. We also pass in the position in the
chain (with 0 being the first certificate).

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2020)

Remove BIO_seek/BIO_tell from evp_test.c

BIO_seek and BIO_tell can cause problems with evp_test.c on some platforms.
Avoid them by using a temporary memory BIO to store key PEM data.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2183)

Don't run MSBLOB conversion tests when RSA or DSA are disabled

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2174)