SERIALIZER: add support for serializing EVP_PKEYs

The following public functions is added:

- OSSL_SERIALIZER_CTX_new_by_EVP_PKEY()
- OSSL_SERIALIZER_CTX_set_cipher()
- OSSL_SERIALIZER_CTX_set_passphrase()
- OSSL_SERIALIZER_CTX_set_passphrase_cb()
- OSSL_SERIALIZER_CTX_set_passphrase_ui()

OSSL_SERIALIZER_CTX_new_by_EVP_PKEY() selects a suitable serializer
for the given EVP_PKEY, and sets up the OSSL_SERIALIZER_CTX to
function together with OSSL_SERIALIZER_to_bio() and
OSSL_SERIALIZER_to_fp().

OSSL_SERIALIZER_CTX_set_cipher() indicates what cipher should be used
to produce an encrypted serialization of the EVP_PKEY.  This is passed
directly to the provider using OSSL_SERIALIZER_CTX_set_params().

OSSL_SERIALIZER_CTX_set_passphrase() can be used to set a pass phrase
to be used for the encryption.  This is passed directly to the
provider using OSSL_SERIALIZER_CTX_set_params().

OSSL_SERIALIZER_CTX_set_passphrase_cb() and
OSSL_SERIALIZER_CTX_set_passphrase_ui() sets up a callback to be used
to prompt for a passphrase.  This is stored in the context, and is
called via an internal intermediary at the time of serialization.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

CORE: expose the property parsers and checker to the rest of the libraries

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

SERIALIZER: add functions for serialization to file

These functions are added:

- OSSL_SERIALIZER_to_bio()
- OSSL_SERIALIZER_to_fp() (unless 'no-stdio')

OSSL_SERIALIZER_to_bio() and OSSL_SERIALIZER_to_fp() work as wrapper
functions, and call an internal "do_output" function with the given
serializer context and a BIO to output the serialized result to.

The internal "do_output" function must have intimate knowledge of the
object being output.  This will defined independently with context
creators for specific OpenSSL types.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

SERIALIZER: New API for serialization of objects through providers

Serialization is needed to be able to take a provider object (such as
the provider side key data) and output it in PEM form, DER form, text
form (for display), and possibly other future forms (XML? JSON? JWK?)

The idea is that a serializer should be able to handle objects it has
intimate knowledge of, as well as object data in OSSL_PARAM form.  The
latter will allow libcrypto to serialize some object with a different
provider than the one holding the data, if exporting of that data is
allowed and there is a serializer that can handle it.

We will provide serializers for the types of objects we know about,
which should be useful together with any other provider that provides
implementations of the same type of object.

Serializers are selected by method name and a couple of additional
properties:

- format        used to tell what format the output should be in.
                Possibilities could include "format=text",
                "format=pem", "format=der", "format=pem-pkcs1"
                (traditional), "format=der-pkcs1" (traditional)
- type          used to tell exactly what type of data should be
                output, for example "type=public" (the public part of
                a key), "type=private" (the private part of a key),
                "type=domainparams" (domain parameters).

This also adds a passphrase callback function type,
OSSL_PASSPHRASE_CALLBACK, which is a bit like OSSL_CALLBACK, but it
takes a few extra arguments to place the result in.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

CORE: pass the full algorithm definition to the method constructor

So far, the API level method constructors that are called by
ossl_method_construct_this() were passed the algorithm name string and
the dispatch table and had no access to anything else.

This change gives them access to the full OSSL_ALGORITHM item, thereby
giving them access to the property definition.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

CORE: ossl_namemap_add_names(): new function to add multiple names

This was originally the private add_names_to_namemap() in
crypto/evp/evp_fetch.c, but made more generally useful.

To make for more consistent function naming, ossl_namemap_add() and
ossl_namemap_add_n() are renamed to ossl_namemap_add_name() and
ossl_namemap_add_name_n().

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

Disable mem leak checking for the self test lock

The fips self test lock is deallocated in platform specific ways that may
occur after we do mem leak checking. If we don't know how to free it for
a particular platform then we just leak it deliberately. So we
temporarily disable the mem leak checking while we allocate the lock.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9939)

Make sure we only run the self tests once

Fixes #9909

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9939)

Check the return from OPENSSL_buf2hexstr()

The function OPENSSL_buf2hexstr() can return NULL if it fails to allocate
memory so the callers should check its return value.

Fixes #10525

Reported-by: Ziyang Li (@Liby99)
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10526)

Check that OPENSSL_zalloc was successful when creating EVP types

We were missing a NULL check in a few very similar places following an
OPENSSL_zalloc() call.

Reported-by: Ziyang Li (@Liby99)
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10526)

Add a test for NULL chunks in encrypt/decrypt

Issue #8675 describes a problem where calling EVP_DecryptUpdate() with an
empty chunk causes the result to be different compared to if you do not
use an empty chunk. This adds a test for that case.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10530)

Make sure we handle input NULL with length 0

If we call EVP_EncryptUpdate/EVP_DecryptUpdate with length 0 we should
be able to handle it. Most importantly we shouldn't get different
results if we do this compared to if we don't!

An exception is made for CCM mode which has special handling for this in
the low level cipher function.

Fixes #8675

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10530)

Configure: make it possible to have generated generators

This makes it possible to chain GENERATOR statements, which allows
constructs like this:

    GENERATE[something.html]=something.pod
    GENERATE[something.pod]=something.pod.in

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6236)

Adapt *.tmpl to generate docs at build time

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6236)

Add doc/build.info to build the documentation

This build.info is entirely generated when configuring

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6236)

Add the possibility to generate documentation at build time

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6236)

configdata.pm.in, util/dofile.pl: load 'platform' unconditionally

The 'platform' module handles defaults fine, there's no need to add
extra conditions on it being loaded.

Fixes #10513

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10514)

apps/ocsp.c: sock_timeout -> socket_timeout

It appears that 'sock_timeout' is defined at least with DJGPP, so we
rename our symbol and hope the new name isn't taken.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10515)

Fix L<EVP_MD_CTX_set_pkey_ctx> links

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10420)

(cherry picked from commit ca7c982821cd6d7546f92623da61a6f43ee32d0a)

chacha/asm/chacha-armv8.pl: preserve FP registers d8 and d9 correctly

Depending on the size of the input, we may take different paths through
the accelerated arm64 ChaCha20 routines, each of which use a different
subset of the FP registers, some of which need to be preserved and
restored, as required by the AArch64 calling convention (AAPCS64)

In some cases, (e.g., when the input size is 640 bytes), we call the 512
byte NEON path followed directly by the scalar path, and in this case,
we preserve and restore d8 and d9, only to clobber them again
immediately before handing over to the scalar path which does not touch
the FP registers at all, and hence does not restore them either.

Fix this by moving the restoration of d8 and d9 to a later stage in the
512 byte routine, either before calling the scalar path, or when exiting
the function.

Fixes #10470
CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10497)

Use option/parameter consistently.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10135)

Document more env var stuff, fix some typo's

Add openssl-env.pod
Also fix up many other environment page formatting nits.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10135)

test/cipher_overhead_test.c: build unconditionally

Build it against static libraries always, since that's the only way it
can work as intended.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10516)

util/find-doc-nits: Better parsing of links

When checking links, we need to peal away stuff that aren't part of
the link proper first.  That makes it easier to check the link
itself.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10507)

doc/man7/proxy-certificates.pod: New guide for proxy certificates

This replaces doc/HOWTO/proxy_certificates.txt

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10507)

Property: fix memory leak.

Coverity 1456008.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10509)

Cache constants for fetched EVP_cipher

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/10461)

Support ciphersuites using a SHA2 384 digest in FreeBSD KTLS.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #10372

Property: fix intermittent test failure.

The reduction in the cache flush threshold in #10408 caused the stochastic test
to fail with noticeable probability.  Revert that part of the change.

Also add a comment to help avoid this in future.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10505)

Remove RANDFILE settings from configuration files

OpenSSL 1.1.1 introduced a new CSPRNG with an improved seeding
mechanism, which makes it dispensable to define a RANDFILE for
saving and restoring randomness. This commit removes the RANDFILE
declarations from our own configuration files and adds documentation
that this option is not needed anymore and retained mainly for
compatibility reasons.

Fixes #10433

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10436)

Remove deadlock that was caused by calling pthread_rwlock_wrlock() on same thread twice. This can be reproduce only with musl.

CLA: trivial

Signed-off-by: Matus Kysel <mkysel@tachyum.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10501)

Process GOST ClientKeyExchange message in SSL_trace

Use ssl_print_hex to print message in case of GOST key exchange algorithm.

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9995)

UI_UTIL_wrap_read_pem_callback(): when |cb| is NULL, use PEM_def_callback

Fixes #10444

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10447)

Thread: Avoid a NULL dereference after failed initialisation.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10493)

main: avoid a NULL dereference on initialisation.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10495)

PROV: Avoid NULL dereference in SHA3 dup call.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10487)

PROV: check for memory allocation failure in digest _dupctx.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10487)

kdfs: make free calls check for NULL.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10496)

Fix a race condition in SNI handling

As was done for ciphers, supported groups, and EC point formats in
https://github.com/openssl/openssl/pull/9162, only write the negotiated
SNI hostname value to the session object when not resuming, even for
TLS 1.3 resumptions.  Otherwise, when using a stateful session cache
(as is done by default when 0-RTT data is enabled), we can have multiple
SSLs active using the same in-memory session object, which leads to
double-frees and similar race conditions in the SNI handler prior
to this commit.

Fortunately, since draft-ietf-tls-tls13-22, there is no requirement
that the SNI hostname be preserved across TLS 1.3 resumption, and thus
not a need to continually update the session object with the "current"
value (to be used when producing session tickets, so that the subsequent
resumption can be checked against the current value).  So we can just
relax the logic and only write to the session object for initial handshakes.
This still leaves us in a somewhat inconsistent state, since if the SNI value
does change across handshakes, the session object will continue to record
the initial handshake's value, even if that bears no relation to the
current handshake.  The current SSL_get_servername() implementation
prefers the value from the session if s->hit, but a more complete fix
for that and related issues is underway in
https://github.com/openssl/openssl/pull/10018; there is no need to wait
for the complete fix for SNI name handling in order to close the
race condition and avoid runtime crashes.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10441)

Core: allow NULL argument to stored_namemap_free().

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10488)

Documentation updates due to naming tweaks

Also documents our new canonical naming.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10092)

Tweak some algorithm naming inconsistencies

Make some algorithms names better match our "canonical" style.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10092)

EVP p_lib: Add NULL check to EVP_PKEY_missing_parameters.

Check for NULL and return error if so.
This can possibly be called from apps/ca.c with a NULL argument.

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
(Merged from https://github.com/openssl/openssl/pull/10474)

Engine: Add NULL check.

Add NULL check for return from pkey_asn1_meths.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10473)

ECDSA: don't clear free memory after verify.

Verifications are public, there is no need to clear the used storage before
freeing it.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10475)

Replumbing: pre-populate the EVP namemap with commonly known names

This adds ossl_namemap_empty(), to detect if a namemap is empty and
can thereby be pre-populated.

This also affects the way legacy NIDs are looked up in
evp_cipher_from_dispatch() and evp_md_from_dispatch().  Instead of
trying to find the NID directly, look up the legacy method structure
and grab the NID from there.  The reason is that NIDs can be aliases
for other NIDs, which looks like a clash even if wasn't really one.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8984)

Fix sha512_block_data_order_avx2 backtrace info

We store a secondary frame pointer info for the debugger
in the red zone.

Fixes #8853

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9624)

Allow specifying the tag after AAD in CCM mode (2)

In addition to 67c81ec3 which introduced this behavior in CCM mode
docs but only implemented it for AES-CCM.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10331)

Dont pass zero length input to asm modules for ciphers

The asm modules may assume an input length > 0.

Fixes: #9262
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10323)

Document command parameters.

Add documentation for all commands that have parameters.
Fix a couple of minor doc and programming bugs, too.

Fixes #10313

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10371)

Add test for non-default sized keys in variable key size ciphers

Various ciphers have the ability to support varaible length keys.
This adds some test cases for algorithms where we did not test a key
length other than the default size.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10449)

EVP_CIPHER_CTX_set_keylen should not succeed if a bad keylen is passed

EVP_CIPHER_CTX_set_keylen() was succeeding even though a bad key length
is passed to it. This is because the set_ctx_params() were all accepting
this parameter and blindly changing the keylen even though the cipher did
not accept a variable key length. Even removing this didn't entirely
resolve the issue because set_ctx_params() functions succeed even if
passed a parameter they do not recognise.

This should fix various issues found by OSSfuzz/Cryptofuzz.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10449)

fix s390 compile error due to missing comma.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10467)

Fix the issue that aes-xxx-siv algorithms failed in OpenSSL speed test

Speed test, aes-siv related cases fail on both x86 and arm.
The return value of siv_init() causes this problem, remove
the iv check to fix it.
Verify it locally, the result is pass.

Fixes #10416

Change-Id: If1a18599f3d0f56f22a1ce4f8f114b8db0f68cca

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10419)

Properties: make query cache reference count aware.

The property query cache was not reference count aware and this could cause
problems if the property store removes an algorithm while it is being returned
from an asynchronous query.  This change makes the cache reference count aware
and avoids disappearing algorithms.

A side effect of this change is that the reference counts are now owned by the
cache and store.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10408)

Fix Use after free when copying cipher ctx

Fixes #10438
issue found by clusterfuzz/ossfuzz

The dest was getting a copy of the src structure which contained a pointer that should point to an offset inside itself - because of the copy it was pointing to the original structure.

The setup for a ctx is mainly done by the initkey method in the PROV_CIPHER_HW structure. Because of this it makes sense that the structure should also contain a copyctx method that is use to resolve any pointers that need to be setup.

A dup_ctx has been added to the cipher_enc tests in evp_test. It does a dup after setup and then frees the original ctx. This detects any floating pointers in the duplicated context that were pointing back to the freed ctx.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10443)

PEM: constify PEM_write_ routines

There's no reason why the object to be written, or the key string
given by the caller should be non-const.

This makes the IMPLEMENT_PEM_..._const and  DECLARE_PEM_..._const
macros superfluous, so we keep them around but mark them deprecated.

In all places where IMPLEMENT_PEM_..._const and  DECLARE_PEM_..._const
are used, they are replaced with the corresponding macros without
'_const'.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10452)

s390x assembly pack: fix bn_mul_comba4

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10454)

Add missing EVP_PKEY_METHOD accessors for digestsign and digestverify

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10388)

commit a6efbe123af3d98b4d10d4fcdfe68dc5303212f8
Author: raniervf <ranier_gyn@hotmail.com>
Date:   Thu Nov 7 18:59:11 2019 -0300

Avoid calling strlen repeatedly in loops.

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10380)

Fix some broken doc links

Replace L<> link to header-file with a C<> reference.
Change some broken L<provider(3)> links to L<provider(7)>.
For consistency, rename four cipher pages to have a specific mode.
Fix up all references to any "generic" names to point to specific names.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10100)

Make sure we free the CONF structure allocated by confdump

A travis build was failing because of a memory leak in confdump due to
us not freeing an allocated CONF structure.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10429)

travis.yml: add arch s390x target

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10446)

testutil/init.c rename to testutil/testutil_init.c

Avoid conflicts with some linkers.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10439)

A few fixes in the EVP_KDF-KB and EVP_KDF-KRB5KDF manpages

The context and label is info and salt respectively - fix
the example in KB manpage for that.

There are some typos and bug in EVP_KDF_derive call in the
KRB5KDF example.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10426)

Delete recently documented functions from missingcrypto

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10432)

Fix an uninitialised read in conf_def.c

PR 8882 added a new field to the CONF structure. Unfortunately this
structure was created using OPENSSL_malloc() and the new field was not
explicitly initialised in the "init" function. Therefore when we came to
read it for the first time we got an uninitialised read.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10428)

Fix no-blake2

We modify the build.info file to exclude the legacy_blake2.c file in
the event that blake2 support has been disabled.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10425)

Fix no-engine

Fix a misplaced "#endif" which was disabling a little too much code.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10423)

CORE & PROV: make export of key data leaner through callback

Exporting data from a provider owned domainparams or key is quite an
ordeal, with having to figure out what parameter keys an
implementation supports, call the export function a first time to find
out how large each parameter buffer must be, allocate the necessary
space for it, and call the export function again.

So how about letting the export function build up the key data params
and call back with that?  This change implements exactly such a
mechanism.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10414)

Fix no-cmac and no-camellia

Guard two tests that depend on CMAC and Camellia so that we don't fail
if those algorithms are not available.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10424)

Add confdump to .gitignore

We added a new executable to the test directory which didn't match the
existing gitignore rules, so we add it explicitly.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10430)

Document various new/changes EVP_PKEY_CTX_* functions

We have converted a number of macros to functions and made them work
with providers. We've also added some *_ex() variants that needed
documenting.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Document the public EVP_ASYM_CIPHER releated functions

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Add documentation for the Asymmetric Cipher Operation

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Move RSA Asym cipher code to the default provider

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Increase OSSL_PARAM_BLD_MAX for multi-prime RSA

The old value of 10 for OSSL_PARAM_BLD_MAX is insufficient for multi-prime
RSA. That code has this assert:

        if (!ossl_assert(/* n, e */ 2 + /* d */ 1 + /* numprimes */ 1
                         + numprimes + numexps + numcoeffs
                         <= OSSL_PARAM_BLD_MAX))
            goto err;

So we increase OSSL_PARAM_BLD_MAX which would be enough for 7 primes
(more than you would ever reasonably want).

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Implement provider support for Asym Ciphers

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Allow strings in params to be of zero length

Sometimes it is useful to be able to pass NULL/zero length strings

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

Allow setting a NULL pointer in a params structure

Sometimes it is valid to send a NULL pointer in params.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10152)

CORE: Add a generic callback function type

This offers a very generic way to define a callback as well as calling
it.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10412)

chacha_enc.c: fix for EBCDIC platforms

Signed-off-by: Joerg Schmidbauer <jschmidb@de.ibm.com>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10417)

Extend docs for EC_POINT conversion functions

Add more explicit documentation about the relation between
EC_POINT_point2oct(), EC_POINT_point2hex(), EC_POINT_point2bn() and
their reverse.

In particular highlight that EC_POINT_point2oct() and
EC_POINT_oct2point() conform to, respectively, Sec. 2.3.3 and Sec. 2.3.4
of the SECG SEC 1 standard (which is the normative reference for the
already mentioned RFC 5480), highlighting with a note how this affect
the encoding/decoding of the point at infinity (which in contrast with
any other valid generic point of a curve is assigned an exceptional
fixed octet string encoding, i.e., 0x00).

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10329)

Add self-test for EC_POINT_hex2point

Adds tests for each curve to ensure that encodings obtained through
EC_POINT_hex2point() can be fed to EC_POINT_point2hex() yielding a point
identical to the one from which the encoding is generated.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10329)

Fix EC_POINT_bn2point() for BN_zero()

EC_POINT_bn2point() rejected BIGNUMs with a zero value.

This behavior indirectly caused failures when converting a point
at infinity through EC_POINT_point2hex() and then back to a point with
EC_POINT_hex2point().

With this change such BIGNUMs are treated like any other and exported to
an octet buffer filled with zero.
It is then EC_POINT_oct2point() (either the default implementation or
the custom one in group->meth->oct2point) to determine if such encoding
maps to a valid point (generally the point at infinity is encoded as
0x00).

Fixes #10258

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10329)

libcrypto.num: remove some unused EVP_MAC symbols

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10421)

Add more tests for apps/req

https://github.com/openssl/openssl/issues/10224#issuecomment-546593113
highlighted that existing testing infrastructure is not covering common
usage patterns of the `req` app.

This commit explicitly adds request generations thorugh the CLI using
RSA, DSA and ECDSA (P-256) keys.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/pr-10312)

Add missing docs for some PKCS12 functions

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9752)

add OPENSSL_FUNC.pod documenting OPENSSL_MSTR, OPENSSL_FUNC, and friends

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/10236)

re-add definition of OPENSSL_MSTR deleted from opensslv.h in macros.h

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/10236)

SSL: Document SSL_add_{file,dir,store}_cert_subjects_to_stack()

This also removes the incorrect documentation comments by those
functions, and fixes a bug in SSL_add_store_cert_subjects_to_stack(),
where the condition for recursive addition was 'depth == 0' when it
should be 'depth > 0'.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10402)

Add a tester of OpenSSL config files and test the current .pragma

test/confdump.c reads an OpenSSL config file and prints out the
processed result.  This can be used to check that a config file is
processed correctly.

We add a test recipe and the necessary data to test the dollarid
pragma.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8882)

Add a .pragma directive for configuration files

Currently added pragma:

.pragma dollarid:on

This allows dollar signs to be a keyword character unless it's
followed by a opening brace or parenthesis.

Fixes #8207

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8882)

Make sure KDF reason codes are conserved in their current state

Because KDF errors are deprecated and only conserved for backward
compatibilty, we must make sure that they remain untouched.  A simple
way to signal that is by modifying crypto/err/openssl.ec and replace
the main header file (include/openssl/kdf.h in this case) with 'NONE',
while retaining the error table file (crypto/kdf/kdf_err.c).

util/mkerr.pl is modified to silently ignore anything surrounding a
conserved lib when such a .ec line is found.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10368)

Deprecate ERR_load_KDF_strings()

Not only deprecate, but also remove the reason strings and make
ERR_load_KDF_strings() do nothing.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10368)

util/perl/OpenSSL/ParseC.pm: Fix 'extern "C" DEPRECATEDIN_x_y(...)'

The parser expected an 'extern "C"' followed by a single declaration
to always end with a semicolon.  Then came along something like this:

    extern "C" DEPRECATEDIN_3_0(int ERR_load_KDF_strings(void))

This change adjusts the detector of 'extern "C"' to also take in
accound a declaration that ends with a parenthesis.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10368)

Reinstate the KDF error macros

For minimum breakage with existing applications that might use them.

This reverts commit fe6ec26b204a056aee2a24b79df09a45b2308603 and
37ed62107112d95f7b7c9bf75602a6ac40883a89.

Fixes #10340

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10368)

Fix cut'n'paste typos in the provider-digest pod

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10398)

Fix misspelled resumption_label for CHARSET_EBCDIC

The resumption_label variable when CHARSET_EBCDIC was enabled, was misspelled.
Instead of evaluating to 'res binder' as expected, it evaluated to 'red binder'.

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10396)

Fix no-dsa

Make sure we don't try and load a DSA key in the tests if DSA has been
disabled.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10375)