Delete old KDF bridge EVP_PKEY_METHODS

The KDF bridge is now done provider side so the old EVP_PKEY_METHODS for
this are no longer required.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12573)

Extend the EVP_PKEY KDF to KDF provider bridge to also support Scrypt

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12573)

Extend the EVP_PKEY KDF to KDF provider bridge to also support HKDF

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12573)

Implement a EVP_PKEY KDF to KDF provider bridge

Some KDF implementations were available before the current EVP_KDF API.
They were used via EVP_PKEY_derive. There exists a bridge between the old
API and the EVP_KDF API however this bridge itself uses a legacy
EVP_PKEY_METHOD. This commit implements a provider side bridge without
having to use any legacy code.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12573)

Initial Apple Silicon support.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12591)

Fix memory leak in drbgtest

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/12613)

Add 'on demand self test' and status test to providers

The default and legacy providers currently return 1 for status and self test checks.
Added test to show the 3 different stages the self test can be run (for installation, loading and on demand).

For the fips provider:
  - If the on demand self test fails, then any subsequent fetches should also fail. To implement this the
    cached algorithms are flushed on failure.
  - getting the self test callback in the fips provider is a bit complicated since the callback hangs off the core
    libctx (as it is set by the application) not the actual fips library context. Also the callback can be set at
    any time not just during the OSSL_provider_init() so it is calculated each time before doing any self test.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11752)

Add some of the missing CMS API documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Change CMS tests to use a library context.

A DHX related test has been commented out and TODO(3) added, until DHX is added correctly to a provider.
Added generated files.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Add EVP signature with libctx methods.

-Added EVP_SignFinal_with_libctx() and EVP_VerifyFinal_with_libctx()
-Renamed EVP_DigestSignInit_ex() and EVP_DigestVerifyInit_with_libctx() to
  EVP_DigestSignInit_with_libctx() and EVP_DigestVerifyInit_with_libctx()

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Use libctx for EVP_CIPHER_CTX_rand_key() method.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Add libctx to ecdh_KDF_X9_63.

Code is now correctly included in the fips provider.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Fix EVP_PKEY_CTX_get_rsa_oaep_md() & EVP_PKEY_CTX_get_rsa_mgf1_md() so they use a libctx to retrieve the digest

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Add libctx support to PKCS7.

-Public PKCS7 methods that create a PKCS7 object now have variants that also add a libctx and propq.
 This includes PKCS7_new_with_libctx(), PKCS7_sign_with_libctx() and PKCS7_encrypt_with_libctx()
-Added SMIME_read_PKCS7_ex() so that a created PKCS7 object can be passed to the read.
-d2i_PKCS7_bio() has been modified so that after it loads the PKCS7 object it then resolves any subobjects that require
 the libctx/propq (such as objects containing X509 certificates).

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Add libctx to SMIME ASN1

Added SMIME_write_ASN1_with_libctx() since it fetches rand internally.
Added SMIME_read_CMS_ex() so that a created object (CMS_ContentInfo) can be passed to the read.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Add internal method x509_set0_libctx().

This should only be called during (or right after) using d2iXXX on a object that contains embedded certificate(s)
that require a non default library context. X509_new_with_libctx() should be used if possible.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

Add libctx support to CMS.

-Public CMS methods that create a CMS_ContentInfo object now have variants that also add a libctx and propq.
 This includes CMS_ContentInfo_new_with_libctx(), CMS_sign_with_libctx(), CMS_data_create_with_libctx(),
 CMS_digest_create_with_libctx(), CMS_EncryptedData_encrypt_with_libctx(), CMS_EnvelopedData_create_with_libctx().
-Added CMS_ReceiptRequest_create0_with_libctx().
-Added SMIME_read_CMS_ex() so that a new CMS_ContentInfo object (created using CMS_ContentInfo_new_with_libctx()) can
be passed to the read.
-d2i_CMS_bio() has been modified so that after it loads the CMS_ContentInfo() it then resolves any subobjects that require
 the libctx/propq (such as objects containing X509 certificates).

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11884)

rand: fix typo in parameter name

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/12608)

Update gost-engine to fix API rename

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12599)

Use .cnf for config files, not .conf

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12592)

Add evp_test fixes.

Changed many tests so they also test fips (and removed 'availablein = default' from some tests).
Seperated the monolithic evppkey.txt file into smaller maintainable groups.
Changed the availablein option so it must be first - this then skips the entire test before any fetching happens.
Changed the code so that all the OPENSSL_NO_XXXX tests are done in code via methods such as is_cipher_disabled(alg),
before the fetch happens.
Added missing libctx's found by adding a libctx to test_evp.
Broke up large data files for cipher, kdf's and mac's into smaller pieces so they no longer need 'AvailableIn = default'
Added missing algorithm aliases for cipher/digests to the providers.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12236)

drbgtest: avoid a memory leak

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/12509)

rand_drbg: remove RAND_DRBG.

The RAND_DRBG API did not fit well into the new provider concept as
implemented by EVP_RAND and EVP_RAND_CTX. The main reason is that the
RAND_DRBG API is a mixture of 'front end' and 'back end' API calls
and some of its API calls are rather low-level. This holds in particular
for the callback mechanism (RAND_DRBG_set_callbacks()) and the RAND_DRBG
type changing mechanism (RAND_DRBG_set()).

Adding a compatibility layer to continue supporting the RAND_DRBG API as
a legacy API for a regular deprecation period turned out to come at the
price of complicating the new provider API unnecessarily. Since the
RAND_DRBG API exists only since version 1.1.1, it was decided by the OMC
to drop it entirely.

Other related changes:

Use RNG instead of DRBG in EVP_RAND documentation.  The documentation was
using DRBG in places where it should have been RNG or CSRNG.

Move the RAND_DRBG(7) documentation to EVP_RAND(7).

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/12509)

PROV: Fix MSBLOB / PVK deserializer

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12601)

EVP: Fix the returned value for ASN1_PKEY_CTRL_DEFAULT_MD_NID

Trust the returned value from EVP_PKEY_get_default_digest_name()!  It
mimics exactly the values that EVP_PKEY_get_default_digest_nid() is
supposed to return, and that value should simply be passed unchanged.
Callers depend on it.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12586)

TEST: Adjust the serdes test to include MSBLOB and PVK

Because PVK uses RC4, we must ensure that default + legacy providers
are active.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12574)

PEM: Fix i2b_PvK to use EVP_Encrypt calls consistently

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12574)

PROV: Add MSBLOB and PVK to DSA and RSA deserializers

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12574)

DESERIALIZER: Adjust to allow the use several deserializers with same name

A key type may be deserialized from one of several sources, which
means that more than one deserializer with the same name should be
possible to add to the stack of deserializers to try, in the
OSSL_DESERIALIZER_CTX collection.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12574)

PEM: Make general MSBLOB reader functions exposed internally

Fly-by fix is to move crypto/include/internal/pem_int.h to
include/internal/pem.h.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12574)

RSA: Be less strict on PSS parameters when exporting to provider

We have a key in test/recipes/30-test_evp_data/evppkey.txt with bad
PSS parameters (RSA-PSS-BAD), which is supposed to trigger signature
computation faults.  However, if this key needs to be exported to the
RSA provider implementation, the result would be an earlier error,
giving the computation that's supposed to be checked n chance to even
be reached.

Either way, the legacy to provider export is no place to validate the
values of the key.

We also ensure that the provider implementation can handle and detect
signed (negative) saltlen values.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12583)

mac: add some consistency to setting the XXX_final output length.

The various MACs were all over the place with respects to what they did with
the output length in the final call.  Now they all unconditionally set the
output length and the EVP layer handles the possibility of a NULL pointer.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12582)

gettables: documentation changes to pass the provider context.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12581)

gettables: test changes to pass the provider context.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12581)

gettables: provider changes to pass the provider context.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12581)

gettables: core changes to pass the provider context.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12581)

Add new APIs to get PKCS12 secretBag OID and value

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10063)

apps/pkcs12: Change defaults from RC2 to PBES2 with PBKDF2

Fixes #11672

Add "-legacy" option to load the legacy provider and
fall back to the old legacy default algorithms.

doc/man1/openssl-pkcs12.pod.in: updates documentation about the new
"-legacy" option

Signed-off-by: Sahana Prasad <sahana@redhat.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12540)

Prepare for 3.0 alpha 7

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>

Prepare for release of 3.0 alpha 6

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>

Update copyright year

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12595)

Fix provider cipher reinit after init/update with a partial update block.

The test added previously used a 16 byte block during the update which does not cause internal buffering in the provider.
Some internal variables related to the buffering were not being cleared in the init, which meant that the second
update would use the buffered data from the first update.
Added test for this scenario with exclusions for ciphers that do not support partial block updates.

Found by guidovranken.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12523)

Mark an argument of an inline function as unused

This allows users of this header file to compile their own code with
the gcc option -Wunused-parameter.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12459)

openssl-cmp.pod.in: Update and extend example using Insta Demo CA

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12534)

DESERIALIZER: Fix EVP_PKEY construction by export

When the keymgmt provider and the deserializer provider differ,
deserialization uses the deserializer export function instead of the
keymgmt load, with a selection of what parts should be exported.  That
selection was set to OSSL_KEYMGMT_SELECT_ALL_PARAMETERS when it should
have been OSSL_KEYMGMT_SELECT_ALL.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12571)

Fix error message on setting cert validity period in apps/cmp.c

Fixes #12268

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

apps: Correct and extend diagnostics of parse_name()

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

Add 'section=...' info in error output of X509V3_EXT_nconf() as far as appropriate

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

Rename misleading X509V3_R_INVALID_NULL_NAME to X509V3_R_INVALID_EMPTY_NAME

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

Correct confusing X509V3 conf error output by removing needless 'section:<NULL>' etc.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

Correct misleading diagnostics of OBJ_txt2obj on unknown object name

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

apps/cmp.c: Defer diagnostic output on server+proxy to be contacted

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12296)

PROV: Make the DER to KEY deserializer decode parameters too

It should be noted that this may be dodgy if we ever encounter
parameter objects that look like something else.  However, experience
with the OSSL_STORE 'file:' loader, which does exactly this kind of
thing, has worked fine so far.

A possibility could be that to decode parameters specifically, we
demand that there's an incoming data type specifying this, which
demands by extension that parameters can only come from a file format
that has the parameter type encoded, such as PEM.  This would be a
future effort.

Fixes #12568

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12569)

Coverity Fixes for issue #12531

Fixes #12531 on master branch.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12557)

Change the provider implementation of X942kdf to use wpacket to do der encoding of sharedInfo

Added der_writer functions for writing octet string primitives.
Generate OID's for key wrapping algorithms used by X942 KDF.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12554)

Add CHANGES.md entry for SSL_set1_host()/SSL_add1_host() taking IP literals

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9201)

Disallow setting more than one IP address with SSL_add1_host()

The X509_VERIFY_PARAM can only take a single IP address, although it can
have multiple hostnames. When SSL_add1_host() is given an IP address,
don't accept it if there is already one configured.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9201)

Fix certificate validation for IPv6 literals in sconnect demo

Instead of naïvely trying to truncate at the first colon, use
BIO_get_conn_hostname(). That handles IPv6 literals correctly, even
stripping the [] from around them.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9201)

Make SSL_set1_host() and SSL_add1_host() take IP addresses

There is a slight mismatch here because X509_VERIFY_PARAM copes only
with a single IP address, and doesn't let it be cleared once it's set.
But this fixes up the major use case, making things easier for users to
get it right.

The sconnect demo now works for Legacy IP literals; for IPv6 it needs to
fix up the way it tries to split the host:port string, which will happen
in a subsequent patch.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9201)

81-test_cmp_cli.t: Skip tests with mock server if server cannot be started

Fixes #12514

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12537)

Fix an ENGINE leak in asn1_item_digest_with_libctx

Commit 6725682d introduced a call to ENGINE_get_digest_engine() into
the function asn1_item_digest_with_libctx() to determine whether there
is an ENGINE registered to handle the specified digest. However that
function increases the ref count on the returned ENGINE object, so it
must be freed.

Fixes #12558

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/12560)

DESERIALIZER: Small bugfix in the deser_process()

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

DESERIALIZER: Make OSSL_DESERIALIZER_from_{bio,fp} use BIO_tell() / BIO_seek()

Depending on the BIO used, using BIO_reset() may lead to "interesting"
results.  For example, a BIO_f_buffer() on top of another BIO that
handles BIO_reset() as a BIO_seek(bio, 0), the deserialization process
may find itself with a file that's rewound more than expected.

Therefore, OSSL_DESERIALIZER_from_{bio,fp}'s behaviour is changed to
rely purely on BIO_tell() / BIO_seek(), and since BIO_s_mem() is used
internally, it's changed to handle BIO_tell() and BIO_seek() better.

This does currently mean that OSSL_DESERIALIZER can't be easily used
with streams that don't support BIO_tell() / BIO_seek().

Fixes #12541

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

DESERIALIZER: Refactor the constructor setting API

It's not the best idea to set a whole bunch of parameters in one call,
that leads to functions that are hard to update.  Better to re-model
this into several function made to set one parameter each.

This also renames "finalizer" to "constructor", which was suggested
earlier but got lost at the time.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

TEST: Add testutil tests to compare unterminated strings of different lengths

We use this in test/serdes_test.c, to compare serializations into PEM,
which aren't necessarily terminated with a NUL byte when they were
written to a BIO_s_mem().

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

DESERIALIZER: Add deserializers for the rest of our asymmetric key types

To be able to implement this, there was a need for the standard
EVP_PKEY_set1_, EVP_PKEY_get0_ and EVP_PKEY_get1_ functions for
ED25519, ED448, X25519 and X448, as well as the corresponding
EVP_PKEY_assign_ macros.  There was also a need to extend the list of
hard coded names that EVP_PKEY_is_a() recognise.

Along with this, OSSL_FUNC_keymgmt_load() are implemented for all
those key types.

The deserializers for these key types are all implemented generically,
in providers/implementations/serializers/deserializer_der2key.c.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

DESERIALIZER: Make it possible to deserialize public keys too

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

DESERIALIZER: Rethink password handling

The OSSL_DESERIALIZER API makes the incorrect assumption that the
caller must cipher and other pass phrase related parameters to the
individual desserializer implementations, when the reality is that
they only need a passphrase callback, and will be able to figure out
the rest themselves from the input they get.

We simplify it further by never passing any explicit passphrase to the
provider implementation, and simply have them call the passphrase
callback unconditionally when they need, leaving it to libcrypto code
to juggle explicit passphrases, cached passphrases and actual
passphrase callback calls.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

RSA: Better synchronisation between ASN1 PSS params and RSA_PSS_PARAMS_30

This is needed so RSA keys created from different code paths have a
chance to compare as equal.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

DER writer: Make context-specific tags constructed (i.e. explicit)

For now, that's what we see being used.  It's possible that we will
have to figure out a way to specific if these should be implicit or
explicit on a case by case basis.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

PROV: Fix small logic error in ec_kmgmt.c matching function

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12544)

Add OSSL_CMP_MSG_write(), use it in apps/cmp.c

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12421)

Export ossl_cmp_msg_load() as OSSL_CMP_MSG_read(), use it in apps/cmp.c

Fixes #12403

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12421)

apps/cmp.c: Improve documentation of -recipient option

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12421)

deserialisation: add deserialisation to the base provider

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12104)

serialisation: Add a built-in base provider.

Move the libcrypto serialisation functionality into a place where it can
be provided at some point. The serialisation still remains native in the
default provider.

Add additional code to the list command to display what kind of serialisation
each entry is capable of.

Having the FIPS provider auto load the base provider is a future
(but necessary) enhancement.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12104)

unify spelling of serialize

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12104)

Fix test_cmp_cli for extended tests

The test_cmp_cli was failing in the extended tests on cross-compiled
mingw builds. This was due to the test not using wine when it should do.
The simplest solution is to just skip the test in this case.

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12555)

Don't fallback to legacy in DigestSignInit/DigestVerifyInit too easily

The only reason we should fallback to legacy codepaths in DigestSignInit/
DigestVerifyInit, is if we have an engine, or we have a legacy algorithm
that does not (yet) have a provider based equivalent (e.g. SM2, HMAC, etc).
Currently we were falling back even if we have a suitable key manager but
the export of the key fails. This might be for legitimate reasons (e.g.
we only have the FIPS provider, but we're trying to export a brainpool key).
In those circumstances we don't want to fallback to the legacy code.

Therefore we tighten then checks for falling back to legacy. Eventually this
particular fallback can be removed entirely (once all legacy algorithms have
provider based key managers).

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12550)

Export crm_new() of cmp_msg.c under the name OSSL_CMP_CTX_setup_CRM()

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12431)

Streamline the CMP request session API, adding the generalized OSSL_CMP_exec_certreq()

Fixes #12395

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12431)

[test][ectest] Minor touches to custom_generator_test

Minor changes to `custom_generator_test`:

- this is to align to the 1.1.1 version of the test (simplify the code
  as there is no need to use `EC_GROUP_get_field_type()`)
- add comment to explain how the buffer size is computed

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12507)

[test] Vertically test explicit EC params API patterns

This commit adds a new test (run on all the built-in curves) to create
`EC_GROUP` with **unknown** *explicit parameters*: from a built-in group
we create an alternative group from scratch that differs in the
generator used.

At the `EC_GROUP` layer we perform a basic math check to ensure that the
math on the alternative group still makes sense, using comparable
results from the origin group.

We then create two `EC_KEY` objects on top of this alternative group and
run key generation from the `EC_KEY` layer.

Then we promote these two `EC_KEY`s to `EVP_PKEY` objects and try to
run the derive operation at the highest abstraction layer, comparing
results in both directions.

Finally, we create provider-native keys using `EVP_PKEY_fromdata` and
data derived from the previous objects, we compute an equivalent shared
secret from these provider keys, and compare it to the result obtained
from the previous steps.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12507)

namemap: fix threading issue

The locking was too fine grained when adding entries to a namemap.
Refactored the working code into unlocked functions and call these with
appropriate locking.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12545)

Fix a test_verify failure

A recently added certificate in test/certs expired causing test_verify to fail.
This add a replacement certificate with a long expiry date.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12549)

Deprecate -nodes in favor of -noenc in pkcs12 and req app

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12495)

TEST: Add RSA-PSS cases in test/serdes_test.c

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12492)

PROV: Add a DER to RSA-PSS deserializer implementation

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12492)

EVP, PROV: Add misc missing bits for RSA-PSS

- EVP_PKEY_is_a() didn't recognise "RSA-PSS" for legacy keys.
- The RSA-PSS keymgmt didn't have a OSSL_FUNC_keymgmt_match() function.
- ossl_prov_prepare_rsa_params() didn't return 1 for unrestricted
  RSA-PSS params.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12492)

Fix no-ec2m

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12526)

DER to RSA deserializer: fix inclusion

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12535)

Fix no-dh and no-dsa

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12525)

Update EVP_EncodeInit.pod

Fix EVP_EncodeBlock description using incorrect parameter name for encoding length

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/12518)

EVP: Fix key type check logic in evp_pkey_cmp_any()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)

TEST: Update the serialization/deserialization test with legacy PEM encryption

This adds legacy PEM variants of already existing tests.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)

PROV: Update the PEM to DER deserializer to handle encrypted legacy PEM

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)

TEST: Update the serialization/deserialization test with encryption

This adds variants of already existing tests, but where the object
is encrypted / decrypted along the way as well.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)

PROV: Update the DER to RSA deserializer to handle encrypted PKCS#8

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)

DESERIALIZER: Implement decryption of password protected objects

This implements these functions:

OSSL_DESERIALIZER_CTX_set_cipher()
OSSL_DESERIALIZER_CTX_set_passphrase()
OSSL_DESERIALIZER_CTX_set_passphrase_ui()
OSSL_DESERIALIZER_CTX_set_passphrase_cb()

To be able to deal with multiple deserializers trying to work on the
same byte array and wanting to decrypt it while doing so, the
deserializer caches the passphrase.  This cache is cleared at the end
of OSSL_DESERIALIZER_from_bio().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)

SERIALIZER: No enc argument for OSSL_SERIALIZER_CTX_set_passphrase_cb()

Serialization will only encrypt, so there's no point telling
OSSL_SERIALIZER_CTX_set_passphrase_cb() that's going to happen.

We fix the declaration of OSSL_DESERIALIZER_CTX_set_passphrase_cb()
the same way.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12410)