APPS: Add OSSL_STORE loader for engine keys

The idea is to be able to have our apps load engine keys using a URI:

    org.openssl.engine:{engineid}:{keyid}

This is legacy, but added for the time being to support keys given to
the application like this:

    -engine {engineid} -key {keyid} -keyform ENGINE

This latter form is recognised internally, and rewritten into the URI
form.

Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/13570)

Fix no-dtls

Ensure we correctly detect if DTLS has been disabled in the client auth
test_ssl_new tests.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13587)

Fix a compile error with the no-sock option

BIO_do_connect() can work even in no-sock builds (non socket based BIOs
have the right ctrls). Therefore we move the macro outside of the
OPENSSL_NO_SOCK guards

Fixes #12207

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13587)

openssl dgst: add option to specify output length for XOF

This adds the -xoflen option to control the output length of the XOF
algorithms, such as SHAKE128 and SHAKE256.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13245)

Adapt everything else to the updated OSSL_ENCODER_CTX_new_by_EVP_PKEY()

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13545)

ENCODER: Don't pass libctx to OSSL_ENCODER_CTX_new_by_EVP_PKEY()

The passed 'pkey' already contains a library context, and the encoder
implementations should be found within the same context, so passing an
explicit library context seems unnecessary, and potentially dangerous.

It should be noted that it's possible to pass an EVP_PKEY with a
legacy internal key.  The condition there is that it doesn't have a
library context assigned to it, so the NULL library context is used
automatically, thus requiring that appropriate encoders are available
through that context.

Fixes #13544

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13545)

cmp_client.c: Fix indentation and remove empty line

Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13488)

cmp_client.c: Remove dead code of variable 'txt' in cert_response()

Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13488)

Don't Overflow when printing Thawte Strong Extranet Version

When printing human readable info on the Thawte Strong Extranet extension
the version number could overflow if the version number == LONG_MAX. This
is undefined behaviour.

Issue found by OSSFuzz.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/13452)

Add EVP_KDF-X942 to the fips module

The X942 KDF had been modified so that it supports all optional fields - not
just the fields used by CMS.

As there are 2 types of KDF for X942 - this has been made a bit clearer
by adding an X942KDF-ASN1 alias. X942KDF-CONCAT has also been added as an
alias of X963KDF.

This work was instigated as a result of the ACVP tests optionally being
able to use keybits for the supp_pubinfo field.
Setting the parameter OSSL_KDF_PARAM_X942_USE_KEYBITS to 0 allows this
to be disabled.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13418)

Fix s390 EDDSA HW support in providers.

Fixes #12476

Note this stopped working when ECX was swapped over to using
providers. The ECX_KEY keygen and exchange were converted, but not the ED
signing support.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/13508)

Print random seed on test failure.

Tests randomisation wasn't reliably printing the seed used on failure in the CIs.

Fixes: #13572
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13573)

Correct system guessing for darwin64-arm64 target

Previously the system guessing logic would incorrectly guess
i686-apple-darwin as the fallback for any unspecified architecture
that is a Darwin target

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13517)

apps/pkcs12.c: Correct default legacy algs and make related doc consistent

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13534)

Fix simpledynamic.c - a typo and missed a header

CLA: trivial

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13584)

TEST: Add a simple module loader, and test the FIPS module with it

This very simple module loader is only linked with the standard C
library, so cannot and should not provide any other symbol to the
module it tries to load.  It can thereby be used to verify that the
module it tries to load doesn't have any surprising dependencies when
it's supposed to be self contained.

A test recipe is added to verify the FIPS module with this loader.

Fixes #11020

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13507)

TEST: Break out the local dynamic loading code from shlibloadtest.c

The result is "simpledynamic.c", or "sd" for short.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13507)

endecode_test.c: Add warning that 512-bit DH key size is for testing only

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13568)

test cleanup: move helper .c and .h files to test/helpers/

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13568)

remove obsolete test/drbg_cavs_data.h

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13568)

remove obsolete test/drbg_extra_test.h

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13568)

RSA: correct digestinfo_ripemd160_der[]

A couple of numbers were incorrect.

Fixes #13559

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13562)

aes/asm/aesv8-armx.pl: avoid 32-bit lane assignment in CTR mode

ARM Cortex-A57 and Cortex-A72 cores running in 32-bit mode are affected
by silicon errata #1742098 [0] and #1655431 [1], respectively, where the
second instruction of a AES instruction pair may execute twice if an
interrupt is taken right after the first instruction consumes an input
register of which a single 32-bit lane has been updated the last time it
was modified.

This is not such a rare occurrence as it may seem: in counter mode, only
the least significant 32-bit word is incremented in the absence of a
carry, which makes our counter mode implementation susceptible to these
errata.

So let's shuffle the counter assignments around a bit so that the most
recent updates when the AES instruction pair executes are 128-bit wide.

[0] ARM-EPM-049219 v23 Cortex-A57 MPCore Software Developers Errata Notice
[1] ARM-EPM-012079 v11.0 Cortex-A72 MPCore Software Developers Errata Notice

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13504)

Fix builds that specify both no-dh and no-ec

Various sections of code assumed that at least one of dh or ec would be
available. We also now also need to handle cases where a provider has
a key exchange algorithm and TLS-GROUP that we don't know about.

Fixes #13536

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13549)

Fix TLS1.2 CHACHA20-POLY1305 ciphersuites with OPENSSL_SMALL_FOOTPRINT

If OPENSSL_SMALL_FOOTPRINT was defined then the CHACHA20-POLY1305
implementation for TLS went down a different codepath that failed to
adjust the payload length to remove the tag.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13513)

Fix instances of pointer addition with the NULL pointer

Addition using the NULL pointer (even when adding 0) is undefined
behaviour. Recent versions of ubsan are now complaining about this, so
we fix various instances.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13513)

endecode_test.c: Significant speedup in generating DH and DHX keys

Fixes #13495

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13552)

Collapse two identical if statements into a single body.

These two bodies should be grouped together anyway as the reason for
the call to BIO_flush() is to permit using BIO_set_ktls_ctrl_msg().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/13090)

Allow zero-byte writes to be reported as success.

When using KTLS, empty fragments sent as a mitigation for known-IV
weakenesses in TLS 1.0 are sent as writes of 0 bytes.  The TLS header
and trailer are added to the empty fragment by the kernel.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/13090)

TEST: Fix path length in test/ossl_store_test.c

The URI length was set to 80 chars, but the URI being built up may
need more space, all depending on the paths used to get to the files
that are to be loaded.  If the result needs more than 80 chars, the
test will fail.

Fixed by using PATH_MAX.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13546)

[DOC]Fix two broken links in INSTALL.md; Change name of zlib flag to the current one.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13519)

README: Move Travis link to .com from .org.

CLA: trivial

Fixes #12423

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13538)

ERR: drop err_delete_thread_state() TODO marker

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13466)

ERR: Drop or deprecate dangerous or overly confusing functions

ERR_get_error_line() is deprecated, and ERR_get_error_func() and
ERR_get_error_data() are removed in favor of ERR_get_error_all(),
since they pop the error record, leaving the caller with only partial
error record data and no way to get the rest if the wish.

If it's desirable to retrieve data piecemeal, the caller should
consider using the diverse ERR_peek functions and finish off with
ERR_get_error().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13466)

Fix no-engine

If we specify no-engine then this should cascade to also mean
no-dynamic-engine. The store test was only checking whether
dynamic-engine was disabled, meaning that some tests were failing
in a no-engine build.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13502)

X509_dup: fix copying of libctx and propq using new ASN1_OP_DUP_POST cb operation

Fixes #12680

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12687)

asn1t.h: Improve comments documenting ASN1_ITYPE_... and the 'funcs' field

Also move the #define(s) for the ASN1_ITYPE_.. before their first use.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12687)

EVP_PKEY & DSA: Make DSA EVP_PKEY_CTX parameter ctrls / setters more available

EVP_PKEY_CTX_set_dsa_ functions were only available when DSA was
enabled ('no-dsa' not configured).  However, that makes it impossible
to use these functions with an engine or a provider that happens to
implement DSA.  This change solves that problem by shuffling these
functions to more appropriate places.

Fixes #13529

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13530)

Update bio_ok.c

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13515)

Remove d2i_DHparams.pod and move documentation to d2i_RSAPrivateKey.pod

d2i_RSAPrivateKey.pod is the more generic page for these deprecated
functions and provides advice and guidance on how to translate the old
style functions into new ones.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Updates the CHANGES.md entry regarding DH deprecation

Extend the existing CHANGES.md entry with information about the
additional functions that have also been deprecated.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Convert DH deprecations to the new way of deprecating functions

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Deprecate more DH functions

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Don't test a deprecated function in a no-deprecated build

EVP_PKEY_set1_DH is deprecated so there is no need to test it in a
no-deprecated build.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Remove fuzzing of deprecated functions in a no-deprecated build

d2i_DHparams and i2d_DHparam as well as the equivalent DHX functions are
deprecated.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Deprecate the DHparams and DHxparams PEM routines

The functions return a DH object and therefore need to be deprecated.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Deprecate EVP_PKEY_assign_DH and other similar macros

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Deprecate functions for getting and setting DH values in an EVP_PKEY

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

Deprecate DH_new as well as i2d_DHparams and d2i_DHparams

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13138)

TEST: Adapt test/errtest for the 'no-err' configuration

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13510)

ERR: Restore the similarity of ERR_print_error_cb() and ERR_error_string_n()

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13510)

Fix no-deprecated configuration

pem_read_depr_test needed to be setup in build info so that it only
exists inside an IF[{- !$disabled{'deprecated-3.0'} -}] block.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/13543)

Fix EVP_CIPHER_CTX_set_padding for legacy path

Fixes #13057

When using an engine, there is no cipher->prov so a call to
EVP_CIPHER_CTX_set_padding() returns an error when
evp_do_ciph_ctx_setparams() is called. For the legacy path it needs to
avoid doing the call and just return 1.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/13437)

EVP: don't touch the lock for evp_pkey_downgrade

This commit tries to address a locking issue in evp_pkey_reset_unlocked
which can occur when it is called from evp_pkey_downgrade.

evp_pkey_downgrade will acquire a lock for pk->lock and if successful
then call evp_pkey_reset_unlocked. evp_pkey_reset_unlocked will call
memset on pk, and then create a new lock and set pk->lock to point to
that new lock. I believe there are two problems with this.

The first is that after the call to memset, another thread would try to
acquire a lock for NULL as that is what the value of pk->lock would be
at that point.

The second issue is that after the new lock has been assigned to
pk->lock, that lock is different from the one currently locked so
another thread trying to acquire the lock will succeed which can lead to
strange behaviour. More details and a reproducer can be found in the
Refs link below.

This changes the evp_pkey_reset_unlocked to not touch the lock
and the creation of a new lock is done in EVP_PKEY_new.

Refs:
https://github.com/danbev/learning-libcrypto/blob/master/notes/issues.md#openssl-investigationtroubleshooting
https://github.com/nodejs/node/issues/29817

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13374)

Documentation improvements for EVP_DigestInit_ex and related functions

Documenting when EVP_MD_CTX_reset() is implicitly called and when
type can be set to NULL.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13402)

Fix regression in EVP_DigestInit_ex: crash when called with NULL type

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13402)

Add test for no reset after DigestFinal_ex and DigestFinalXOF

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13402)

EVP_DigestFinalXOF must not reset the EVP_MD_CTX

It does not do it in legacy path and 1.1.1 so that must not change.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13402)

Fix no-posix-io

The "multi" variable should only be used within HTTP_DAEMON guards.
However there were a few spots where this was not the case, which
causes no-posix-io builds to fail.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13498)

APPS: Modify apps/cmp.c to use set_base_ui_method() for its -batch option

Fixes #13511

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13512)

APPS: Make it possible for apps to set the base (fallback) UI_METHOD

The apps UI method acts as a proxy that bases its activity on a base
(was called fallback) UI_METHOD, which defaults to UI_OpenSSL() under
normal circumstances.

However, some apps might want to have it based on another UI_METHOD,
such as UI_null() to avoid prompting (typical for a -batch run).  The
new function set_base_ui_method() allows them to do precisely this.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13512)

Prepare for 3.0 alpha 10

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>

Prepare for release of 3.0 alpha 9

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>

Update copyright year

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13533)

DOC: Add note on how to terminate an OSSL_PARAM array

The examples are also updated to have correct terminators.

doc/man3/OSSL_PARAM.pod is deliberately written with no help from the
constructor macros described in OSSL_PARAM_int.pod.  Therefore, use of
OSSL_PARAM_END isn't shown directly here, only leaving a link to its
man-page to indicate that there is that option.

Fixes #11280

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13478)

Fix no-rc2

Skip a test that relies on RC2 being present in a no-rc2 build.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13496)

Remove deprecation warning suppression from genpkey

genpkey was supressing deprecation warnings in order to support ENGINE
functionality. We move all of that into a separate file so that we don't
need to suppress the warnings anymore.

Fixes #13118

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13454)

apps/cmp.c: fix crash with -batch option on OPENSSL_NO_UI_CONSOLE

Also make clear we cannot use get_ui_method() at this point.

Fixes #13494

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13497)

apps/cmp.c: Improve description of key loaded due to -newkew option

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13497)

re-encrypt 81-test_cmp_cli_data/Mock/signer.p12 with AES-256-CBC (avoiding DES)

Fixes #13494

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13497)

Re-enable testing of ciphersuites

Commit be9d82bb3 inadvertently disabled ciphersuite testing. This masked
some issues. Therefore we fix this testing.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13378)

Fix RC4-MD5 based ciphersuites

The RC4-MD5 ciphersuites were not removing the length of the MAC when
calculating the length of decrypted TLS data. Since RC4 is a streamed
cipher that doesn't use padding we separate out the concepts of fixed
length TLS data to be removed, and TLS padding.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13378)

Ensure Stream ciphers know how to remove a TLS MAC

We previously updated the block ciphers to know how to remove a TLS
MAC when using Encrypt-then-MAC. We also need to do the same for stream
ciphers.

Fixes #13363

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13378)

Test that OSSL_STORE can load various types of params

There have been instances where OSSL_STORE got confused between DSA and
DH params (e.g. see issue #13046) due the DER encoding of DH and DSA params
looking identical. Therefore we test that we get the types that we expect.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13329)

Test various deprecated PEM_read_bio_* APIs

Add tests for various deprecated PEM_read_bio_*() functions to ensure
they can still read the various files.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13329)

Don't forget the datatype when decoding a PEM file

The OSSL_STORE code was forgetting the datatype that we read from the
PEM header when decoding the DER.

Fixes #13046

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13329)

APPS: Guard use of IPv6 functions and constants with a check of AF_INET6

Fixes #13482

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13484)

Undeprecate the -dsaparam option in the dhparam app

The -dsaparam option was deprecated because it was previously using
deprecated functions in order to operate. This is no longer the case
and therefore does not need to be deprecated.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13485)

Add missing ERR_load_KDF_strings(3) to util/missingcrypto111.txt as well.

This partially reverts commit a308acb2c2809cb9ac30e8e987b2bdfb21f096e0.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13390)

ERR: Rebuild all generated error headers and source files

This is the result of 'make errors ERROR_REBUILD=-rebuild'

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13390)

Modify the ERR init functions to use the internal ERR string loaders

This deprecates all the ERR_load_ functions, and moves their definition to
separate C source files that can easily be removed when those functions are
finally removed.

This also reduces include/openssl/kdferr.h to include cryptoerr_legacy.h,
moves the declaration of ERR_load_ERR_strings() from include/openssl/err.h
to include/openssl/cryptoerr_legacy.h, and finally removes the declaration
of ERR_load_DSO_strings(), which was entirely internal anyway.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13390)

ERR: Modify util/mkerr.pl to produce internal err string loaders

This also modifies the .ec L statement to take a third file, which is
the internal header file to declare internal things.  This is only
useful for our internal declarations and will not affect engines.

Fixes #10527

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13390)

Turn on Github CI

As an interim measure until we work out our longer term CI strategy
this PR enables some basic CI tests using the Github CI capability.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13489)

Fix double-free in decoder_pkey.c

Fix for the issue #13472. The decoderctx has to be initialized in every
cycle as its constructor may not be called due to lazy evaluation of
the if-condition.

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13473)

TEST: Make our test data binary

Our test data (test/data.txt and test/data2.txt) are text files, but
declaring them binary means that there will be no line ending
transformation done on them.  This is necessary for testing on
non-Unix platforms, where certain tests could otherwise give results
that don't match expected results.

Fixes #13474

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13477)

Move some libssl global variables into SSL_CTX

disabled_enc_mask et al were global. Now that cipher loading is done
individually for each SSL_CTX, based on the libctx configured for that
SSL_CTX this means that some things will be disabled for one SSL_CTX but
not for another. The global variables set up the potential for different
SSL_CTXs to trample on each other. We move these variables into the SSL_CTX
structure.

Fixes #12040

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13465)

Add a test for the dhparam CLI application

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13231)

Remove some unneeded variables from dhparam

Previously changes left some variables behind that were no longer needed.
We now remove them.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13231)

Add encoder support to dhparam

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13231)

Convert dhparam to be fully based on EVP

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13231)

Support for Android NDK r22-beta1

I think builds using standalone toolchain are fine so I left them alone,
but `Configure` will fail if using the NDK directly because the
`platforms` and `sysroot` directories were removed.

If `sysroot` is missing, omit the `--sysroot` and `-gcc-toolchain`
arguments and use the triplet form clang command.

Also since `platforms` was being used for the default API level, use
`meta/platforms.json` instead if needed.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13434)

RSA: Fix guard mixup

A OSSL_DEPRECATEDIN_0_9_8 function was surrounded by a
OPENSSL_NO_DEPRECATED_3_0 guard.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13448)

x509_vfy.c: Remove superfluous assignment to 'ret' in check_chain()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13458)

util/fix-deprecation: DEPRECATEDIN conversion util for public headers

Use this for quick and easy conversion of old-style to new-style deprecation:

    perl util/fix-deprecation \
         < include/openssl/rsa.h > include/openssl/rsa.h.new
    mv include/openssl/rsa.h.new include/openssl/rsa.h

This is not a perfect utility, but it does the job.  It doesn't try to
re-indent, that's left for manual post processing.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13239)

ossl_cmp_certreq_new(): Fix POPO key mismatch in case newPkey is just public key

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13409)

CMP: prevent misleading PKIStatusInfo output if not response available

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13409)

apps/cmp.c: Improve diagnostics on -server URL parse error

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13409)

apps/cmp.c: Add diagnostics on config file section(s) used

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13409)

apps.c: re-enable loading single certs and CRLs over HTTP

Fixes #13403

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13404)

SSL: Change SSLerr() to ERR_raise()

This was probably due to a merge

Fixes #13449

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/13450)