Don't send -1 as the length of the hmac key
The dgst app was using an undocumented behaviour in the
EVP_PKEY_new_raw_private_key() function when setting a key length for
a MAC. The old EVP_PKEY to MAC bridge, probably by accident, converts a
-1 length to a strlen() call, by virtue of the fact that it eventually
calls ASN1_STRING_set() which has this feature.
As noted above this is undocumented, and unexpected since the len
parameter to EVP_PKEY_new_raw_private_key() is an unsigned value (size_t).
In the old bridge it was later (silently) cast to an int, and therefore
the original -1 value was restored. This only works because sizeof(int) <=
sizeof(size_t). If we ever run on a platform where sizeof(int) >
sizeof(size_t) then it would have failed. The behaviour also doesn't hold
for EVP_PKEY_new_raw_private_key() in general - only when the old MAC
bridge was in use.
Rather than restore the original behaviour I think it is best to simply
fix the dgst app to not assume it exists. We should not bake in this
backwards and inconsistent behaviour.
Fixes #12837
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/12850)