Fix range_should_be_prefix() to actually return the correct result
range_should_be_prefix() was misidentifying whether an IP address range
should in fact be represented as a prefix. This was due to a bug introduced
in commit
42d7d7dd which made this incorrect change:
- OPENSSL_assert(memcmp(min, max, length) <= 0);
+ if (memcmp(min, max, length) <= 0)
+ return -1;
This error leads to incorrect DER being encoded/accepted.
Reported by Theo Buehler (@botovq)
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18524)
(cherry picked from commit
30532e59f475e0066c030693e4d614311a9e0cae)
(cherry picked from commit
2c6550c6db9b1b69dc24f968b4ceb534edcf4841)