Fix range_should_be_prefix() to actually return the correct result
authorMatt Caswell <matt@openssl.org>
Fri, 10 Jun 2022 14:58:58 +0000 (15:58 +0100)
committerTomas Mraz <tomas@openssl.org>
Tue, 28 Jun 2022 15:16:43 +0000 (17:16 +0200)
commit2c6550c6db9b1b69dc24f968b4ceb534edcf4841
tree000115ef49aedb7d4f69be17d9cc459eb68792df
parentdaa014b2061b94832415b1177ff2db6a17fc7274
Fix range_should_be_prefix() to actually return the correct result

range_should_be_prefix() was misidentifying whether an IP address range
should in fact be represented as a prefix. This was due to a bug introduced
in commit 42d7d7dd which made this incorrect change:

-    OPENSSL_assert(memcmp(min, max, length) <= 0);
+    if (memcmp(min, max, length) <= 0)
+        return -1;

This error leads to incorrect DER being encoded/accepted.

Reported by Theo Buehler (@botovq)

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18524)

(cherry picked from commit 30532e59f475e0066c030693e4d614311a9e0cae)
crypto/x509/v3_addr.c
test/v3ext.c