From: David Benjamin <davidben@google.com>
Date: Thu, 28 Apr 2016 00:02:35 +0000 (-0400)
Subject: Avoid overflow issues in X509_cmp.
X-Git-Tag: OpenSSL_1_1_0-pre6~994
X-Git-Url: https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff_plain;h=87a8405b66e94cbfc40c44104c3b52f342a623d5

Avoid overflow issues in X509_cmp.

The length is a long, so returning the difference does not quite work.

Thanks to Torbjörn Granlund for noticing.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
---

diff --git a/crypto/x509/x509_cmp.c b/crypto/x509/x509_cmp.c
index d3b2c199b9..831cfb70f0 100644
--- a/crypto/x509/x509_cmp.c
+++ b/crypto/x509/x509_cmp.c
@@ -187,9 +187,10 @@ int X509_cmp(const X509 *a, const X509 *b)
         return rv;
     /* Check for match against stored encoding too */
     if (!a->cert_info.enc.modified && !b->cert_info.enc.modified) {
-        rv = (int)(a->cert_info.enc.len - b->cert_info.enc.len);
-        if (rv)
-            return rv;
+        if (a->cert_info.enc.len < b->cert_info.enc.len)
+            return -1;
+        if (a->cert_info.enc.len > b->cert_info.enc.len)
+            return 1;
         return memcmp(a->cert_info.enc.enc, b->cert_info.enc.enc,
                       a->cert_info.enc.len);
     }