Avoid overflow issues in X509_cmp.
authorDavid Benjamin <davidben@google.com>
Thu, 28 Apr 2016 00:02:35 +0000 (20:02 -0400)
committerRichard Levitte <levitte@openssl.org>
Fri, 29 Apr 2016 15:01:09 +0000 (17:01 +0200)
The length is a long, so returning the difference does not quite work.

Thanks to Torbjörn Granlund for noticing.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
crypto/x509/x509_cmp.c

index d3b2c199b973d8ebb66c6dfc44ff0ea7814a8083..831cfb70f02827359760909f4ac412dbb1dd3aa3 100644 (file)
@@ -187,9 +187,10 @@ int X509_cmp(const X509 *a, const X509 *b)
         return rv;
     /* Check for match against stored encoding too */
     if (!a->cert_info.enc.modified && !b->cert_info.enc.modified) {
-        rv = (int)(a->cert_info.enc.len - b->cert_info.enc.len);
-        if (rv)
-            return rv;
+        if (a->cert_info.enc.len < b->cert_info.enc.len)
+            return -1;
+        if (a->cert_info.enc.len > b->cert_info.enc.len)
+            return 1;
         return memcmp(a->cert_info.enc.enc, b->cert_info.enc.enc,
                       a->cert_info.enc.len);
     }