Cast to an unsigned type before negating
authorKurt Roeckx <kurt@roeckx.be>
Sun, 17 Jul 2016 13:28:09 +0000 (15:28 +0200)
committerKurt Roeckx <kurt@roeckx.be>
Wed, 20 Jul 2016 17:25:16 +0000 (19:25 +0200)
llvm's ubsan reported:
runtime error: negation of -9223372036854775808 cannot be represented in type
'long'; cast to an unsigned type to negate this value to itself

Found using afl

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1325

crypto/asn1/x_long.c
crypto/bio/b_print.c

index 0af7875..e86e4c7 100644 (file)
@@ -76,7 +76,7 @@ static int long_i2c(ASN1_VALUE **pval, unsigned char *cont, int *putype,
      * set.
      */
     if (ltmp < 0)
-        utmp = -ltmp - 1;
+        utmp = -(unsigned long)ltmp - 1;
     else
         utmp = ltmp;
     clen = BN_num_bits_word(utmp);
index 6808cdc..a46d8b1 100644 (file)
@@ -451,7 +451,7 @@ fmtint(char **sbuffer,
     if (!(flags & DP_F_UNSIGNED)) {
         if (value < 0) {
             signvalue = '-';
-            uvalue = -value;
+            uvalue = -(unsigned LLONG)value;
         } else if (flags & DP_F_PLUS)
             signvalue = '+';
         else if (flags & DP_F_SPACE)