Fix unintended sign extension

The function CRYPTO_128_unwrap_pad uses an 8 byte AIV (Alternative Initial
Value). The least significant 4 bytes of this is placed into the local
variable |ptext_len|. This is done as follows:

    ptext_len = (aiv[4] << 24) | (aiv[5] << 16) | (aiv[6] << 8) | aiv[7];

aiv[4] is an unsigned char, but (aiv[4] << 24) is promoted to a *signed*
int - therefore we could end up shifting into the sign bit and end up with
a negative value. |ptext_len| is a size_t (typically 64-bits). If the
result of the shifts is negative then the upper bits of |ptext_len| will
all be 1.

This commit fixes the issue by explicitly casting to an unsigned int.

Reviewed-by: Richard Levitte <levitte@openssl.org>

diff --git a/crypto/modes/wrap128.c b/crypto/modes/wrap128.c
index ccb58c5a0b3147a3f7d0605c5817f79aca1d7e18..73718ae59feb1871742b8374d5c3c966e015804a 100644 (file)
--- a/crypto/modes/wrap128.c
+++ b/crypto/modes/wrap128.c
@@ -350,7 +350,10 @@ size_t CRYPTO_128_unwrap_pad(void *key, const unsigned char *icv,
      * LSB(32,AIV).
      */
 
-    ptext_len = (aiv[4] << 24) | (aiv[5] << 16) | (aiv[6] << 8) | aiv[7];
+    ptext_len =   ((unsigned int)aiv[4] << 24)
+                | ((unsigned int)aiv[5] << 16)
+                | ((unsigned int)aiv[6] <<  8)
+                |  (unsigned int)aiv[7];
     if (8 * (n - 1) >= ptext_len || ptext_len > 8 * n) {
         OPENSSL_cleanse(out, inlen);
         return 0;