Fix arithmetic expression overflow

If the value of a->length is large (>= 2^12), then an integer overflow will
occur for the signed type, which according to the C standard is UB.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/23274)

(cherry picked from commit 486ab0fb003d05f89620662260486d31bd3faa8c)

diff --git a/crypto/objects/obj_dat.c b/crypto/objects/obj_dat.c
index ec78962d7b0b68e852b9871269eb43161ba71b1a..40a3eef93cd10f1ae692402807c5f4d07a0764ea 100644 (file)
--- a/crypto/objects/obj_dat.c
+++ b/crypto/objects/obj_dat.c
@@ -136,7 +136,7 @@ static unsigned long added_obj_hash(const ADDED_OBJ *ca)
     a = ca->obj;
     switch (ca->type) {
     case ADDED_DATA:
-        ret = a->length << 20L;
+        ret = (unsigned long)a->length << 20UL;
         p = (unsigned char *)a->data;
         for (i = 0; i < a->length; i++)
             ret ^= p[i] << ((i * 3) % 24);