Fix arithmetic expression overflow

If the value of a->length is large (>= 2^12), then an integer overflow will
occur for the signed type, which according to the C standard is UB.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/23274)

diff --git a/crypto/objects/obj_dat.c b/crypto/objects/obj_dat.c
index ebde56247761c780cc54abbd6951ccf180c1d42b..30a63bbbd9b8ab54aadca82e12186cbad674b847 100644 (file)
--- a/crypto/objects/obj_dat.c
+++ b/crypto/objects/obj_dat.c
@@ -128,7 +128,7 @@ static unsigned long added_obj_hash(const ADDED_OBJ *ca)
     a = ca->obj;
     switch (ca->type) {
     case ADDED_DATA:
-        ret = a->length << 20L;
+        ret = (unsigned long)a->length << 20UL;
         p = (unsigned char *)a->data;
         for (i = 0; i < a->length; i++)
             ret ^= p[i] << ((i * 3) % 24);