From: Matt Caswell <matt@openssl.org>
Date: Wed, 6 Dec 2023 11:09:53 +0000 (+0000)
Subject: Avoid an infinite loop in BN_GF2m_mod_inv
X-Git-Tag: openssl-3.3.0-alpha1~500
X-Git-Url: https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff_plain;h=9c1b8f17ce2471ca37ee3936d07aed29aab10975

Avoid an infinite loop in BN_GF2m_mod_inv

If p is set to 1 when calling BN_GF2m_mod_inv then an infinite loop will
result. Calling this function set 1 when applications call this directly
is a non-sensical value - so this would be considered a bug in the caller.

It does not seem possible to cause OpenSSL internal callers of
BN_GF2m_mod_inv to call it with a value of 1.

So, for the above reasons, this is not considered a security issue.
Reported by Bing Shi.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Todd Short <todd.short@me.com>
(Merged from https://github.com/openssl/openssl/pull/22960)
---

diff --git a/crypto/bn/bn_gf2m.c b/crypto/bn/bn_gf2m.c
index 83e1f11e18..444c5ca7a3 100644
--- a/crypto/bn/bn_gf2m.c
+++ b/crypto/bn/bn_gf2m.c
@@ -730,14 +730,20 @@ int BN_GF2m_mod_inv(BIGNUM *r, const BIGNUM *a, const BIGNUM *p, BN_CTX *ctx)
 {
     BIGNUM *b = NULL;
     int ret = 0;
+    int numbits;
 
     BN_CTX_start(ctx);
     if ((b = BN_CTX_get(ctx)) == NULL)
         goto err;
 
+    /* Fail on a non-sensical input p value */
+    numbits = BN_num_bits(p);
+    if (numbits <= 1)
+        goto err;
+
     /* generate blinding value */
     do {
-        if (!BN_priv_rand_ex(b, BN_num_bits(p) - 1,
+        if (!BN_priv_rand_ex(b, numbits - 1,
                              BN_RAND_TOP_ANY, BN_RAND_BOTTOM_ANY, 0, ctx))
             goto err;
     } while (BN_is_zero(b));