Avoid an infinite loop in BN_GF2m_mod_inv

If p is set to 1 when calling BN_GF2m_mod_inv then an infinite loop will
result. Calling this function set 1 when applications call this directly
is a non-sensical value - so this would be considered a bug in the caller.

It does not seem possible to cause OpenSSL internal callers of
BN_GF2m_mod_inv to call it with a value of 1.

So, for the above reasons, this is not considered a security issue.
Reported by Bing Shi.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Todd Short <todd.short@me.com>
(Merged from https://github.com/openssl/openssl/pull/22960)

(cherry picked from commit 9c1b8f17ce2471ca37ee3936d07aed29aab10975)

diff --git a/crypto/bn/bn_gf2m.c b/crypto/bn/bn_gf2m.c
index 304c2ea08d0e950b9ad8d26f5fc8e5b580ffc156..c811ae82d6b15a82fccd2d13eaf581f4d19259b4 100644 (file)
--- a/crypto/bn/bn_gf2m.c
+++ b/crypto/bn/bn_gf2m.c
@@ -734,14 +734,20 @@ int BN_GF2m_mod_inv(BIGNUM *r, const BIGNUM *a, const BIGNUM *p, BN_CTX *ctx)
 {
     BIGNUM *b = NULL;
     int ret = 0;
+    int numbits;
 
     BN_CTX_start(ctx);
     if ((b = BN_CTX_get(ctx)) == NULL)
         goto err;
 
+    /* Fail on a non-sensical input p value */
+    numbits = BN_num_bits(p);
+    if (numbits <= 1)
+        goto err;
+
     /* generate blinding value */
     do {
-        if (!BN_priv_rand_ex(b, BN_num_bits(p) - 1,
+        if (!BN_priv_rand_ex(b, numbits - 1,
                              BN_RAND_TOP_ANY, BN_RAND_BOTTOM_ANY, 0, ctx))
             goto err;
     } while (BN_is_zero(b));