If |q| >= |p| then the q value is obviously wrong as q
is supposed to be a prime divisor of p-1.
We check if p is overly large so this added test implies that
q is not large either when performing subsequent tests using that
q value.
Otherwise if it is too large these additional checks of the q value
such as the primality test can then trigger DoS by doing overly long
computations.
Fixes CVE-2023-3817
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21551)
int DH_check(const DH *dh, int *ret)
{
- int ok = 0, r;
+ int ok = 0, r, q_good = 0;
BN_CTX *ctx = NULL;
BIGNUM *t1 = NULL, *t2 = NULL;
if (t2 == NULL)
goto err;
- if (dh->q) {
+ if (dh->q != NULL) {
+ if (BN_ucmp(dh->p, dh->q) > 0)
+ q_good = 1;
+ else
+ *ret |= DH_CHECK_INVALID_Q_VALUE;
+ }
+
+ if (q_good) {
if (BN_cmp(dh->g, BN_value_one()) <= 0)
*ret |= DH_NOT_SUITABLE_GENERATOR;
else if (BN_cmp(dh->g, dh->p) >= 0)