Update control logic for BN_gcd
authorCesar Pereida Garcia <cesar.pereidagarcia@tut.fi>
Mon, 21 Oct 2019 11:41:01 +0000 (14:41 +0300)
committerNicola Tuveri <nic.tuv@gmail.com>
Wed, 23 Oct 2019 09:06:02 +0000 (12:06 +0300)
commit8aca4bfe8213402c80abc06fe25121461f79128d
treea74f7d45df77091e1fc5ed1050bca928b2933975
parent777182a0c77ee374e43b94546f49b25f37945c0e
Update control logic for BN_gcd

PR https://github.com/openssl/openssl/pull/10122 introduced changes to
the BN_gcd function and the control logic inside it accessed `g->d[0]`
irrespective of `g->top`.

When BN_add is called, in case the result is zero, `BN_zero` is called.
The latter behaves differently depending on the API compatibility level
flag: normally `g->d[0]` is cleared but in `no-deprecated` builds only
`g->top` is set to zero.

This commit uses bitwise logic to ensure that `g` is treated as zero if
`g->top` is zero, irrespective of `g->d[0]`.

Co-authored-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10232)
crypto/bn/bn_gcd.c