Fix OSSL_PARAM_set_BN() to fill the given buffer correctly.
authorRichard Levitte <levitte@openssl.org>
Fri, 1 Nov 2019 19:44:14 +0000 (20:44 +0100)
committerPatrick Steuer <patrick.steuer@de.ibm.com>
Sun, 3 Nov 2019 10:19:04 +0000 (11:19 +0100)
OSSL_PARAM_set_BN() filled the buffer from the left with as many bytes
as that the BIGNUM takes, regardless of buffer size or native
endianness.  This was due to BN_bn2nativepad() being given the size of
the BIGNUM rather than the size of the buffer (which meant it never
had to pad anything).

The fix is to given BN_bn2nativepad() the size of the buffer instead.
This aligns well with the corresponding _set_ functions for native
integer types work.

Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/10326)

crypto/params.c

index b2ceb132781fb9902414005500e5265a28fc3b0f..0cd13e3b81d5d15511b2b6de4f7c774112f93ef8 100644 (file)
@@ -640,8 +640,11 @@ int OSSL_PARAM_set_BN(OSSL_PARAM *p, const BIGNUM *val)
     p->return_size = bytes;
     if (p->data == NULL)
         return 1;
-    return p->data_size >= bytes
-        && BN_bn2nativepad(val, p->data, bytes) >= 0;
+    if (p->data_size >= bytes) {
+        p->return_size = p->data_size;
+        return BN_bn2nativepad(val, p->data, p->data_size) >= 0;
+    }
+    return 0;
 }
 
 OSSL_PARAM OSSL_PARAM_construct_BN(const char *key, unsigned char *buf,