If an engine comes up explicitely, it must also come down explicitely
authorRichard Levitte <levitte@openssl.org>
Wed, 28 Sep 2016 22:40:20 +0000 (00:40 +0200)
committerRichard Levitte <levitte@openssl.org>
Thu, 20 Oct 2016 07:04:00 +0000 (09:04 +0200)
commitaa01b82c69eeb0cfd255174111fc34a7ed5f8429
treea4d6ec8b46ed816e54bc17f1d3c28bb8b8b6f7b4
parent10e60f26cef02a6310d20cc2c918184fc9100d14
If an engine comes up explicitely, it must also come down explicitely

In apps/apps.c, one can set up an engine with setup_engine().
However, we freed the structural reference immediately, which means
that for engines that don't already have a structural reference
somewhere else (because it has registered at least one cipher or digest
algorithm method, and therefore gets a functional reference through the
ENGINE_set_default() call), we end up returning an invalid reference.

Instead, the function release_engine() is added, and called at the end
of the routines that call setup_engine().

Originally, the ENGINE API wasn't designed for this to happen, an
engine had to register at least one algorithm method, and was
especially expected to register the algorithms corresponding to the
key types that could be stored and hidden in hardware.  However, it
turns out that some engines will not register those algorithms with
the ENGINE_set_{algo}, ENGINE_set_cipher or ENGINE_set_digest
functions, as they only want the methods to be used for keys, not as
general crypto accelerator methods.  That may cause ENGINE_set_default()
to do nothing, and no functional reference is therefore made, leading
to a premature deallocation of the engine and it thereby becoming
unavailable when trying to fetch a key.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1644)
32 files changed:
apps/apps.c
apps/apps.h
apps/ca.c
apps/cms.c
apps/dgst.c
apps/dhparam.c
apps/dsa.c
apps/dsaparam.c
apps/ec.c
apps/ecparam.c
apps/enc.c
apps/gendsa.c
apps/genpkey.c
apps/genrsa.c
apps/pkcs12.c
apps/pkcs7.c
apps/pkcs8.c
apps/pkey.c
apps/pkeyparam.c
apps/pkeyutl.c
apps/rand.c
apps/req.c
apps/rsa.c
apps/rsautl.c
apps/s_client.c
apps/s_server.c
apps/smime.c
apps/speed.c
apps/spkac.c
apps/srp.c
apps/verify.c
apps/x509.c