NOTES, THOUGHTS, and EVERYTHING ------------------------------- (1) Concurrency and locking ... I made a change to the ENGINE_free code because I spotted a potential hold-up in proceedings (doing too much inside a lock including calling a callback), there may be other bits like this. What do the speed/optimisation freaks think of this aspect of the code and design? There's lots of locking for manipulation functions and I need that to keep things nice and solid, but this manipulation is mostly (de)initialisation, I would think that most run-time locking is purely in the ENGINE_init and ENGINE_finish calls that might be made when getting handles for RSA (and friends') structures. These would be mostly reference count operations as the functional references should always be 1 or greater at run-time to prevent init/deinit thrashing. (2) nCipher support, via the HWCryptoHook API, is now in the code. Apparently this hasn't been tested too much yet, but it looks good. :-) Atalla support has been added too, but shares a lot in common with Ben's original hooks in bn_exp.c (although it has been ENGINE-ified, and error handling wrapped around it) and it's also had some low-volume testing, so it should be usable. (3) Of more concern, we need to work out (a) how to put together usable RAND_METHODs for units that just have one "get n or less random bytes" function, (b) we also need to determine how to hook the code in crypto/rand/ to use the ENGINE defaults in a way similar to what has been done in crypto/rsa/, crypto/dsa/, etc. (4) ENGINE should really grow to encompass more than 3 public key algorithms and randomness gathering. The structure/data level of the engine code is hidden from code outside the crypto/engine/ directory so change shouldn't be too viral. More important though is how things should evolve ... this needs thought and discussion. -----------------------------------==*==----------------------------------- More notes 2000-08-01 --------------------- Geoff Thorpe, who designed the engine part, wrote a pretty good description of the thoughts he had when he built it, good enough to include verbatim here (with his permission) -- Richard Levitte Date: Tue, 1 Aug 2000 16:54:08 +0100 (BST) From: Geoff Thorpe Subject: Re: The thoughts to merge BRANCH_engine into the main trunk are emerging Hi there, I'm going to try and do some justice to this, but I'm a little short on time and the there is an endless amount that could be discussed on this subject. sigh ... please bear with me :-) > The changes in BRANCH_engine dig deep into the core of OpenSSL, for example > into the RSA and RAND routines, adding a level of indirection which is needed > to keep the abstraction, as far as I understand. It would be a good thing if > those who do play with those things took a look at the changes that have been > done in the branch and say out loud how much (or hopefully little) we've made > fools of ourselves. The point here is that the code that has emerged in the BRANCH_engine branch was based on some initial requirements of mine that I went in and addressed, and Richard has picked up the ball and run with it too. It would be really useful to get some review of the approach we've taken, but first I think I need to describe as best I can the reasons behind what has been done so far, in particular what issues we have tried to address when doing this, and what issues we have intentionally (or necessarily) tried to avoid. methods, engines, and evps -------------------------- There has been some dicussion, particularly with Steve, about where this ENGINE stuff might fit into the conceptual picture as/when we start to abstract algorithms a little bit to make the library more extensible. In particular, it would desirable to have algorithms (symmetric, hash, pkc, etc) abstracted in some way that allows them to be just objects sitting in a list (or database) ... it'll just happen that the "DSA" object doesn't support encryption whereas the "RSA" object does. This requires a lot of consideration to begin to know how to tackle it; in particular how encapsulated should these things be? If the objects also understand their own ASN1 encodings and what-not, then it would for example be possible to add support for elliptic-curve DSA in as a new algorithm and automatically have ECC-DSA certificates supported in SSL applications. Possible, but not easy. :-) Whatever, it seems that the way to go (if I've grok'd Steve's comments on this in the past) is to amalgamate these things in EVP as is already done (I think) for ciphers or hashes (Steve, please correct/elaborate). I certainly think something should be done in this direction because right now we have different source directories, types, functions, and methods for each algorithm - even when conceptually they are very much different feathers of the same bird. (This is certainly all true for the public-key stuff, and may be partially true for the other parts.) ENGINE was *not* conceived as a way of solving this, far from it. Nor was it conceived as a way of replacing the various "***_METHOD"s. It was conceived as an abstraction of a sort of "virtual crypto device". If we lived in a world where "EVP_ALGO"s (or something like them) encapsulated particular algorithms like RSA,DSA,MD5,RC4,etc, and "***_METHOD"s encapsulated interfaces to algorithms (eg. some algo's might support a PKC_METHOD, a HASH_METHOD, or a CIPHER_METHOD, who knows?), then I would think that ENGINE would encapsulate an implementation of arbitrarily many of those algorithms - perhaps as alternatives to existing algorithms and/or perhaps as new previously unimplemented algorithms. An ENGINE could be used to contain an alternative software implementation, a wrapper for a hardware acceleration and/or key-management unit, a comms-wrapper for distributing cryptographic operations to remote machines, or any other "devices" your imagination can dream up. However, what has been done in the ENGINE branch so far is nothing more than starting to get our toes wet. I had a couple of self-imposed requirements when putting the initial abstraction together, and I may have already posed these in one form or another on the list, but briefly; (i) only bother with public key algorithms for now, and maybe RAND too (motivated by the need to get hardware support going and the fact this was a comparitively easy subset to address to begin with). (ii) don't change (if at all possible) the existing crypto code, ie. the implementations, the way the ***_METHODs work, etc. (iii) ensure that if no function from the ENGINE code is ever called then things work the way they always did, and there is no memory allocation (otherwise the failure to cleanup would be a problem - this is part of the reason no STACKs were used, the other part of the reason being I found them inappropriate). (iv) ensure that all the built-in crypto was encapsulated by one of these "ENGINE"s and that this engine was automatically selected as the default. (v) provide the minimum hooking possible in the existing crypto code so that global functions (eg. RSA_public_encrypt) do not need any extra parameter, yet will use whatever the current default ENGINE for that RSA key is, and that the default can be set "per-key" and globally (new keys will assume the global default, and keys without their own default will be operated on using the global default). NB: Try and make (v) conflict as little as possible with (ii). :-) (vi) wrap the ENGINE code up in duct tape so you can't even see the corners. Ie. expose no structures at all, just black-box pointers. (v) maintain internally a list of ENGINEs on which a calling application can iterate, interrogate, etc. Allow a calling application to hook in new ENGINEs, remove ENGINEs from the list, and enforce uniqueness within the global list of each ENGINE's "unique id". (vi) keep reference counts for everything - eg. this includes storing a reference inside each RSA structure to the ENGINE that it uses. This is freed when the RSA structure is destroyed, or has its ENGINE explicitly changed. The net effect needs to be that at any time, it is deterministic to know whether an ENGINE is in use or can be safely removed (or unloaded in the case of the other type of reference) without invalidating function pointers that may or may not be used indavertently in the future. This was actually one of the biggest problems to overcome in the existing OpenSSL code - implementations had always been assumed to be ever-present, so there was no trivial way to get round this. (vii) distinguish between structural references and functional references. A *little* detail ----------------- While my mind is on it; I'll illustrate the bit in item (vii). This idea turned out to be very handy - the ENGINEs themselves need to be operated on and manipulated simply as objects without necessarily trying to "enable" them for use. Eg. most host machines will not have the necessary hardware or software to support all the engines one might compile into OpenSSL, yet it needs to be possible to iterate across the ENGINEs, querying their names, properties, etc - all happening in a thread-safe manner that uses reference counts (if you imagine two threads iterating through a list and one thread removing the ENGINE the other is currently looking at - you can see the gotcha waiting to happen). For all of this, *structural references* are used and operate much like the other reference counts in OpenSSL. The other kind of reference count is for *functional* references - these indicate a reference on which the caller can actually assume the particular ENGINE to be initialised and usable to perform the operations it implements. Any increment or decrement of the functional reference count automatically invokes a corresponding change in the structural reference count, as it is fairly obvious that a functional reference is a restricted case of a structural reference. So struct_ref >= funct_ref at all times. NB: functional references are usually obtained by a call to ENGINE_init(), but can also be created implicitly by calls that require a new functional reference to be created, eg. ENGINE_set_default(). Either way the only time the underlying ENGINE's "init" function is really called is when the (functional) reference count increases to 1, similarly the underlying "finish" handler is only called as the count goes down to 0. The effect of this, for example, is that if you set the default ENGINE for RSA operations to be "cswift", then its functional reference count will already be at least 1 so the CryptoSwift shared-library and the card will stay loaded and initialised until such time as all RSA keys using the cswift ENGINE are changed or destroyed and the default ENGINE for RSA operations has been changed. This prevents repeated thrashing of init and finish handling if the count keeps getting down as far as zero. Otherwise, the way the ENGINE code has been put together I think pretty much reflects the above points. The reason for the ENGINE structure having individual RSA_METHOD, DSA_METHOD, etc pointers is simply that it was the easiest way to go about things for now, to hook it all into the raw RSA,DSA,etc code, and I was trying to the keep the structure invisible anyway so that the way this is internally managed could be easily changed later on when we start to work out what's to be done about these other abstractions. Down the line, if some EVP-based technique emerges for adequately encapsulating algorithms and all their various bits and pieces, then I can imagine that "ENGINE" would turn into a reference-counting database of these EVP things, of which the default "openssl" ENGINE would be the library's own object database of pre-built software implemented algorithms (and such). It would also be cool to see the idea of "METHOD"s detached from the algorithms themselves ... so RSA, DSA, ElGamal, etc can all expose essentially the same METHOD (aka interface), which would include any querying/flagging stuff to identify what the algorithm can/can't do, its name, and other stuff like max/min block sizes, key sizes, etc. This would result in ENGINE similarly detaching its internal database of algorithm implementations from the function definitions that return interfaces to them. I think ... As for DSOs etc. Well the DSO code is pretty handy (but could be made much more so) for loading vendor's driver-libraries and talking to them in some generic way, but right now there's still big problems associated with actually putting OpenSSL code (ie. new ENGINEs, or anything else for that matter) in dynamically loadable libraries. These problems won't go away in a hurry so I don't think we should expect to have any kind of shared-library extensions any time soon - but solving the problems is a good thing to aim for, and would as a side-effect probably help make OpenSSL more usable as a shared-library itself (looking at the things needed to do this will show you why). One of the problems is that if you look at any of the ENGINE implementations, eg. hw_cswift.c or hw_ncipher.c, you'll see how it needs a variety of functionality and definitions from various areas of OpenSSL, including crypto/bn/, crypto/err/, crypto/ itself (locking for example), crypto/dso/, crypto/engine/, crypto/rsa, etc etc etc. So if similar code were to be suctioned off into shared libraries, the shared libraries would either have to duplicate all the definitions and code and avoid loader conflicts, or OpenSSL would have to somehow expose all that functionality to the shared-library. If this isn't a big enough problem, the issue of binary compatibility will be - anyone writing Apache modules can tell you that (Ralf? Ben? :-). However, I don't think OpenSSL would need to be quite so forgiving as Apache should be, so OpenSSL could simply tell its version to the DSO and leave the DSO with the problem of deciding whether to proceed or bail out for fear of binary incompatibilities. Certainly one thing that would go a long way to addressing this is to embark on a bit of an opaqueness mission. I've set the ENGINE code up with this in mind - it's so draconian that even to declare your own ENGINE, you have to get the engine code to create the underlying ENGINE structure, and then feed in the new ENGINE's function/method pointers through various "set" functions. The more of the code that takes on such a black-box approach, the more of the code that will be (a) easy to expose to shared libraries that need it, and (b) easy to expose to applications wanting to use OpenSSL itself as a shared-library. From my own explorations in OpenSSL, the biggest leviathan I've seen that is a problem in this respect is the BIGNUM code. Trying to "expose" the bignum code through any kind of organised "METHODs", let alone do all the necessary bignum operations solely through functions rather than direct access to the structures and macros, will be a massive pain in the "r"s. Anyway, I'm done for now - hope it was readable. Thoughts? Cheers, Geoff -----------------------------------==*==-----------------------------------