Simplify SSL BIO buffering logic

The write BIO for handshake messages is bufferred so that we only write
out to the network when we have a complete flight. There was some
complexity in the buffering logic so that we switched buffering on and
off at various points through out the handshake. The only real reason to
do this was historically it complicated the state machine when you wanted
to flush because you had to traverse through the "flush" state (in order
to cope with NBIO). Where we knew up front that there was only going to
be one message in the flight we switched off buffering to avoid that.

In the new state machine there is no longer a need for a flush state so
it is simpler just to have buffering on for the whole handshake. This
also gives us the added benefit that we can simply call flush after every
flight even if it only has one message in it. This means that BIO authors
can implement their own buffering strategies and not have to be aware of
the state of the SSL object (previously they would have to switch off
their own buffering during the handshake because they could not rely on
a flush being received when they really needed to write data out). This
last point addresses GitHub Issue #322.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Fixup READLINE case

RT#4543

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>

Fix nits in pod files.

Add doc-nit-check to help find future issues.
Make podchecker be almost clean.
Remove trailing whitespace.
Tab expansion

Reviewed-by: Richard Levitte <levitte@openssl.org>

Use the process_docs script to install docs on Windows and VMS

Reviewed-by: Rich Salz <rsalz@openssl.org>

Documentation processor in perl, for platforms that don't have sh

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use correct EOL in headers.

RT#1817

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move Haiku configuration to separate config file to denote
the fact that it's community-supported target.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add assembly CRYPTO_memcmp.

GH: #102

Reviewed-by: Richard Levitte <levitte@openssl.org>

rand/randfile.c: remove _XOPEN_SOURCE definition.

Defintions of macros similar to _XOPEN_SOURCE belong in command line
or in worst case prior first #include directive in source. As for
macros is was allegedly controlling. One can argue that we are
probably better off demanding S_IS* macros but there are systems
that just don't comply, hence this compromise solution...

Reviewed-by: Rich Salz <rsalz@openssl.org>

Check that the obtained public key is valid

In the X509 app check that the obtained public key is valid before we
attempt to use it.

Issue reported by Yuan Jochen Kang.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Add copyright

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add copyright to manpages

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove needless license terms (for docs)

Reviewed-by: Richard Levitte <levitte@openssl.org>

Ensure =cut is last line in every file.

Reviewed-by: Richard Levitte <levitte@openssl.org>

ok was uninitialised on failure.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>

When strict SCT fails record verification failure

Since with SSL_VERIFY_NONE, the connection may continue and the
session may even be cached, we should save some evidence that the
chain was not sufficiently verified and would have been rejected
with SSL_VERIFY_PEER.  To that end when a CT callback returs failure
we set the verify result to X509_V_ERR_NO_VALID_SCTS.

Note: We only run the CT callback in the first place if the verify
result is still X509_V_OK prior to start of the callback.

RT #4502

Reviewed-by: Tim Hudson <tjh@openssl.org>

make update

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Improve and document low-level PEM read routines

PEM_read(), PEM_read_bio(), PEM_get_EVP_CIPHER_INFO() and
PEM_do_header().

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Fixes non __GNUC__ compilation

adds missing check for defined(__GNUC__)
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1094)

Fix SRP client key computation

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #1017

Small typo, a tab where there should have been a space

Reviewed-by: Rich Salz <rsalz@openssl.org>

Cleanup openssl.ec

STORE doesn't exist for now

HMAC doesn't have any error codes

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix util/mkerr.pl

- Adjust mkerr.pl to produce the line length we used for source
  reformating.

- Have mkerr.pl keep track of preprocessor directive indentation

  Among others, do not spuriously throw away a #endif at the end of
  header files.

- Make sure mkerr.pl specifies any header inclusion correctly

Reviewed-by: Rich Salz <rsalz@openssl.org>

Ensure verify error is set when X509_verify_cert() fails

Set ctx->error = X509_V_ERR_OUT_OF_MEM when verificaiton cannot
continue due to malloc failure.  Also, when X509_verify_cert()
returns <= 0 make sure that the verification status does not remain
X509_V_OK, as a last resort set it it to X509_V_ERR_UNSPECIFIED,
just in case some code path returns an error without setting an
appropriate value of ctx->error.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Clarify negative return from X509_verify_cert()

Reviewed-by: Richard Levitte <levitte@openssl.org>

Make OPENSSL_die as noreturn

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/526)

Document the esc_2254 command line name option

RT#1466

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

RT#1466

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make it possible to have RFC2254 escapes with ASN1_STRING_print_ex()

Also adds 'esc_2254' to the possible command line name options

RT#1466

Reviewed-by: Rich Salz <rsalz@openssl.org>

OID code tidy up.

Tidy up and simplify OBJ_dup() and OBJ_create().

Sanity check added OIDs: don't allow duplicates.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Consolidate copyright for demos

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add some error messages for malloc fails

Reviewed-by: Richard Levitte <levitte@openssl.org>

Better checks for malloc failure in various METHOD functions

A number of the METHOD functions weren't properly handling malloc failures.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix some malloc failures in b_addr.c

There were some unchecked calls to OPENSSL_strdup().

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add some check for malloc failure in the CAPI engine

In the CAPI engine there were some unchecked calls to OPENSSL_strdup().

GH Issue #830

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix ts app help message

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Locally declare some variables

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Few cleanups in s_client, s_server apps.

Discard useless static engine_id
Add a const qualifier
Fix some spelling

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix a NULL dereference in chacha20_poly1305_init_key()

chacha20_poly1305_init_key() dereferences NULL when called with inkey !=
NULL && iv == NULL. This function is called by EVP_EncryptInit_ex()
family, whose documentation allows setting key and iv in separate calls.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Manual fixes after copyright consolidation

Reviewed-by: Richard Levitte <levitte@openssl.org>

X509_STORE_CTX accessors.

Add some functions that were missing when a number of X509
objects became opaque (thanks, Roumen!)

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 05/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 09/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 08/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 07/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 06/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 04/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 03/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 02/10

Reviewed-by: Richard Levitte <levitte@openssl.org>

Copyright consolidation 01/10

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Add a comment to explain the use of |num_recs|

In the SSLV2ClientHello processing code in ssl3_get_record, the value of
|num_recs| will always be 0. This isn't obvious from the code so a comment
is added to explain it.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Fix RSA dasync engine bug

When RSA went opaque a bug was introduced into the dasync engine where
the wrong function was being set for the rsa_priv_dec operation.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Use the current record offset in ssl3_get_record

The function ssl3_get_record() can obtain multiple records in one go
as long as we are set up for pipelining and all the records are app
data records. The logic in the while loop which reads in each record is
supposed to only continue looping if the last record we read was app data
and we have an app data record waiting in the buffer to be processed. It
was actually checking that the first record had app data and we have an
app data record waiting. This actually amounts to the same thing so wasn't
wrong - but it looks a bit odd because it uses the |rr| array without an
offset.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

There is only one read buffer

Pipelining introduced the concept of multiple records being read in one
go. Therefore we work with an array of SSL3_RECORD objects. The pipelining
change erroneously made a change in ssl3_get_record() to apply the current
record offset to the SSL3_BUFFER we are using for reading. This is wrong -
there is only ever one read buffer. This reverts that change. In practice
this should make little difference because the code block in question is
only ever used when we are processing a single record.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Fix typos in apps/enc.c
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix some out of date comments

Fix various references to s3_clnt.c and s3_srvr.c which don't exist
any more.

GitHub Issue #765

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove repeated condition from if in X509_NAME_oneline

An if checks the value of |type| to see if it is V_ASN1_VISIBLESTRING
twice. We only need to do it once.

GitHub Issue #656

Reviewed-by: Richard Levitte <levitte@openssl.org>

Correct documentation error

SSL_get_async_wait_fd() was replaced by SSL_get_all_async_fds() and
SSL_get_changed_async_fds().

Reviewed-by: Richard Levitte <levitte@openssl.org>

test/evp_test.c: exercise in-place encryption.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

ARMv8 assembly pack: add OPENSSL_cleanse.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove an unneccessary check of cipher

Due to short-circuiting we only need to check "cipher" for NULL once.

PR#699

Reviewed-by: Richard Levitte <levitte@openssl.org>

Use OPENSSL_hexchar2int

Reviewed-by: Richard Levitte <levitte@openssl.org>

Use app_malloc; two missing cases.

Reviewed-by: Matt Caswell <matt@openssl.org>

const correctness: make HMAC_size() take a const *

CLA: none; trivial

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1070

session tickets: use more sizeof

Reviewed-by: Matt Caswell <matt@openssl.org>
MR: #2153

Use AES256 for the default encryption algoritm for TLS session tickets

This involves providing more session ticket key data, for both the cipher and
the digest

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #515, MR: #2153

session tickets: Use sizeof() for the various fields

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #515, MR: #2153

Don't load same config file twice.

RT#4215

Reviewed-by: Richard Levitte <levitte@openssl.org>

Documentation: Clarify sizes for UI_add_input_string()

The given sizes to not include the final NUL character.

RT#2622

Reviewed-by: Matt Caswell <matt@openssl.org>

Workaround an IO::Socket::IP bug

Workaround an apparent IO:Socket::IP bug where a seemingly valid
server socket is being returned even though a valid connection does not
exist. This causes the tests to intermittently hang. We additionally check
that the peerport looks ok to verify that the returned socket looks usable.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fold threads.h into crypto.h making API public

Document thread-safe lock creation

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't require any length of password when decrypting

RT#2534

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add support for RC / WINDRES env variables

RT#2558

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add missing initialiser in e_chil.c

RT#2616

Reviewed-by: Tim Hudson <tjh@openssl.org>

do not remove links found in .git directory

Some setups use links inside .git directory and make clean should not
remove them to avoid breaking git meta-information.

Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
CLA: none; trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1078)

Windows: Add CRYPT32.LIB to the libraries to link you app with

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1063)

Do not access SSL struct directly in TLS1_get_version and TLS1_get_client_version macros

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add SSL_client_version() getter function

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add some documentation of SSL_CTX_set_tlsext_status_type()

The previous commit added SSL_CTX_set_tlsext_status_type(). This one adds
some documentation for it.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Propagate tlsext_status_type from SSL_CTX to SSL

To allow OCSP stapling to work with libcurl.

Github PR #200

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix NPN protocol name list validation

Since 50932c4 "PACKETise ServerHello processing",
ssl_next_proto_validate() incorrectly allows empty protocol name.
draft-agl-tls-nextprotoneg-04[1] says "Implementations MUST ensure that
the empty string is not included and that no byte strings are
truncated."
This patch restores the old correct behavior.

[1] https://tools.ietf.org/html/draft-agl-tls-nextprotoneg-04

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Unify <TYPE>_up_ref methods signature and behaviour.

Add a status return value instead of void.
Add some sanity checks on reference counter value.
Update the docs.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Small MSVC build fixes.

- "/Ox /O2 /Ob2" get's reduced to "/O2", the reason being:

    /Ox = /Ob2 /Og /Oi /Ot /Oy /Gs
    /O2 = /Ob2 /Og /Oi /Ot /Oy /Gs /GF /Gy

- apps/openssl.cnf gets installed.

- always delete files quietly, as they might not be there.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1075)

Fix some missing OBJ_dup failure checks.

Fix some missing OBJ_dup failure checks.
Merged from
https://boringssl.googlesource.com/boringssl/+/0ce78a757d815c0dde9ed5884229f3a5b2cb3e9c%5E!

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1057)

openssl_{startup,shutdown}.com.in are in the source directory

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix various methods declaration in pod file

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1042)

Increment size limit for ClientHello messages

The current limit of 2^14 bytes is too low (e.g. RFC 5246 specifies the
maximum size of just the extensions field to be 2^16-1), and may cause
bogus failures.

RT#4063

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/413)

Fix engine cryptodev: pointer to IV

Currently point to wrong address

Signed-off-by: Kirill Marinushkin <k.marinushkin@gmail.com>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix OpenSSL_memdup error handling

check source's kdf_ukm, not destination's

use != NULL, instead of implicit checking

don't free internal data structure like pkey_rsa_copy()

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

VMS perl: Fix glob output

In some cases, perl's glob() thinks it needs to return file names with
generation numbers, such as when a file name pattern includes two
periods.  Constructing other file names by simple appending to file
names with generation numbers isn't a good idea, so for the VMS case,
just peal the generation numbers if they are there.
Fortunately, this is easy, as the returned generation number delimiter
will always be a semi-colon.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add some additional NewSessionTicket tests

If the server does not send a session ticket extension, it should not then
send the NewSessionTicket message.

If the server sends the session ticket extension, it MUST then send the
NewSessionTicket message.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

The NewSessionTicket message is not optional.

Per RFC 4507, section 3.3:

   This message [NewSessionTicket] MUST be sent if the
   server included a SessionTicket extension in the ServerHello.  This
   message MUST NOT be sent if the server did not include a
   SessionTicket extension in the ServerHello.

The presence of the NewSessionTicket message should be determined
entirely from the ServerHello without probing.

RT#4389

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix stack constification definitions.

RT#4471

Reviewed-by: Matt Caswell <matt@openssl.org>

Correct documentation on digest used.

RT#4302

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Fix signer option and support format SMIME.

Fix -signer option in smime utility to output signer certificates
when verifying.

Add support for format SMIME for -inform and -outform with cms and
smime utilities.

PR#4215

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Windows: When installing libraries and executables, install .pdb files as well

Reviewed-by: Rich Salz <rsalz@openssl.org>

DJGPP adjustments

* Configure: Replaced -DTERMIO by -DTERMIOS in CFLAGS.

* crypto/bio/bss_dgram.c [WATT32]: Remove obsolete redefinition of
  function names: sock_write, sock_read and sock_puts.

* crypto/bio/bss_sock.c [WATT32]: For Watt-32 2.2.11 sock_write,
  sock_read and sock_puts are redefined to their private names so
  their names must be undefined first before they can be redefined
  again.

* crypto/bio/bss_file.c (file_fopen) [__DJGPP__]: Make a copy of the
  passed file name and replace the leading dots in the dirname part
  and the basname part of the file name, unless LFN is supported.

* e_os.h [__DJGPP__]: Undefine macro DEVRANDOM_EGD. Neither MS-DOS nor
  FreeDOS provide 'egd' sockets.
  New macro HAS_LFN_SUPPORT checks if underlying file system supports
  long file names or not.
  Include sys/un.h.
  Define WATT32_NO_OLDIES.

* INSTALL.DJGPP: Update URL of WATT-32 library.

Submitted by Juan Manuel Guerrero <juan.guerrero@gmx.de>

RT#4217

Reviewed-by: Andy Polyakov <appro@openssl.org>

Move the DJGPP target to its own config.

DJGPP is a 3rd party configuration, we rely entirely on the OpenSSL to
help us fine tune and test.  Therefore, it's moved to its own config.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Fix uninitialized variable

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Remove proxy tests. Add verify callback tests.

The old proxy tests test the implementation of an application proxy
policy callback defined in the test itself, which is not particularly
useful.

It is, however, useful to test cert verify overrides in
general. Therefore, replace these tests with tests for cert verify
callback behaviour.

Also glob the ssl test inputs on the .in files to catch missing
generated files.

Reviewed-by: Rich Salz <rsalz@openssl.org>