Fix write failure handling in DTLS1.2

The DTLS code is supposed to drop packets if we try to write them out but
the underlying BIO write buffers are full. ssl3_write_pending() contains
an incorrect test for DTLS that controls this. The test only checks for
DTLS1 so DTLS1.2 does not correctly clear the internal OpenSSL buffer which
can later cause an assert to be hit. This commit changes the test to cover
all DTLS versions.

RT#3967

Reviewed-by: Tim Hudson <tjh@openssl.org>

diff --git a/ssl/record/rec_layer_s3.c b/ssl/record/rec_layer_s3.c
index 79d3c21fbe8a88f4d41512876b4260251e6bcd9a..d6e922c652ac6a2da7e250e17265f7f2b68a9e34 100644 (file)
--- a/ssl/record/rec_layer_s3.c
+++ b/ssl/record/rec_layer_s3.c
@@ -929,7 +929,7 @@ int ssl3_write_pending(SSL *s, int type, const unsigned char *buf,
             s->rwstate = SSL_NOTHING;
             return (s->rlayer.wpend_ret);
         } else if (i <= 0) {
             s->rwstate = SSL_NOTHING;
             return (s->rlayer.wpend_ret);
         } else if (i <= 0) {

-            if (s->version == DTLS1_VERSION || s->version == DTLS1_BAD_VER) {
+            if (SSL_IS_DTLS(s)) {

                 /*
                  * For DTLS, just drop it. That's kind of the whole point in
                  * using a datagram service
                 /*
                  * For DTLS, just drop it. That's kind of the whole point in
                  * using a datagram service