Allow an ALPN callback to pretend to not exist
authorBenjamin Kaduk <bkaduk@akamai.com>
Tue, 7 Feb 2017 22:23:16 +0000 (16:23 -0600)
committerMatt Caswell <matt@openssl.org>
Tue, 11 Apr 2017 12:46:13 +0000 (13:46 +0100)
commita504540fcbc779e7e2b855dccaae9e4d7156c88d
tree81560a8d50096a258b0d602892f9732546ca6fb6
parent6d1ab3f5c87e44f9092f3eafe88dc5014e1af2f1
Allow an ALPN callback to pretend to not exist

RFC 7301 mandates that the server SHALL respond with a fatal
"no_application_protocol" alert when there is no overlap between
the client's supplied list and the server's list of supported protocols.
In commit 062178678f5374b09f00d70796f6e692e8775aca we changed from
ignoring non-success returns from the supplied alpn_select_cb() to
treating such non-success returns as indicative of non-overlap and
sending the fatal alert.

In effect, this is using the presence of an alpn_select_cb() as a proxy
to attempt to determine whether the application has configured a list
of supported protocols.  However, there may be cases in which an
application's architecture leads it to supply an alpn_select_cb() but
have that callback be configured to take no action on connections that
do not have ALPN configured; returning SSL_TLSEXT_ERR_NOACK from
the callback would be the natural way to do so.  Unfortunately, the
aforementioned behavior change also treated SSL_TLSEXT_ERR_NOACK as
indicative of no overlap and terminated the connection; this change
supplies special handling for SSL_TLSEXT_ERR_NOACK returns from the
callback.  In effect, it provides a way for a callback to obtain the
behavior that would have occurred if no callback was registered at
all, which was not possible prior to this change.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3158)
(cherry picked from commit 8313a787d770ac1d7ddafcbc41b13e7fb5841eae)
doc/ssl/SSL_CTX_set_alpn_select_cb.pod
ssl/t1_lib.c
test/handshake_helper.c