Index: vendor/serf/dist/CHANGES =================================================================== --- vendor/serf/dist/CHANGES (revision 286498) +++ vendor/serf/dist/CHANGES (revision 286499) @@ -1,283 +1,291 @@ +Serf 1.3.8 [2014-10-20, from /tags/1.3.8, rxxxx] +Fix issue #152: CRC calculation error for gzipped http reponses > 4GB. +Fix issue #153: SSPI CredHandle not freed when APR pool is destroyed. +Fix issue #154: Disable SSLv2 and SSLv3 as both or broken. + + Serf 1.3.7 [2014-08-11, from /tags/1.3.7, r2411] Handle NUL bytes in fields of an X.509 certificate. (r2393, r2399) + Serf 1.3.6 [2014-06-09, from /tags/1.3.6, r2372] Revert r2319 from serf 1.3.5: this change was making serf call handle_response multiple times in case of an error response, leading to unexpected behavior. + Serf 1.3.5 [2014-04-27, from /tags/1.3.5, r2355] Fix issue #125: no reverse lookup during Negotiate authentication for proxies. Fix a crash caused by incorrect reuse of the ssltunnel CONNECT request (r2316) Cancel request if response parsing failed + authn callback set (r2319) Update the expired certificates in the test suite. Serf 1.3.4 [2014-02-08, from /tags/1.3.4, r2310] Fix issue #119: Endless loop during ssl tunnel setup with Negotiate authn Fix issue #123: Can't setup ssl tunnel which sends Connection close header Fix a race condition when initializing OpenSSL from multiple threads (r2263) Fix issue #138: Incorrect pkg-config file when GSSAPI isn't configured Serf 1.3.3 [2013-12-09, from /tags/1.3.3, r2242] Fix issue 129: Try more addresses of multihomed servers Handle X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE correctly (r2225) Return APR_TIMEUP from poll() to enable detecting connection timeouts (r2183) Serf 1.3.2 [2013-10-04, from /tags/1.3.2, r2195] Fix issue 130: HTTP headers should be treated case-insensitively Fix issue 126: Compilation breaks with Codewarrior compiler Fix crash during cleanup of SSL buckets in apr_terminate() (r2145) Fix Windows build: Also export functions with capital letters in .def file Fix host header when url contains a username or password (r2170) Ensure less TCP package fragmentation on Windows (r2145) Handle authentication for responses to HEAD requests (r2178,-9) Improve serf_get: add option to add request headers, allow url with query, allow HEAD requests (r2143,r2175,-6) Improve RFC conformance: don't expect body for certain responses (r2011,-2) Do not invoke progress callback when no data was received (r2144) And more test suite fixes and build warning cleanups SCons-related fixes: Fix build when GSSAPI not in default include path (2155) Fix OpenBSD build: always map all LIBPATH entries into RPATH (r2156) Checksum generation in Windows shared libraries for release builds (2162) Mac OS X: Use MAJOR version only in dylib install name (r2161) Use both MAJOR and MINOR version for the shared library name (2163) Fix the .pc file when installing serf in a non-default LIBDIR (r2191) Serf 1.3.1 [2013-08-15, from /tags/1.3.1, r2138] Fix issue 77: Endless loop if server doesn't accept Negotiate authentication. Fix issue 114: ssl/tls renegotiation fails Fix issue 120: error with ssl tunnel over proxy with KeepAlive off and Basic authentication. Fixed bugs with authentication (r2057,2115,2118) SCons-related fixes: Fix issue 111: add flag to set custom library path Fix issue 112: add soname Fix issue 113: add gssapi libs in the serf pc file Fix issue 115: Setting RPATH on Solaris broken in SConstruct Fix issue 116: scons check should return non-zero exit staths Fix issue 121: make CFLAGS, LIBS, LINKFLAGS and CPPFLAGS take a space- separated list of flags. Fix issue 122: make scons PREFIX create the folder if it doesn't exist Mac OS X: Fix scons --install-sandbox Solaris: Fix build with cc, don't use unsupported compiler flags Require SCons version 2.3.0 or higher now (for the soname support). Serf 1.3.0 [2013-07-23, from /tags/1.3.0, r2074] Fix issue 83: use PATH rather than URI within an ssltunnel (r1952) Fix issue 108: improved error reporting from the underlying socket (r1951) NEW: Switch to the SCons build system; retire serfmake, serf.mak, autotools Improved Basic and Digest authentication: - remember credentials on a per-server basis - properly manage authentication realms - continue functioning when a server sets KeepAlive: off Windows: add support for NTLM authentication Improved 2617 compliance: always use strongest authentication (r1968,1971) Fixed bugs with proxy authentication and SSL tunneling through a proxy Fixed bugs the response parser (r2032,r2036) SSL connection performance improvements Huge expansion of the test suite Serf 1.2.1 [2013-06-03, from /tags/1.2.1, r1906] Fix issue 95: add gssapi switches to configure (r1864, r1900) Fix issue 97: skip mmap bucket if APR_HAS_MMAP is undefined (r1877) Fix issue 100: building against an old Windows Platform SDK (r1881) Fix issue 102: digest authentication failures (r1885) Improve error return values in SSPI authentication (r1804) Ensure serf-1.pc is constructed by serfmake (r1865) Optimize SPNego authentication processing (r1868) Reject certs that application does not like (r1794) Fix possible endless loop in serf_linebuf_fetch() (r1816) Windows build: dereference INTDIR in serf.mak (r1882) Serf 1.2.0 [2013-02-22, from /tags/1.2.0, r1726] Fixed issue 94: Serf can enter an infinite loop when server aborts conn. Fixed issue 91: Serf doesn't handle an incoming 408 Timeout Request Fixed issue 80: Serf is not handling Negotiate authentication correctly Fixed issue 77: Endless loop if server doesn't accept Negotiate authn Fixed issue 93: cleanup-after-fork interferes with parent (r1714) Fixed most of issue 89: Support REAL SPNEGO authentication Enable Negotiate/Kerberos support for proxy servers. Return error when C-L, chunked, gzip encoded response bodies were truncated (due to aborted connection) (r1688) Add a logging mechanism that can be enabled at compile-time. Don't lookup server address if a proxy was configured. (r1706) Fix an off-by-one in buffer sizing (r1695) Disable SSL compression by default + API to enable it (r1692) New serf_connection_get_latency() for estimated network latency (r1689) New error code and RFC compliance for the HTTPS tunnel (r1701, r1644) Handle EINTR when a user suspends and then backgrounds the app (r1708) Minor fixes and test suite improvements. Serf 1.1.1 [2012-10-04, from /tags/1.1.1, r1657] Fixed issue 86: ensure requeued requests are correctly handled. This fixes: - infinite loop with multiple connection resets or SIGPIPE errors - "connection" hang where we would not re-queue requests that are held after we re-connect Fixed issue 74: test_all goes in an endless loop Fix memleak when conn. is closed explicitly/due to pool cleanups (r1623) Windows: Fix https connection aborts (r1628..-30,-33,-34,-37) Add new error codes for the SSL bucket Serf 1.1.0 [2012-06-07, from /tags/1.1.0, r1617] New: serf_bucket_request_set_CL() for C-L based, non-chunked requests New: serf_ssl_server_cert_chain_callback_set() for full-chain validation Serf 1.0.3 [2012-03-20, from /tags/1.0.3, r1586] Map more OpenSSL errors into SERF_SSL_CERT_UNKNOWNCA (r1573) Serf 1.0.2 Not released. Serf 1.0.1 [2012-02-15, from /tags/1.0.1, r1569] FreeBSD fixes in the test suite (r1560, r1565) Minor build fixes Serf 1.0.0 [2011-07-15, from /tags/1.0.0, r1540] Fixed issue 38: enable builds using non-GNU make Fixed issue 49: support SSL tunnels for HTTPS via a proxy Fixed issue 56: allow Subject Alternative Name, and enable SNI Fixed issue 61: include order dependencies Fixed issue 66: improved error reporting when creating install dirs Fixed issue 71: handle ECONNREFUSED on Windows Fixed issue 79: destroy the APR allocator, if we create one Fixed issue 81: build failed on APR 0.9.x Major performance improvements and bug fixes for SSL buckets/handling (r1462) Add a new "iovec" bucket type (r1434) Minimize network packet writes based on ra_serf analysis (r1467, r1471) Fix out of order issue with multiple priority requests (r1469) Work around broken WSAPoll() impl on Windows introduced in APR 1.4.0 (r1506) Fix 100% CPU usage with many pipelined requests (r1456) Corrected contents of build/serf.def; it now includes bucket types (r1512) Removed "snapshot" feature from buckets (r1503) Various improvements to the test system Various memory leak fixes Serf 0.7.2 [2011-03-12, from /tags/0.7.2, r1452] Actually disable Nagle when creating a connection (r1441) Return error when app asks for HTTPS over proxy connection (r1433) Serf 0.7.1 [2011-01-25, from /tags/0.7.1, r1432] Fix memory leak when using SSL (r1408, r1416) Fix build for blank apr-util directory (r1421) Serf 0.7.0 [2010-08-25, from /tags/0.7.0, r1407] Fix double free abort when destroying request buckets Fix test server in unit test framework to avoid random test failures Allow older Serf programs which don't use the new authn framework to still handle authn without forcing them to switch to the new framework. (r1401) Remove the SERF_DECLARE macros, preferring a .DEF file for Windows Barrier buckets now pass read_iovec to their wrapped bucket Fix HTTP header parsing to allow for empty header values Serf 0.6.1 [2010-05-14, from /tags/0.6.1, r1370] Generally: this release fixes problems with the 0.4.0 packaging Small compilation fix in outgoing.c for Windows builds Serf 0.6.0 Not released. Serf 0.5.0 Not released. Serf 0.4.0 WITHDRAWN: this release misstated itself as 0.5.0; use a later release Provide authn framework, supporting Basic, Digest, Kerberos (SSPI, GSS), along with proxy authn using Basic or Digest Added experimental listener framework, along with test_server.c Improvements and fixes to SSL support, including connection setup changes Experimental support for unrequested, arriving ("async") responses Experimental BWTP support using the async arrival feature Headers are combined on read (not write), to ease certian classes of parsing Experimental feature on aggregate buckets for a callback-on-empty Fix the bucket allocator for when APR is using its pool debugging features Proxy support in the serf_get testing utility Fix to include the port number in the Host header serf_get propagates errors from the response, instead of aborting (Issue 52) Added serf_lib_version() for runtime version tests Serf 0.3.1 [2010-02-14, from /tags/0.3.1, r1322] Fix loss of error on request->setup() callback. (Issue 47) Support APR 2.x. (Issue 48) Fixed slowdown in aggregate bucket with millions of child buckets Avoid hang in apr_pollset_poll() by unclosed connections after fork() Serf 0.3.0 [2009-01-26, from /tags/0.3.0, r1217] Support LTFLAGS override as a config-time env. variable (Issue 44) Fix CUTest test harness compilation on Solaris (Issue 43) Fix small race condition in OpenSSL initialization (Issue 39) Handle content streams larger than 4GB on 32-bit OSes (Issue 41) Fix test_ssl.c compilation with mingw+msys Fix conn close segfault by explicitly closing conn when pool is destroyed Expose the depth of the SSL certificate so the validator can use that info Fix socket address family issue when opening a connection to a proxy Provide new API to take snapshots of buckets Implement snapshot API for simple and aggregate buckets Build with bundled apr and apr-util VPATH builds Build with bundled OpenSSL builds Serf 0.2.0 [2008-06-06, from /tags/0.2.0, r1189] Enable use of external event loop: serf_create_context_ex Enable adding new requests at the beginning of the request queue Handle 'Connection:close' headers Enable limiting the number of outstanding requests Add readline function to simple buckets Concatenate repeated headers using comma as separator, as per RFC 2616, section 4.2. (Issue 29) Add proxy server support Add progress feedback support. (Issue 11) Provide new API to simplify use of proxy and progress feedback support Add callback to validate SSL server certificates. (Issue 31) Add new test framework Send current version string in the test programs (Issue 21) Bugfixes: Fix segfault with epoll when removing a NULL socket Reset OpenSSL thread-safety callbacks when apr_terminate() called Do not remove the socket from the pollset on pool cleanup Do not issue double close on skt w/second one being close(-1) (Issue 33) Serf 0.1.2 [2007-06-18, from /tags/0.1.2, r1115] Enable thread-safety with OpenSSL (Issue 19) Teach serfmake to install headers into include/serf-0 Be more tolerant when servers close the connection without telling us Do not open the connection until we have requests to deliver Fix serfmake to produce the library that corresponds to the minor version Fix a memory leak with the socket bucket (Issue 14) Fix uninitialized branch in serf_spider (Issue 15) Serf 0.1.1 [2007-05-12, from /tags/0.1.1, r1105] Add SSL client certificate support Implement optimized iovec reads for header buckets Fix up 'make clean' and 'make distclean' (Issues 9, 10) Add SERF_VERSION_AT_LEAST macro Remove abort() calls (Issue 13) Serf 0.1.0 [2006-12-14, from /tags/0.1.0, r1087] Initial packaged release Index: vendor/serf/dist/auth/auth_spnego_sspi.c =================================================================== --- vendor/serf/dist/auth/auth_spnego_sspi.c (revision 286498) +++ vendor/serf/dist/auth/auth_spnego_sspi.c (revision 286499) @@ -1,298 +1,298 @@ /* Copyright 2010 Justin Erenkrantz and Greg Stein * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ #include "auth_spnego.h" #include "serf.h" #include "serf_private.h" #ifdef SERF_USE_SSPI #include #include #define SECURITY_WIN32 #include /* SEC_E_MUTUAL_AUTH_FAILED is not defined in Windows Platform SDK 5.0. */ #ifndef SEC_E_MUTUAL_AUTH_FAILED #define SEC_E_MUTUAL_AUTH_FAILED _HRESULT_TYPEDEF_(0x80090363L) #endif struct serf__spnego_context_t { CredHandle sspi_credentials; CtxtHandle sspi_context; BOOL initalized; apr_pool_t *pool; /* Service Principal Name (SPN) used for authentication. */ const char *target_name; /* One of SERF_AUTHN_* authentication types.*/ int authn_type; }; /* Map SECURITY_STATUS from SSPI to APR error code. Some error codes mapped * to our own codes and some to Win32 error codes: * http://support.microsoft.com/kb/113996 */ static apr_status_t map_sspi_status(SECURITY_STATUS sspi_status) { switch(sspi_status) { case SEC_E_INSUFFICIENT_MEMORY: return APR_FROM_OS_ERROR(ERROR_NO_SYSTEM_RESOURCES); case SEC_E_INVALID_HANDLE: return APR_FROM_OS_ERROR(ERROR_INVALID_HANDLE); case SEC_E_UNSUPPORTED_FUNCTION: return APR_FROM_OS_ERROR(ERROR_INVALID_FUNCTION); case SEC_E_TARGET_UNKNOWN: return APR_FROM_OS_ERROR(ERROR_BAD_NETPATH); case SEC_E_INTERNAL_ERROR: return APR_FROM_OS_ERROR(ERROR_INTERNAL_ERROR); case SEC_E_SECPKG_NOT_FOUND: case SEC_E_BAD_PKGID: return APR_FROM_OS_ERROR(ERROR_NO_SUCH_PACKAGE); case SEC_E_NO_IMPERSONATION: return APR_FROM_OS_ERROR(ERROR_CANNOT_IMPERSONATE); case SEC_E_NO_AUTHENTICATING_AUTHORITY: return APR_FROM_OS_ERROR(ERROR_NO_LOGON_SERVERS); case SEC_E_UNTRUSTED_ROOT: return APR_FROM_OS_ERROR(ERROR_TRUST_FAILURE); case SEC_E_WRONG_PRINCIPAL: return APR_FROM_OS_ERROR(ERROR_WRONG_TARGET_NAME); case SEC_E_MUTUAL_AUTH_FAILED: return APR_FROM_OS_ERROR(ERROR_MUTUAL_AUTH_FAILED); case SEC_E_TIME_SKEW: return APR_FROM_OS_ERROR(ERROR_TIME_SKEW); default: return SERF_ERROR_AUTHN_FAILED; } } /* Cleans the SSPI context object, when the pool used to create it gets cleared or destroyed. */ static apr_status_t cleanup_ctx(void *data) { serf__spnego_context_t *ctx = data; if (SecIsValidHandle(&ctx->sspi_context)) { DeleteSecurityContext(&ctx->sspi_context); SecInvalidateHandle(&ctx->sspi_context); } if (SecIsValidHandle(&ctx->sspi_credentials)) { - FreeCredentialsHandle(&ctx->sspi_context); - SecInvalidateHandle(&ctx->sspi_context); + FreeCredentialsHandle(&ctx->sspi_credentials); + SecInvalidateHandle(&ctx->sspi_credentials); } return APR_SUCCESS; } static apr_status_t cleanup_sec_buffer(void *data) { FreeContextBuffer(data); return APR_SUCCESS; } apr_status_t serf__spnego_create_sec_context(serf__spnego_context_t **ctx_p, const serf__authn_scheme_t *scheme, apr_pool_t *result_pool, apr_pool_t *scratch_pool) { SECURITY_STATUS sspi_status; serf__spnego_context_t *ctx; const char *sspi_package; ctx = apr_pcalloc(result_pool, sizeof(*ctx)); SecInvalidateHandle(&ctx->sspi_context); SecInvalidateHandle(&ctx->sspi_credentials); ctx->initalized = FALSE; ctx->pool = result_pool; ctx->target_name = NULL; ctx->authn_type = scheme->type; apr_pool_cleanup_register(result_pool, ctx, cleanup_ctx, apr_pool_cleanup_null); if (ctx->authn_type == SERF_AUTHN_NEGOTIATE) sspi_package = "Negotiate"; else sspi_package = "NTLM"; sspi_status = AcquireCredentialsHandle( NULL, sspi_package, SECPKG_CRED_OUTBOUND, NULL, NULL, NULL, NULL, &ctx->sspi_credentials, NULL); if (FAILED(sspi_status)) { return map_sspi_status(sspi_status); } *ctx_p = ctx; return APR_SUCCESS; } static apr_status_t get_canonical_hostname(const char **canonname, const char *hostname, apr_pool_t *pool) { struct addrinfo hints; struct addrinfo *addrinfo; ZeroMemory(&hints, sizeof(hints)); hints.ai_flags = AI_CANONNAME; if (getaddrinfo(hostname, NULL, &hints, &addrinfo)) { return apr_get_netos_error(); } if (addrinfo) { *canonname = apr_pstrdup(pool, addrinfo->ai_canonname); } else { *canonname = apr_pstrdup(pool, hostname); } freeaddrinfo(addrinfo); return APR_SUCCESS; } apr_status_t serf__spnego_reset_sec_context(serf__spnego_context_t *ctx) { if (SecIsValidHandle(&ctx->sspi_context)) { DeleteSecurityContext(&ctx->sspi_context); SecInvalidateHandle(&ctx->sspi_context); } ctx->initalized = FALSE; return APR_SUCCESS; } apr_status_t serf__spnego_init_sec_context(serf_connection_t *conn, serf__spnego_context_t *ctx, const char *service, const char *hostname, serf__spnego_buffer_t *input_buf, serf__spnego_buffer_t *output_buf, apr_pool_t *result_pool, apr_pool_t *scratch_pool ) { SECURITY_STATUS status; ULONG actual_attr; SecBuffer sspi_in_buffer; SecBufferDesc sspi_in_buffer_desc; SecBuffer sspi_out_buffer; SecBufferDesc sspi_out_buffer_desc; apr_status_t apr_status; const char *canonname; if (!ctx->initalized && ctx->authn_type == SERF_AUTHN_NEGOTIATE) { apr_status = get_canonical_hostname(&canonname, hostname, scratch_pool); if (apr_status) { return apr_status; } ctx->target_name = apr_pstrcat(scratch_pool, service, "/", canonname, NULL); serf__log_skt(AUTH_VERBOSE, __FILE__, conn->skt, "Using SPN '%s' for '%s'\n", ctx->target_name, hostname); } else if (ctx->authn_type == SERF_AUTHN_NTLM) { /* Target name is not used for NTLM authentication. */ ctx->target_name = NULL; } /* Prepare input buffer description. */ sspi_in_buffer.BufferType = SECBUFFER_TOKEN; sspi_in_buffer.pvBuffer = input_buf->value; sspi_in_buffer.cbBuffer = input_buf->length; sspi_in_buffer_desc.cBuffers = 1; sspi_in_buffer_desc.pBuffers = &sspi_in_buffer; sspi_in_buffer_desc.ulVersion = SECBUFFER_VERSION; /* Output buffers. Output buffer will be allocated by system. */ sspi_out_buffer.BufferType = SECBUFFER_TOKEN; sspi_out_buffer.pvBuffer = NULL; sspi_out_buffer.cbBuffer = 0; sspi_out_buffer_desc.cBuffers = 1; sspi_out_buffer_desc.pBuffers = &sspi_out_buffer; sspi_out_buffer_desc.ulVersion = SECBUFFER_VERSION; status = InitializeSecurityContext( &ctx->sspi_credentials, ctx->initalized ? &ctx->sspi_context : NULL, ctx->target_name, ISC_REQ_ALLOCATE_MEMORY | ISC_REQ_MUTUAL_AUTH | ISC_REQ_CONFIDENTIALITY, 0, /* Reserved1 */ SECURITY_NETWORK_DREP, &sspi_in_buffer_desc, 0, /* Reserved2 */ &ctx->sspi_context, &sspi_out_buffer_desc, &actual_attr, NULL); if (sspi_out_buffer.cbBuffer > 0) { apr_pool_cleanup_register(result_pool, sspi_out_buffer.pvBuffer, cleanup_sec_buffer, apr_pool_cleanup_null); } ctx->initalized = TRUE; /* Finish authentication if SSPI requires so. */ if (status == SEC_I_COMPLETE_NEEDED || status == SEC_I_COMPLETE_AND_CONTINUE) { CompleteAuthToken(&ctx->sspi_context, &sspi_out_buffer_desc); } output_buf->value = sspi_out_buffer.pvBuffer; output_buf->length = sspi_out_buffer.cbBuffer; switch(status) { case SEC_I_COMPLETE_AND_CONTINUE: case SEC_I_CONTINUE_NEEDED: return APR_EAGAIN; case SEC_I_COMPLETE_NEEDED: case SEC_E_OK: return APR_SUCCESS; default: return map_sspi_status(status); } } #endif /* SERF_USE_SSPI */ Index: vendor/serf/dist/buckets/deflate_buckets.c =================================================================== --- vendor/serf/dist/buckets/deflate_buckets.c (revision 286498) +++ vendor/serf/dist/buckets/deflate_buckets.c (revision 286499) @@ -1,384 +1,403 @@ /* Copyright 2002-2004 Justin Erenkrantz and Greg Stein * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ #include #include /* This conditional isn't defined anywhere yet. */ #ifdef HAVE_ZUTIL_H #include #endif #include "serf.h" #include "serf_bucket_util.h" /* magic header */ static char deflate_magic[2] = { '\037', '\213' }; #define DEFLATE_MAGIC_SIZE 10 #define DEFLATE_VERIFY_SIZE 8 #define DEFLATE_BUFFER_SIZE 8096 static const int DEFLATE_WINDOW_SIZE = -15; static const int DEFLATE_MEMLEVEL = 9; typedef struct { serf_bucket_t *stream; serf_bucket_t *inflate_stream; int format; /* Are we 'deflate' or 'gzip'? */ enum { STATE_READING_HEADER, /* reading the gzip header */ STATE_HEADER, /* read the gzip header */ STATE_INIT, /* init'ing zlib functions */ STATE_INFLATE, /* inflating the content now */ STATE_READING_VERIFY, /* reading the final gzip CRC */ STATE_VERIFY, /* verifying the final gzip CRC */ STATE_FINISH, /* clean up after reading body */ STATE_DONE, /* body is done; we'll return EOF here */ } state; z_stream zstream; char hdr_buffer[DEFLATE_MAGIC_SIZE]; unsigned char buffer[DEFLATE_BUFFER_SIZE]; unsigned long crc; int windowSize; int memLevel; int bufferSize; /* How much of the chunk, or the terminator, do we have left to read? */ apr_size_t stream_left; /* How much are we supposed to read? */ apr_size_t stream_size; int stream_status; /* What was the last status we read? */ } deflate_context_t; /* Inputs a string and returns a long. */ static unsigned long getLong(unsigned char *string) { return ((unsigned long)string[0]) | (((unsigned long)string[1]) << 8) | (((unsigned long)string[2]) << 16) | (((unsigned long)string[3]) << 24); } serf_bucket_t *serf_bucket_deflate_create( serf_bucket_t *stream, serf_bucket_alloc_t *allocator, int format) { deflate_context_t *ctx; ctx = serf_bucket_mem_alloc(allocator, sizeof(*ctx)); ctx->stream = stream; ctx->stream_status = APR_SUCCESS; ctx->inflate_stream = serf_bucket_aggregate_create(allocator); ctx->format = format; ctx->crc = 0; /* zstream must be NULL'd out. */ memset(&ctx->zstream, 0, sizeof(ctx->zstream)); switch (ctx->format) { case SERF_DEFLATE_GZIP: ctx->state = STATE_READING_HEADER; break; case SERF_DEFLATE_DEFLATE: /* deflate doesn't have a header. */ ctx->state = STATE_INIT; break; default: /* Not reachable */ return NULL; } /* Initial size of gzip header. */ ctx->stream_left = ctx->stream_size = DEFLATE_MAGIC_SIZE; ctx->windowSize = DEFLATE_WINDOW_SIZE; ctx->memLevel = DEFLATE_MEMLEVEL; ctx->bufferSize = DEFLATE_BUFFER_SIZE; return serf_bucket_create(&serf_bucket_type_deflate, allocator, ctx); } static void serf_deflate_destroy_and_data(serf_bucket_t *bucket) { deflate_context_t *ctx = bucket->data; if (ctx->state > STATE_INIT && ctx->state <= STATE_FINISH) inflateEnd(&ctx->zstream); /* We may have appended inflate_stream into the stream bucket. * If so, avoid free'ing it twice. */ if (ctx->inflate_stream) { serf_bucket_destroy(ctx->inflate_stream); } serf_bucket_destroy(ctx->stream); serf_default_destroy_and_data(bucket); } static apr_status_t serf_deflate_read(serf_bucket_t *bucket, apr_size_t requested, const char **data, apr_size_t *len) { deflate_context_t *ctx = bucket->data; - unsigned long compCRC, compLen; apr_status_t status; const char *private_data; apr_size_t private_len; int zRC; while (1) { switch (ctx->state) { case STATE_READING_HEADER: case STATE_READING_VERIFY: status = serf_bucket_read(ctx->stream, ctx->stream_left, &private_data, &private_len); if (SERF_BUCKET_READ_ERROR(status)) { return status; } memcpy(ctx->hdr_buffer + (ctx->stream_size - ctx->stream_left), private_data, private_len); ctx->stream_left -= private_len; if (ctx->stream_left == 0) { ctx->state++; if (APR_STATUS_IS_EAGAIN(status)) { *len = 0; return status; } } else if (status) { *len = 0; return status; } break; case STATE_HEADER: if (ctx->hdr_buffer[0] != deflate_magic[0] || ctx->hdr_buffer[1] != deflate_magic[1]) { return SERF_ERROR_DECOMPRESSION_FAILED; } if (ctx->hdr_buffer[3] != 0) { return SERF_ERROR_DECOMPRESSION_FAILED; } ctx->state++; break; case STATE_VERIFY: + { + unsigned long compCRC, compLen, actualLen; + /* Do the checksum computation. */ compCRC = getLong((unsigned char*)ctx->hdr_buffer); if (ctx->crc != compCRC) { return SERF_ERROR_DECOMPRESSION_FAILED; } compLen = getLong((unsigned char*)ctx->hdr_buffer + 4); - if (ctx->zstream.total_out != compLen) { + /* The length in the trailer is module 2^32, so do the same for + the actual length. */ + actualLen = ctx->zstream.total_out; + actualLen &= 0xFFFFFFFF; + if (actualLen != compLen) { return SERF_ERROR_DECOMPRESSION_FAILED; } ctx->state++; break; + } case STATE_INIT: zRC = inflateInit2(&ctx->zstream, ctx->windowSize); if (zRC != Z_OK) { return SERF_ERROR_DECOMPRESSION_FAILED; } ctx->zstream.next_out = ctx->buffer; ctx->zstream.avail_out = ctx->bufferSize; ctx->state++; break; case STATE_FINISH: inflateEnd(&ctx->zstream); serf_bucket_aggregate_prepend(ctx->stream, ctx->inflate_stream); ctx->inflate_stream = 0; ctx->state++; break; case STATE_INFLATE: /* Do we have anything already uncompressed to read? */ status = serf_bucket_read(ctx->inflate_stream, requested, data, len); if (SERF_BUCKET_READ_ERROR(status)) { return status; } /* Hide EOF. */ if (APR_STATUS_IS_EOF(status)) { status = ctx->stream_status; if (APR_STATUS_IS_EOF(status)) { /* We've read all of the data from our stream, but we * need to continue to iterate until we flush * out the zlib buffer. */ status = APR_SUCCESS; } } if (*len != 0) { return status; } /* We tried; but we have nothing buffered. Fetch more. */ /* It is possible that we maxed out avail_out before * exhausting avail_in; therefore, continue using the * previous buffer. Otherwise, fetch more data from * our stream bucket. */ if (ctx->zstream.avail_in == 0) { /* When we empty our inflated stream, we'll return this * status - this allow us to eventually pass up EAGAINs. */ ctx->stream_status = serf_bucket_read(ctx->stream, ctx->bufferSize, &private_data, &private_len); if (SERF_BUCKET_READ_ERROR(ctx->stream_status)) { return ctx->stream_status; } if (!private_len && APR_STATUS_IS_EAGAIN(ctx->stream_status)) { *len = 0; status = ctx->stream_status; ctx->stream_status = APR_SUCCESS; return status; } ctx->zstream.next_in = (unsigned char*)private_data; ctx->zstream.avail_in = private_len; } - zRC = Z_OK; - while (ctx->zstream.avail_in != 0) { - /* We're full, clear out our buffer, reset, and return. */ - if (ctx->zstream.avail_out == 0) { + + while (1) { + + zRC = inflate(&ctx->zstream, Z_NO_FLUSH); + + /* We're full or zlib requires more space. Either case, clear + out our buffer, reset, and return. */ + if (zRC == Z_BUF_ERROR || ctx->zstream.avail_out == 0) { serf_bucket_t *tmp; ctx->zstream.next_out = ctx->buffer; private_len = ctx->bufferSize - ctx->zstream.avail_out; ctx->crc = crc32(ctx->crc, (const Bytef *)ctx->buffer, private_len); /* FIXME: There probably needs to be a free func. */ tmp = SERF_BUCKET_SIMPLE_STRING_LEN((char *)ctx->buffer, private_len, bucket->allocator); serf_bucket_aggregate_append(ctx->inflate_stream, tmp); ctx->zstream.avail_out = ctx->bufferSize; break; } - zRC = inflate(&ctx->zstream, Z_NO_FLUSH); if (zRC == Z_STREAM_END) { serf_bucket_t *tmp; private_len = ctx->bufferSize - ctx->zstream.avail_out; ctx->crc = crc32(ctx->crc, (const Bytef *)ctx->buffer, private_len); /* FIXME: There probably needs to be a free func. */ tmp = SERF_BUCKET_SIMPLE_STRING_LEN((char *)ctx->buffer, private_len, bucket->allocator); serf_bucket_aggregate_append(ctx->inflate_stream, tmp); ctx->zstream.avail_out = ctx->bufferSize; /* Push back the remaining data to be read. */ tmp = serf_bucket_aggregate_create(bucket->allocator); serf_bucket_aggregate_prepend(tmp, ctx->stream); ctx->stream = tmp; /* We now need to take the remaining avail_in and * throw it in ctx->stream so our next read picks it up. */ tmp = SERF_BUCKET_SIMPLE_STRING_LEN( (const char*)ctx->zstream.next_in, ctx->zstream.avail_in, bucket->allocator); serf_bucket_aggregate_prepend(ctx->stream, tmp); switch (ctx->format) { case SERF_DEFLATE_GZIP: ctx->stream_left = ctx->stream_size = DEFLATE_VERIFY_SIZE; ctx->state++; break; case SERF_DEFLATE_DEFLATE: /* Deflate does not have a verify footer. */ ctx->state = STATE_FINISH; break; default: /* Not reachable */ return APR_EGENERAL; } break; } + + /* Any other error? */ if (zRC != Z_OK) { return SERF_ERROR_DECOMPRESSION_FAILED; } + + /* As long as zRC == Z_OK, just keep looping. */ } /* Okay, we've inflated. Try to read. */ status = serf_bucket_read(ctx->inflate_stream, requested, data, len); /* Hide EOF. */ if (APR_STATUS_IS_EOF(status)) { status = ctx->stream_status; - /* If our stream is finished too, return SUCCESS so - * we'll iterate one more time. + + /* If the inflation wasn't finished, return APR_SUCCESS. */ + if (zRC != Z_STREAM_END) + return APR_SUCCESS; + + /* If our stream is finished too and all data was inflated, + * return SUCCESS so we'll iterate one more time. */ if (APR_STATUS_IS_EOF(status)) { /* No more data to read from the stream, and everything inflated. If all data was received correctly, state should have been advanced to STATE_READING_VERIFY or STATE_FINISH. If not, then the data was incomplete and we have an error. */ if (ctx->state != STATE_INFLATE) return APR_SUCCESS; else return SERF_ERROR_DECOMPRESSION_FAILED; } } return status; case STATE_DONE: /* We're done inflating. Use our finished buffer. */ return serf_bucket_read(ctx->stream, requested, data, len); default: /* Not reachable */ return APR_EGENERAL; } } /* NOTREACHED */ } /* ### need to implement */ #define serf_deflate_readline NULL #define serf_deflate_peek NULL const serf_bucket_type_t serf_bucket_type_deflate = { "DEFLATE", serf_deflate_read, serf_deflate_readline, serf_default_read_iovec, serf_default_read_for_sendfile, serf_default_read_bucket, serf_deflate_peek, serf_deflate_destroy_and_data, }; Index: vendor/serf/dist/buckets/ssl_buckets.c =================================================================== --- vendor/serf/dist/buckets/ssl_buckets.c (revision 286498) +++ vendor/serf/dist/buckets/ssl_buckets.c (revision 286499) @@ -1,1898 +1,1900 @@ /* Copyright 2002-2004 Justin Erenkrantz and Greg Stein * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * * ---- * * For the OpenSSL thread-safety locking code: * * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * * Originally developed by Aaron Bannert and Justin Erenkrantz, eBuilt. */ #include #include #include #include #include #include #include #include "serf.h" #include "serf_private.h" #include "serf_bucket_util.h" #include #include #include #include #include #ifndef APR_VERSION_AT_LEAST /* Introduced in APR 1.3.0 */ #define APR_VERSION_AT_LEAST(major,minor,patch) \ (((major) < APR_MAJOR_VERSION) \ || ((major) == APR_MAJOR_VERSION && (minor) < APR_MINOR_VERSION) \ || ((major) == APR_MAJOR_VERSION && (minor) == APR_MINOR_VERSION && \ (patch) <= APR_PATCH_VERSION)) #endif /* APR_VERSION_AT_LEAST */ #ifndef APR_ARRAY_PUSH #define APR_ARRAY_PUSH(ary,type) (*((type *)apr_array_push(ary))) #endif /* * Here's an overview of the SSL bucket's relationship to OpenSSL and serf. * * HTTP request: SSLENCRYPT(REQUEST) * [context.c reads from SSLENCRYPT and writes out to the socket] * HTTP response: RESPONSE(SSLDECRYPT(SOCKET)) * [handler function reads from RESPONSE which in turn reads from SSLDECRYPT] * * HTTP request read call path: * * write_to_connection * |- serf_bucket_read on SSLENCRYPT * |- serf_ssl_read * |- serf_databuf_read * |- common_databuf_prep * |- ssl_encrypt * |- 1. Try to read pending encrypted data; If available, return. * |- 2. Try to read from ctx->stream [REQUEST bucket] * |- 3. Call SSL_write with read data * |- ... * |- bio_bucket_read can be called * |- bio_bucket_write with encrypted data * |- store in sink * |- 4. If successful, read pending encrypted data and return. * |- 5. If fails, place read data back in ctx->stream * * HTTP response read call path: * * read_from_connection * |- acceptor * |- handler * |- ... * |- serf_bucket_read(SSLDECRYPT) * |- serf_ssl_read * |- serf_databuf_read * |- ssl_decrypt * |- 1. SSL_read() for pending decrypted data; if any, return. * |- 2. Try to read from ctx->stream [SOCKET bucket] * |- 3. Append data to ssl_ctx->source * |- 4. Call SSL_read() * |- ... * |- bio_bucket_write can be called * |- bio_bucket_read * |- read data from ssl_ctx->source * |- If data read, return it. * |- If an error, set the STATUS value and return. * */ typedef struct bucket_list { serf_bucket_t *bucket; struct bucket_list *next; } bucket_list_t; typedef struct { /* Helper to read data. Wraps stream. */ serf_databuf_t databuf; /* Our source for more data. */ serf_bucket_t *stream; /* The next set of buckets */ bucket_list_t *stream_next; /* The status of the last thing we read. */ apr_status_t status; apr_status_t exhausted; int exhausted_reset; /* Data we've read but not processed. */ serf_bucket_t *pending; } serf_ssl_stream_t; struct serf_ssl_context_t { /* How many open buckets refer to this context. */ int refcount; /* The pool that this context uses. */ apr_pool_t *pool; /* The allocator associated with the above pool. */ serf_bucket_alloc_t *allocator; /* Internal OpenSSL parameters */ SSL_CTX *ctx; SSL *ssl; BIO *bio; serf_ssl_stream_t encrypt; serf_ssl_stream_t decrypt; /* Client cert callbacks */ serf_ssl_need_client_cert_t cert_callback; void *cert_userdata; apr_pool_t *cert_cache_pool; const char *cert_file_success; /* Client cert PW callbacks */ serf_ssl_need_cert_password_t cert_pw_callback; void *cert_pw_userdata; apr_pool_t *cert_pw_cache_pool; const char *cert_pw_success; /* Server cert callbacks */ serf_ssl_need_server_cert_t server_cert_callback; serf_ssl_server_cert_chain_cb_t server_cert_chain_callback; void *server_cert_userdata; const char *cert_path; X509 *cached_cert; EVP_PKEY *cached_cert_pw; apr_status_t pending_err; /* Status of a fatal error, returned on subsequent encrypt or decrypt requests. */ apr_status_t fatal_err; }; typedef struct { /* The bucket-independent ssl context that this bucket is associated with */ serf_ssl_context_t *ssl_ctx; /* Pointer to the 'right' databuf. */ serf_databuf_t *databuf; /* Pointer to our stream, so we can find it later. */ serf_bucket_t **our_stream; } ssl_context_t; struct serf_ssl_certificate_t { X509 *ssl_cert; int depth; }; static void disable_compression(serf_ssl_context_t *ssl_ctx); static char * pstrdup_escape_nul_bytes(const char *buf, int len, apr_pool_t *pool); #if SSL_VERBOSE /* Log all ssl alerts that we receive from the server. */ static void apps_ssl_info_callback(const SSL *s, int where, int ret) { const char *str; int w; w = where & ~SSL_ST_MASK; if (w & SSL_ST_CONNECT) str = "SSL_connect"; else if (w & SSL_ST_ACCEPT) str = "SSL_accept"; else str = "undefined"; if (where & SSL_CB_LOOP) { serf__log(SSL_VERBOSE, __FILE__, "%s:%s\n", str, SSL_state_string_long(s)); } else if (where & SSL_CB_ALERT) { str = (where & SSL_CB_READ) ? "read" : "write"; serf__log(SSL_VERBOSE, __FILE__, "SSL3 alert %s:%s:%s\n", str, SSL_alert_type_string_long(ret), SSL_alert_desc_string_long(ret)); } else if (where & SSL_CB_EXIT) { if (ret == 0) serf__log(SSL_VERBOSE, __FILE__, "%s:failed in %s\n", str, SSL_state_string_long(s)); else if (ret < 0) { serf__log(SSL_VERBOSE, __FILE__, "%s:error in %s\n", str, SSL_state_string_long(s)); } } } #endif /* Returns the amount read. */ static int bio_bucket_read(BIO *bio, char *in, int inlen) { serf_ssl_context_t *ctx = bio->ptr; const char *data; apr_status_t status; apr_size_t len; serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_read called for %d bytes\n", inlen); if (ctx->encrypt.status == SERF_ERROR_WAIT_CONN && BIO_should_read(ctx->bio)) { serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_read waiting: (%d %d %d)\n", BIO_should_retry(ctx->bio), BIO_should_read(ctx->bio), BIO_get_retry_flags(ctx->bio)); /* Falling back... */ ctx->encrypt.exhausted_reset = 1; BIO_clear_retry_flags(bio); } status = serf_bucket_read(ctx->decrypt.pending, inlen, &data, &len); ctx->decrypt.status = status; serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_read received %d bytes (%d)\n", len, status); if (!SERF_BUCKET_READ_ERROR(status)) { /* Oh suck. */ if (len) { memcpy(in, data, len); return len; } if (APR_STATUS_IS_EOF(status)) { BIO_set_retry_read(bio); return -1; } } return -1; } /* Returns the amount written. */ static int bio_bucket_write(BIO *bio, const char *in, int inl) { serf_ssl_context_t *ctx = bio->ptr; serf_bucket_t *tmp; serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_write called for %d bytes\n", inl); if (ctx->encrypt.status == SERF_ERROR_WAIT_CONN && !BIO_should_read(ctx->bio)) { serf__log(SSL_VERBOSE, __FILE__, "bio_bucket_write waiting: (%d %d %d)\n", BIO_should_retry(ctx->bio), BIO_should_read(ctx->bio), BIO_get_retry_flags(ctx->bio)); /* Falling back... */ ctx->encrypt.exhausted_reset = 1; BIO_clear_retry_flags(bio); } tmp = serf_bucket_simple_copy_create(in, inl, ctx->encrypt.pending->allocator); serf_bucket_aggregate_append(ctx->encrypt.pending, tmp); return inl; } /* Returns the amount read. */ static int bio_file_read(BIO *bio, char *in, int inlen) { apr_file_t *file = bio->ptr; apr_status_t status; apr_size_t len; BIO_clear_retry_flags(bio); len = inlen; status = apr_file_read(file, in, &len); if (!SERF_BUCKET_READ_ERROR(status)) { /* Oh suck. */ if (APR_STATUS_IS_EOF(status)) { BIO_set_retry_read(bio); return -1; } else { return len; } } return -1; } /* Returns the amount written. */ static int bio_file_write(BIO *bio, const char *in, int inl) { apr_file_t *file = bio->ptr; apr_size_t nbytes; BIO_clear_retry_flags(bio); nbytes = inl; apr_file_write(file, in, &nbytes); return nbytes; } static int bio_file_gets(BIO *bio, char *in, int inlen) { return bio_file_read(bio, in, inlen); } static int bio_bucket_create(BIO *bio) { bio->shutdown = 1; bio->init = 1; bio->num = -1; bio->ptr = NULL; return 1; } static int bio_bucket_destroy(BIO *bio) { /* Did we already free this? */ if (bio == NULL) { return 0; } return 1; } static long bio_bucket_ctrl(BIO *bio, int cmd, long num, void *ptr) { long ret = 1; switch (cmd) { default: /* abort(); */ break; case BIO_CTRL_FLUSH: /* At this point we can't force a flush. */ break; case BIO_CTRL_PUSH: case BIO_CTRL_POP: ret = 0; break; } return ret; } static BIO_METHOD bio_bucket_method = { BIO_TYPE_MEM, "Serf SSL encryption and decryption buckets", bio_bucket_write, bio_bucket_read, NULL, /* Is this called? */ NULL, /* Is this called? */ bio_bucket_ctrl, bio_bucket_create, bio_bucket_destroy, #ifdef OPENSSL_VERSION_NUMBER NULL /* sslc does not have the callback_ctrl field */ #endif }; static BIO_METHOD bio_file_method = { BIO_TYPE_FILE, "Wrapper around APR file structures", bio_file_write, bio_file_read, NULL, /* Is this called? */ bio_file_gets, /* Is this called? */ bio_bucket_ctrl, bio_bucket_create, bio_bucket_destroy, #ifdef OPENSSL_VERSION_NUMBER NULL /* sslc does not have the callback_ctrl field */ #endif }; typedef enum san_copy_t { EscapeNulAndCopy = 0, ErrorOnNul = 1, } san_copy_t; static apr_status_t get_subject_alt_names(apr_array_header_t **san_arr, X509 *ssl_cert, san_copy_t copy_action, apr_pool_t *pool) { STACK_OF(GENERAL_NAME) *names; /* assert: copy_action == ErrorOnNul || (san_arr && pool) */ if (san_arr) { *san_arr = NULL; } /* Get subjectAltNames */ names = X509_get_ext_d2i(ssl_cert, NID_subject_alt_name, NULL, NULL); if (names) { int names_count = sk_GENERAL_NAME_num(names); int name_idx; if (san_arr) *san_arr = apr_array_make(pool, names_count, sizeof(char*)); for (name_idx = 0; name_idx < names_count; name_idx++) { char *p = NULL; GENERAL_NAME *nm = sk_GENERAL_NAME_value(names, name_idx); switch (nm->type) { case GEN_DNS: if (copy_action == ErrorOnNul && strlen(nm->d.ia5->data) != nm->d.ia5->length) return SERF_ERROR_SSL_CERT_FAILED; if (san_arr && *san_arr) p = pstrdup_escape_nul_bytes((const char *)nm->d.ia5->data, nm->d.ia5->length, pool); break; default: /* Don't know what to do - skip. */ break; } if (p) { APR_ARRAY_PUSH(*san_arr, char*) = p; } } sk_GENERAL_NAME_pop_free(names, GENERAL_NAME_free); } return APR_SUCCESS; } static apr_status_t validate_cert_hostname(X509 *server_cert, apr_pool_t *pool) { char buf[1024]; int length; apr_status_t ret; ret = get_subject_alt_names(NULL, server_cert, ErrorOnNul, NULL); if (ret) { return ret; } else { /* Fail if the subject's CN field contains \0 characters. */ X509_NAME *subject = X509_get_subject_name(server_cert); if (!subject) return SERF_ERROR_SSL_CERT_FAILED; length = X509_NAME_get_text_by_NID(subject, NID_commonName, buf, 1024); if (length != -1) if (strlen(buf) != length) return SERF_ERROR_SSL_CERT_FAILED; } return APR_SUCCESS; } static int validate_server_certificate(int cert_valid, X509_STORE_CTX *store_ctx) { SSL *ssl; serf_ssl_context_t *ctx; X509 *server_cert; int err, depth; int failures = 0; apr_status_t status; ssl = X509_STORE_CTX_get_ex_data(store_ctx, SSL_get_ex_data_X509_STORE_CTX_idx()); ctx = SSL_get_app_data(ssl); server_cert = X509_STORE_CTX_get_current_cert(store_ctx); depth = X509_STORE_CTX_get_error_depth(store_ctx); /* If the certification was found invalid, get the error and convert it to something our caller will understand. */ if (! cert_valid) { err = X509_STORE_CTX_get_error(store_ctx); switch(err) { case X509_V_ERR_CERT_NOT_YET_VALID: failures |= SERF_SSL_CERT_NOTYETVALID; break; case X509_V_ERR_CERT_HAS_EXPIRED: failures |= SERF_SSL_CERT_EXPIRED; break; case X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT: case X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN: failures |= SERF_SSL_CERT_SELF_SIGNED; break; case X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY: case X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT: case X509_V_ERR_CERT_UNTRUSTED: case X509_V_ERR_INVALID_CA: case X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE: failures |= SERF_SSL_CERT_UNKNOWNCA; break; case X509_V_ERR_CERT_REVOKED: failures |= SERF_SSL_CERT_REVOKED; break; default: failures |= SERF_SSL_CERT_UNKNOWN_FAILURE; break; } } /* Validate hostname */ status = validate_cert_hostname(server_cert, ctx->pool); if (status) failures |= SERF_SSL_CERT_UNKNOWN_FAILURE; /* Check certificate expiry dates. */ if (X509_cmp_current_time(X509_get_notBefore(server_cert)) >= 0) { failures |= SERF_SSL_CERT_NOTYETVALID; } else if (X509_cmp_current_time(X509_get_notAfter(server_cert)) <= 0) { failures |= SERF_SSL_CERT_EXPIRED; } if (ctx->server_cert_callback && (depth == 0 || failures)) { serf_ssl_certificate_t *cert; apr_pool_t *subpool; apr_pool_create(&subpool, ctx->pool); cert = apr_palloc(subpool, sizeof(serf_ssl_certificate_t)); cert->ssl_cert = server_cert; cert->depth = depth; /* Callback for further verification. */ status = ctx->server_cert_callback(ctx->server_cert_userdata, failures, cert); if (status == APR_SUCCESS) cert_valid = 1; else { /* Even if openssl found the certificate valid, the application told us to reject it. */ cert_valid = 0; /* Pass the error back to the caller through the context-run. */ ctx->pending_err = status; } apr_pool_destroy(subpool); } if (ctx->server_cert_chain_callback && (depth == 0 || failures)) { STACK_OF(X509) *chain; const serf_ssl_certificate_t **certs; int certs_len; apr_pool_t *subpool; apr_pool_create(&subpool, ctx->pool); /* Borrow the chain to pass to the callback. */ chain = X509_STORE_CTX_get_chain(store_ctx); /* If the chain can't be retrieved, just pass the current certificate. */ /* ### can this actually happen with _get_chain() ? */ if (!chain) { serf_ssl_certificate_t *cert = apr_palloc(subpool, sizeof(*cert)); cert->ssl_cert = server_cert; cert->depth = depth; /* Room for the server_cert and a trailing NULL. */ certs = apr_palloc(subpool, sizeof(*certs) * 2); certs[0] = cert; certs_len = 1; } else { int i; certs_len = sk_X509_num(chain); /* Room for all the certs and a trailing NULL. */ certs = apr_palloc(subpool, sizeof(*certs) * (certs_len + 1)); for (i = 0; i < certs_len; ++i) { serf_ssl_certificate_t *cert; cert = apr_palloc(subpool, sizeof(*cert)); cert->ssl_cert = sk_X509_value(chain, i); cert->depth = i; certs[i] = cert; } } certs[certs_len] = NULL; /* Callback for further verification. */ status = ctx->server_cert_chain_callback(ctx->server_cert_userdata, failures, depth, certs, certs_len); if (status == APR_SUCCESS) { cert_valid = 1; } else { /* Even if openssl found the certificate valid, the application told us to reject it. */ cert_valid = 0; /* Pass the error back to the caller through the context-run. */ ctx->pending_err = status; } apr_pool_destroy(subpool); } /* Return a specific error if the server certificate is not accepted by OpenSSL and the application has not set callbacks to override this. */ if (!cert_valid && !ctx->server_cert_chain_callback && !ctx->server_cert_callback) { ctx->pending_err = SERF_ERROR_SSL_CERT_FAILED; } return cert_valid; } /* This function reads an encrypted stream and returns the decrypted stream. */ static apr_status_t ssl_decrypt(void *baton, apr_size_t bufsize, char *buf, apr_size_t *len) { serf_ssl_context_t *ctx = baton; apr_size_t priv_len; apr_status_t status; const char *data; int ssl_len; if (ctx->fatal_err) return ctx->fatal_err; serf__log(SSL_VERBOSE, __FILE__, "ssl_decrypt: begin %d\n", bufsize); /* Is there some data waiting to be read? */ ssl_len = SSL_read(ctx->ssl, buf, bufsize); if (ssl_len > 0) { serf__log(SSL_VERBOSE, __FILE__, "ssl_decrypt: %d bytes (%d); status: %d; flags: %d\n", ssl_len, bufsize, ctx->decrypt.status, BIO_get_retry_flags(ctx->bio)); *len = ssl_len; return APR_SUCCESS; } status = serf_bucket_read(ctx->decrypt.stream, bufsize, &data, &priv_len); if (!SERF_BUCKET_READ_ERROR(status) && priv_len) { serf_bucket_t *tmp; serf__log(SSL_VERBOSE, __FILE__, "ssl_decrypt: read %d bytes (%d); status: %d\n", priv_len, bufsize, status); tmp = serf_bucket_simple_copy_create(data, priv_len, ctx->decrypt.pending->allocator); serf_bucket_aggregate_append(ctx->decrypt.pending, tmp); ssl_len = SSL_read(ctx->ssl, buf, bufsize); if (ssl_len < 0) { int ssl_err; ssl_err = SSL_get_error(ctx->ssl, ssl_len); switch (ssl_err) { case SSL_ERROR_SYSCALL: *len = 0; /* Return the underlying network error that caused OpenSSL to fail. ### This can be a crypt error! */ status = ctx->decrypt.status; break; case SSL_ERROR_WANT_READ: case SSL_ERROR_WANT_WRITE: *len = 0; status = APR_EAGAIN; break; case SSL_ERROR_SSL: *len = 0; if (ctx->pending_err) { status = ctx->pending_err; ctx->pending_err = 0; } else { ctx->fatal_err = status = SERF_ERROR_SSL_COMM_FAILED; } break; default: *len = 0; ctx->fatal_err = status = SERF_ERROR_SSL_COMM_FAILED; break; } } else if (ssl_len == 0) { /* The server shut down the connection. */ int ssl_err, shutdown; *len = 0; /* Check for SSL_RECEIVED_SHUTDOWN */ shutdown = SSL_get_shutdown(ctx->ssl); /* Check for SSL_ERROR_ZERO_RETURN */ ssl_err = SSL_get_error(ctx->ssl, ssl_len); if (shutdown == SSL_RECEIVED_SHUTDOWN && ssl_err == SSL_ERROR_ZERO_RETURN) { /* The server closed the SSL session. While this doesn't necessary mean the connection is closed, let's close it here anyway. We can optimize this later. */ serf__log(SSL_VERBOSE, __FILE__, "ssl_decrypt: SSL read error: server" " shut down connection!\n"); status = APR_EOF; } else { /* A fatal error occurred. */ ctx->fatal_err = status = SERF_ERROR_SSL_COMM_FAILED; } } else { *len = ssl_len; serf__log(SSL_MSG_VERBOSE, __FILE__, "---\n%.*s\n-(%d)-\n", *len, buf, *len); } } else { *len = 0; } serf__log(SSL_VERBOSE, __FILE__, "ssl_decrypt: %d %d %d\n", status, *len, BIO_get_retry_flags(ctx->bio)); return status; } /* This function reads a decrypted stream and returns an encrypted stream. */ static apr_status_t ssl_encrypt(void *baton, apr_size_t bufsize, char *buf, apr_size_t *len) { const char *data; apr_size_t interim_bufsize; serf_ssl_context_t *ctx = baton; apr_status_t status; if (ctx->fatal_err) return ctx->fatal_err; serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: begin %d\n", bufsize); /* Try to read already encrypted but unread data first. */ status = serf_bucket_read(ctx->encrypt.pending, bufsize, &data, len); if (SERF_BUCKET_READ_ERROR(status)) { return status; } /* Aha, we read something. Return that now. */ if (*len) { memcpy(buf, data, *len); if (APR_STATUS_IS_EOF(status)) { status = APR_SUCCESS; } serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: %d %d %d (quick read)\n", status, *len, BIO_get_retry_flags(ctx->bio)); return status; } if (BIO_should_retry(ctx->bio) && BIO_should_write(ctx->bio)) { serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: %d %d %d (should write exit)\n", status, *len, BIO_get_retry_flags(ctx->bio)); return APR_EAGAIN; } /* If we were previously blocked, unblock ourselves now. */ if (BIO_should_read(ctx->bio)) { serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: reset %d %d (%d %d %d)\n", status, ctx->encrypt.status, BIO_should_retry(ctx->bio), BIO_should_read(ctx->bio), BIO_get_retry_flags(ctx->bio)); ctx->encrypt.status = APR_SUCCESS; ctx->encrypt.exhausted_reset = 0; } /* Oh well, read from our stream now. */ interim_bufsize = bufsize; do { apr_size_t interim_len; if (!ctx->encrypt.status) { struct iovec vecs[64]; int vecs_read; status = serf_bucket_read_iovec(ctx->encrypt.stream, interim_bufsize, 64, vecs, &vecs_read); if (!SERF_BUCKET_READ_ERROR(status) && vecs_read) { char *vecs_data; int i, cur, vecs_data_len; int ssl_len; /* Combine the buffers of the iovec into one buffer, as that is with SSL_write requires. */ vecs_data_len = 0; for (i = 0; i < vecs_read; i++) { vecs_data_len += vecs[i].iov_len; } vecs_data = serf_bucket_mem_alloc(ctx->allocator, vecs_data_len); cur = 0; for (i = 0; i < vecs_read; i++) { memcpy(vecs_data + cur, vecs[i].iov_base, vecs[i].iov_len); cur += vecs[i].iov_len; } interim_bufsize -= vecs_data_len; interim_len = vecs_data_len; serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: bucket read %d bytes; "\ "status %d\n", interim_len, status); serf__log(SSL_MSG_VERBOSE, __FILE__, "---\n%.*s\n-(%d)-\n", interim_len, vecs_data, interim_len); /* Stash our status away. */ ctx->encrypt.status = status; ssl_len = SSL_write(ctx->ssl, vecs_data, interim_len); serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: SSL write: %d\n", ssl_len); /* If we failed to write... */ if (ssl_len < 0) { int ssl_err; /* Ah, bugger. We need to put that data back. Note: use the copy here, we do not own the original iovec data buffer so it will be freed on next read. */ serf_bucket_t *vecs_copy = serf_bucket_simple_own_create(vecs_data, vecs_data_len, ctx->allocator); serf_bucket_aggregate_prepend(ctx->encrypt.stream, vecs_copy); ssl_err = SSL_get_error(ctx->ssl, ssl_len); serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: SSL write error: %d\n", ssl_err); if (ssl_err == SSL_ERROR_SYSCALL) { /* Return the underlying network error that caused OpenSSL to fail. ### This can be a decrypt error! */ status = ctx->encrypt.status; if (SERF_BUCKET_READ_ERROR(status)) { return status; } } else { /* Oh, no. */ if (ssl_err == SSL_ERROR_WANT_READ) { status = SERF_ERROR_WAIT_CONN; } else { ctx->fatal_err = status = SERF_ERROR_SSL_COMM_FAILED; } } serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt: SSL write error: %d %d\n", status, *len); } else { /* We're done with this data. */ serf_bucket_mem_free(ctx->allocator, vecs_data); } } } else { interim_len = 0; *len = 0; status = ctx->encrypt.status; } } while (!status && interim_bufsize); /* Okay, we exhausted our underlying stream. */ if (!SERF_BUCKET_READ_ERROR(status)) { apr_status_t agg_status; struct iovec vecs[64]; int vecs_read, i; /* We read something! */ agg_status = serf_bucket_read_iovec(ctx->encrypt.pending, bufsize, 64, vecs, &vecs_read); *len = 0; for (i = 0; i < vecs_read; i++) { memcpy(buf + *len, vecs[i].iov_base, vecs[i].iov_len); *len += vecs[i].iov_len; } serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt read agg: %d %d %d %d\n", status, agg_status, ctx->encrypt.status, *len); if (!agg_status) { status = agg_status; } } if (status == SERF_ERROR_WAIT_CONN && BIO_should_retry(ctx->bio) && BIO_should_read(ctx->bio)) { ctx->encrypt.exhausted = ctx->encrypt.status; ctx->encrypt.status = SERF_ERROR_WAIT_CONN; } serf__log(SSL_VERBOSE, __FILE__, "ssl_encrypt finished: %d %d (%d %d %d)\n", status, *len, BIO_should_retry(ctx->bio), BIO_should_read(ctx->bio), BIO_get_retry_flags(ctx->bio)); return status; } #if APR_HAS_THREADS static apr_pool_t *ssl_pool; static apr_thread_mutex_t **ssl_locks; typedef struct CRYPTO_dynlock_value { apr_thread_mutex_t *lock; } CRYPTO_dynlock_value; static CRYPTO_dynlock_value *ssl_dyn_create(const char* file, int line) { CRYPTO_dynlock_value *l; apr_status_t rv; l = apr_palloc(ssl_pool, sizeof(CRYPTO_dynlock_value)); rv = apr_thread_mutex_create(&l->lock, APR_THREAD_MUTEX_DEFAULT, ssl_pool); if (rv != APR_SUCCESS) { /* FIXME: return error here */ } return l; } static void ssl_dyn_lock(int mode, CRYPTO_dynlock_value *l, const char *file, int line) { if (mode & CRYPTO_LOCK) { apr_thread_mutex_lock(l->lock); } else if (mode & CRYPTO_UNLOCK) { apr_thread_mutex_unlock(l->lock); } } static void ssl_dyn_destroy(CRYPTO_dynlock_value *l, const char *file, int line) { apr_thread_mutex_destroy(l->lock); } static void ssl_lock(int mode, int n, const char *file, int line) { if (mode & CRYPTO_LOCK) { apr_thread_mutex_lock(ssl_locks[n]); } else if (mode & CRYPTO_UNLOCK) { apr_thread_mutex_unlock(ssl_locks[n]); } } static unsigned long ssl_id(void) { /* FIXME: This is lame and not portable. -aaron */ return (unsigned long) apr_os_thread_current(); } static apr_status_t cleanup_ssl(void *data) { CRYPTO_set_locking_callback(NULL); CRYPTO_set_id_callback(NULL); CRYPTO_set_dynlock_create_callback(NULL); CRYPTO_set_dynlock_lock_callback(NULL); CRYPTO_set_dynlock_destroy_callback(NULL); return APR_SUCCESS; } #endif #if !APR_VERSION_AT_LEAST(1,0,0) #define apr_atomic_cas32(mem, with, cmp) apr_atomic_cas(mem, with, cmp) #endif enum ssl_init_e { INIT_UNINITIALIZED = 0, INIT_BUSY = 1, INIT_DONE = 2 }; static volatile apr_uint32_t have_init_ssl = INIT_UNINITIALIZED; static void init_ssl_libraries(void) { apr_uint32_t val; val = apr_atomic_cas32(&have_init_ssl, INIT_BUSY, INIT_UNINITIALIZED); if (!val) { #if APR_HAS_THREADS int i, numlocks; #endif #ifdef SSL_VERBOSE /* Warn when compile-time and run-time version of OpenSSL differ in major/minor version number. */ long libver = SSLeay(); if ((libver ^ OPENSSL_VERSION_NUMBER) & 0xFFF00000) { serf__log(SSL_VERBOSE, __FILE__, "Warning: OpenSSL library version mismatch, compile-time " "was %lx, runtime is %lx.\n", OPENSSL_VERSION_NUMBER, libver); } #endif CRYPTO_malloc_init(); ERR_load_crypto_strings(); SSL_load_error_strings(); SSL_library_init(); OpenSSL_add_all_algorithms(); #if APR_HAS_THREADS numlocks = CRYPTO_num_locks(); apr_pool_create(&ssl_pool, NULL); ssl_locks = apr_palloc(ssl_pool, sizeof(apr_thread_mutex_t*)*numlocks); for (i = 0; i < numlocks; i++) { apr_status_t rv; /* Intraprocess locks don't /need/ a filename... */ rv = apr_thread_mutex_create(&ssl_locks[i], APR_THREAD_MUTEX_DEFAULT, ssl_pool); if (rv != APR_SUCCESS) { /* FIXME: error out here */ } } CRYPTO_set_locking_callback(ssl_lock); CRYPTO_set_id_callback(ssl_id); CRYPTO_set_dynlock_create_callback(ssl_dyn_create); CRYPTO_set_dynlock_lock_callback(ssl_dyn_lock); CRYPTO_set_dynlock_destroy_callback(ssl_dyn_destroy); apr_pool_cleanup_register(ssl_pool, NULL, cleanup_ssl, cleanup_ssl); #endif apr_atomic_cas32(&have_init_ssl, INIT_DONE, INIT_BUSY); } else { /* Make sure we don't continue before the initialization in another thread has completed */ while (val != INIT_DONE) { apr_sleep(APR_USEC_PER_SEC / 1000); val = apr_atomic_cas32(&have_init_ssl, INIT_UNINITIALIZED, INIT_UNINITIALIZED); } } } static int ssl_need_client_cert(SSL *ssl, X509 **cert, EVP_PKEY **pkey) { serf_ssl_context_t *ctx = SSL_get_app_data(ssl); apr_status_t status; if (ctx->cached_cert) { *cert = ctx->cached_cert; *pkey = ctx->cached_cert_pw; return 1; } while (ctx->cert_callback) { const char *cert_path; apr_file_t *cert_file; BIO *bio; PKCS12 *p12; int i; int retrying_success = 0; if (ctx->cert_file_success) { status = APR_SUCCESS; cert_path = ctx->cert_file_success; ctx->cert_file_success = NULL; retrying_success = 1; } else { status = ctx->cert_callback(ctx->cert_userdata, &cert_path); } if (status || !cert_path) { break; } /* Load the x.509 cert file stored in PKCS12 */ status = apr_file_open(&cert_file, cert_path, APR_READ, APR_OS_DEFAULT, ctx->pool); if (status) { continue; } bio = BIO_new(&bio_file_method); bio->ptr = cert_file; ctx->cert_path = cert_path; p12 = d2i_PKCS12_bio(bio, NULL); apr_file_close(cert_file); i = PKCS12_parse(p12, NULL, pkey, cert, NULL); if (i == 1) { PKCS12_free(p12); ctx->cached_cert = *cert; ctx->cached_cert_pw = *pkey; if (!retrying_success && ctx->cert_cache_pool) { const char *c; c = apr_pstrdup(ctx->cert_cache_pool, ctx->cert_path); apr_pool_userdata_setn(c, "serf:ssl:cert", apr_pool_cleanup_null, ctx->cert_cache_pool); } return 1; } else { int err = ERR_get_error(); ERR_clear_error(); if (ERR_GET_LIB(err) == ERR_LIB_PKCS12 && ERR_GET_REASON(err) == PKCS12_R_MAC_VERIFY_FAILURE) { if (ctx->cert_pw_callback) { const char *password; if (ctx->cert_pw_success) { status = APR_SUCCESS; password = ctx->cert_pw_success; ctx->cert_pw_success = NULL; } else { status = ctx->cert_pw_callback(ctx->cert_pw_userdata, ctx->cert_path, &password); } if (!status && password) { i = PKCS12_parse(p12, password, pkey, cert, NULL); if (i == 1) { PKCS12_free(p12); ctx->cached_cert = *cert; ctx->cached_cert_pw = *pkey; if (!retrying_success && ctx->cert_cache_pool) { const char *c; c = apr_pstrdup(ctx->cert_cache_pool, ctx->cert_path); apr_pool_userdata_setn(c, "serf:ssl:cert", apr_pool_cleanup_null, ctx->cert_cache_pool); } if (!retrying_success && ctx->cert_pw_cache_pool) { const char *c; c = apr_pstrdup(ctx->cert_pw_cache_pool, password); apr_pool_userdata_setn(c, "serf:ssl:certpw", apr_pool_cleanup_null, ctx->cert_pw_cache_pool); } return 1; } } } PKCS12_free(p12); return 0; } else { printf("OpenSSL cert error: %d %d %d\n", ERR_GET_LIB(err), ERR_GET_FUNC(err), ERR_GET_REASON(err)); PKCS12_free(p12); } } } return 0; } void serf_ssl_client_cert_provider_set( serf_ssl_context_t *context, serf_ssl_need_client_cert_t callback, void *data, void *cache_pool) { context->cert_callback = callback; context->cert_userdata = data; context->cert_cache_pool = cache_pool; if (context->cert_cache_pool) { apr_pool_userdata_get((void**)&context->cert_file_success, "serf:ssl:cert", cache_pool); } } void serf_ssl_client_cert_password_set( serf_ssl_context_t *context, serf_ssl_need_cert_password_t callback, void *data, void *cache_pool) { context->cert_pw_callback = callback; context->cert_pw_userdata = data; context->cert_pw_cache_pool = cache_pool; if (context->cert_pw_cache_pool) { apr_pool_userdata_get((void**)&context->cert_pw_success, "serf:ssl:certpw", cache_pool); } } void serf_ssl_server_cert_callback_set( serf_ssl_context_t *context, serf_ssl_need_server_cert_t callback, void *data) { context->server_cert_callback = callback; context->server_cert_userdata = data; } void serf_ssl_server_cert_chain_callback_set( serf_ssl_context_t *context, serf_ssl_need_server_cert_t cert_callback, serf_ssl_server_cert_chain_cb_t cert_chain_callback, void *data) { context->server_cert_callback = cert_callback; context->server_cert_chain_callback = cert_chain_callback; context->server_cert_userdata = data; } static serf_ssl_context_t *ssl_init_context(serf_bucket_alloc_t *allocator) { serf_ssl_context_t *ssl_ctx; init_ssl_libraries(); ssl_ctx = serf_bucket_mem_alloc(allocator, sizeof(*ssl_ctx)); ssl_ctx->refcount = 0; ssl_ctx->pool = serf_bucket_allocator_get_pool(allocator); ssl_ctx->allocator = allocator; + /* Use the best possible protocol version, but disable the broken SSLv2/3 */ ssl_ctx->ctx = SSL_CTX_new(SSLv23_client_method()); + SSL_CTX_set_options(ssl_ctx->ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3); SSL_CTX_set_client_cert_cb(ssl_ctx->ctx, ssl_need_client_cert); ssl_ctx->cached_cert = 0; ssl_ctx->cached_cert_pw = 0; ssl_ctx->pending_err = APR_SUCCESS; ssl_ctx->fatal_err = APR_SUCCESS; ssl_ctx->cert_callback = NULL; ssl_ctx->cert_pw_callback = NULL; ssl_ctx->server_cert_callback = NULL; ssl_ctx->server_cert_chain_callback = NULL; SSL_CTX_set_verify(ssl_ctx->ctx, SSL_VERIFY_PEER, validate_server_certificate); SSL_CTX_set_options(ssl_ctx->ctx, SSL_OP_ALL); /* Disable SSL compression by default. */ disable_compression(ssl_ctx); ssl_ctx->ssl = SSL_new(ssl_ctx->ctx); ssl_ctx->bio = BIO_new(&bio_bucket_method); ssl_ctx->bio->ptr = ssl_ctx; SSL_set_bio(ssl_ctx->ssl, ssl_ctx->bio, ssl_ctx->bio); SSL_set_connect_state(ssl_ctx->ssl); SSL_set_app_data(ssl_ctx->ssl, ssl_ctx); #if SSL_VERBOSE SSL_CTX_set_info_callback(ssl_ctx->ctx, apps_ssl_info_callback); #endif ssl_ctx->encrypt.stream = NULL; ssl_ctx->encrypt.stream_next = NULL; ssl_ctx->encrypt.pending = serf_bucket_aggregate_create(allocator); ssl_ctx->encrypt.status = APR_SUCCESS; serf_databuf_init(&ssl_ctx->encrypt.databuf); ssl_ctx->encrypt.databuf.read = ssl_encrypt; ssl_ctx->encrypt.databuf.read_baton = ssl_ctx; ssl_ctx->decrypt.stream = NULL; ssl_ctx->decrypt.pending = serf_bucket_aggregate_create(allocator); ssl_ctx->decrypt.status = APR_SUCCESS; serf_databuf_init(&ssl_ctx->decrypt.databuf); ssl_ctx->decrypt.databuf.read = ssl_decrypt; ssl_ctx->decrypt.databuf.read_baton = ssl_ctx; return ssl_ctx; } static apr_status_t ssl_free_context( serf_ssl_context_t *ssl_ctx) { /* If never had the pending buckets, don't try to free them. */ if (ssl_ctx->decrypt.pending != NULL) { serf_bucket_destroy(ssl_ctx->decrypt.pending); } if (ssl_ctx->encrypt.pending != NULL) { serf_bucket_destroy(ssl_ctx->encrypt.pending); } /* SSL_free implicitly frees the underlying BIO. */ SSL_free(ssl_ctx->ssl); SSL_CTX_free(ssl_ctx->ctx); serf_bucket_mem_free(ssl_ctx->allocator, ssl_ctx); return APR_SUCCESS; } static serf_bucket_t * serf_bucket_ssl_create( serf_ssl_context_t *ssl_ctx, serf_bucket_alloc_t *allocator, const serf_bucket_type_t *type) { ssl_context_t *ctx; ctx = serf_bucket_mem_alloc(allocator, sizeof(*ctx)); if (!ssl_ctx) { ctx->ssl_ctx = ssl_init_context(allocator); } else { ctx->ssl_ctx = ssl_ctx; } ctx->ssl_ctx->refcount++; return serf_bucket_create(type, allocator, ctx); } apr_status_t serf_ssl_set_hostname(serf_ssl_context_t *context, const char * hostname) { #ifdef SSL_set_tlsext_host_name if (SSL_set_tlsext_host_name(context->ssl, hostname) != 1) { ERR_clear_error(); } #endif return APR_SUCCESS; } apr_status_t serf_ssl_use_default_certificates(serf_ssl_context_t *ssl_ctx) { X509_STORE *store = SSL_CTX_get_cert_store(ssl_ctx->ctx); int result = X509_STORE_set_default_paths(store); return result ? APR_SUCCESS : SERF_ERROR_SSL_CERT_FAILED; } apr_status_t serf_ssl_load_cert_file( serf_ssl_certificate_t **cert, const char *file_path, apr_pool_t *pool) { FILE *fp = fopen(file_path, "r"); if (fp) { X509 *ssl_cert = PEM_read_X509(fp, NULL, NULL, NULL); fclose(fp); if (ssl_cert) { *cert = apr_palloc(pool, sizeof(serf_ssl_certificate_t)); (*cert)->ssl_cert = ssl_cert; return APR_SUCCESS; } } return SERF_ERROR_SSL_CERT_FAILED; } apr_status_t serf_ssl_trust_cert( serf_ssl_context_t *ssl_ctx, serf_ssl_certificate_t *cert) { X509_STORE *store = SSL_CTX_get_cert_store(ssl_ctx->ctx); int result = X509_STORE_add_cert(store, cert->ssl_cert); return result ? APR_SUCCESS : SERF_ERROR_SSL_CERT_FAILED; } serf_bucket_t *serf_bucket_ssl_decrypt_create( serf_bucket_t *stream, serf_ssl_context_t *ssl_ctx, serf_bucket_alloc_t *allocator) { serf_bucket_t *bkt; ssl_context_t *ctx; bkt = serf_bucket_ssl_create(ssl_ctx, allocator, &serf_bucket_type_ssl_decrypt); ctx = bkt->data; ctx->databuf = &ctx->ssl_ctx->decrypt.databuf; if (ctx->ssl_ctx->decrypt.stream != NULL) { return NULL; } ctx->ssl_ctx->decrypt.stream = stream; ctx->our_stream = &ctx->ssl_ctx->decrypt.stream; return bkt; } serf_ssl_context_t *serf_bucket_ssl_decrypt_context_get( serf_bucket_t *bucket) { ssl_context_t *ctx = bucket->data; return ctx->ssl_ctx; } serf_bucket_t *serf_bucket_ssl_encrypt_create( serf_bucket_t *stream, serf_ssl_context_t *ssl_ctx, serf_bucket_alloc_t *allocator) { serf_bucket_t *bkt; ssl_context_t *ctx; bkt = serf_bucket_ssl_create(ssl_ctx, allocator, &serf_bucket_type_ssl_encrypt); ctx = bkt->data; ctx->databuf = &ctx->ssl_ctx->encrypt.databuf; ctx->our_stream = &ctx->ssl_ctx->encrypt.stream; if (ctx->ssl_ctx->encrypt.stream == NULL) { serf_bucket_t *tmp = serf_bucket_aggregate_create(stream->allocator); serf_bucket_aggregate_append(tmp, stream); ctx->ssl_ctx->encrypt.stream = tmp; } else { bucket_list_t *new_list; new_list = serf_bucket_mem_alloc(ctx->ssl_ctx->allocator, sizeof(*new_list)); new_list->bucket = stream; new_list->next = NULL; if (ctx->ssl_ctx->encrypt.stream_next == NULL) { ctx->ssl_ctx->encrypt.stream_next = new_list; } else { bucket_list_t *scan = ctx->ssl_ctx->encrypt.stream_next; while (scan->next != NULL) scan = scan->next; scan->next = new_list; } } return bkt; } serf_ssl_context_t *serf_bucket_ssl_encrypt_context_get( serf_bucket_t *bucket) { ssl_context_t *ctx = bucket->data; return ctx->ssl_ctx; } /* Functions to read a serf_ssl_certificate structure. */ /* Takes a counted length string and escapes any NUL bytes so that * it can be used as a C string. NUL bytes are escaped as 3 characters * "\00" (that's a literal backslash). * The returned string is allocated in POOL. */ static char * pstrdup_escape_nul_bytes(const char *buf, int len, apr_pool_t *pool) { int i, nul_count = 0; char *ret; /* First determine if there are any nul bytes in the string. */ for (i = 0; i < len; i++) { if (buf[i] == '\0') nul_count++; } if (nul_count == 0) { /* There aren't so easy case to just copy the string */ ret = apr_pstrdup(pool, buf); } else { /* There are so we have to replace nul bytes with escape codes * Proper length is the length of the original string, plus * 2 times the number of nulls (for two digit hex code for * the value) + the trailing null. */ char *pos; ret = pos = apr_palloc(pool, len + 2 * nul_count + 1); for (i = 0; i < len; i++) { if (buf[i] != '\0') { *(pos++) = buf[i]; } else { *(pos++) = '\\'; *(pos++) = '0'; *(pos++) = '0'; } } *pos = '\0'; } return ret; } /* Creates a hash_table with keys (E, CN, OU, O, L, ST and C). Any NUL bytes in these fields in the certificate will be escaped as \00. */ static apr_hash_t * convert_X509_NAME_to_table(X509_NAME *org, apr_pool_t *pool) { char buf[1024]; int ret; apr_hash_t *tgt = apr_hash_make(pool); ret = X509_NAME_get_text_by_NID(org, NID_commonName, buf, 1024); if (ret != -1) apr_hash_set(tgt, "CN", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); ret = X509_NAME_get_text_by_NID(org, NID_pkcs9_emailAddress, buf, 1024); if (ret != -1) apr_hash_set(tgt, "E", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); ret = X509_NAME_get_text_by_NID(org, NID_organizationalUnitName, buf, 1024); if (ret != -1) apr_hash_set(tgt, "OU", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); ret = X509_NAME_get_text_by_NID(org, NID_organizationName, buf, 1024); if (ret != -1) apr_hash_set(tgt, "O", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); ret = X509_NAME_get_text_by_NID(org, NID_localityName, buf, 1024); if (ret != -1) apr_hash_set(tgt, "L", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); ret = X509_NAME_get_text_by_NID(org, NID_stateOrProvinceName, buf, 1024); if (ret != -1) apr_hash_set(tgt, "ST", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); ret = X509_NAME_get_text_by_NID(org, NID_countryName, buf, 1024); if (ret != -1) apr_hash_set(tgt, "C", APR_HASH_KEY_STRING, pstrdup_escape_nul_bytes(buf, ret, pool)); return tgt; } int serf_ssl_cert_depth(const serf_ssl_certificate_t *cert) { return cert->depth; } apr_hash_t *serf_ssl_cert_issuer( const serf_ssl_certificate_t *cert, apr_pool_t *pool) { X509_NAME *issuer = X509_get_issuer_name(cert->ssl_cert); if (!issuer) return NULL; return convert_X509_NAME_to_table(issuer, pool); } apr_hash_t *serf_ssl_cert_subject( const serf_ssl_certificate_t *cert, apr_pool_t *pool) { X509_NAME *subject = X509_get_subject_name(cert->ssl_cert); if (!subject) return NULL; return convert_X509_NAME_to_table(subject, pool); } apr_hash_t *serf_ssl_cert_certificate( const serf_ssl_certificate_t *cert, apr_pool_t *pool) { apr_hash_t *tgt = apr_hash_make(pool); unsigned int md_size, i; unsigned char md[EVP_MAX_MD_SIZE]; BIO *bio; apr_array_header_t *san_arr; /* sha1 fingerprint */ if (X509_digest(cert->ssl_cert, EVP_sha1(), md, &md_size)) { const char hex[] = "0123456789ABCDEF"; char fingerprint[EVP_MAX_MD_SIZE * 3]; for (i=0; i> 4]; fingerprint[(3*i)+1] = hex[(md[i] & 0x0f)]; fingerprint[(3*i)+2] = ':'; } if (md_size > 0) fingerprint[(3*(md_size-1))+2] = '\0'; else fingerprint[0] = '\0'; apr_hash_set(tgt, "sha1", APR_HASH_KEY_STRING, apr_pstrdup(pool, fingerprint)); } /* set expiry dates */ bio = BIO_new(BIO_s_mem()); if (bio) { ASN1_TIME *notBefore, *notAfter; char buf[256]; memset (buf, 0, sizeof (buf)); notBefore = X509_get_notBefore(cert->ssl_cert); if (ASN1_TIME_print(bio, notBefore)) { BIO_read(bio, buf, 255); apr_hash_set(tgt, "notBefore", APR_HASH_KEY_STRING, apr_pstrdup(pool, buf)); } memset (buf, 0, sizeof (buf)); notAfter = X509_get_notAfter(cert->ssl_cert); if (ASN1_TIME_print(bio, notAfter)) { BIO_read(bio, buf, 255); apr_hash_set(tgt, "notAfter", APR_HASH_KEY_STRING, apr_pstrdup(pool, buf)); } } BIO_free(bio); /* Get subjectAltNames */ if (!get_subject_alt_names(&san_arr, cert->ssl_cert, EscapeNulAndCopy, pool)) apr_hash_set(tgt, "subjectAltName", APR_HASH_KEY_STRING, san_arr); return tgt; } const char *serf_ssl_cert_export( const serf_ssl_certificate_t *cert, apr_pool_t *pool) { char *binary_cert; char *encoded_cert; int len; unsigned char *unused; /* find the length of the DER encoding. */ len = i2d_X509(cert->ssl_cert, NULL); if (len < 0) { return NULL; } binary_cert = apr_palloc(pool, len); unused = (unsigned char *)binary_cert; len = i2d_X509(cert->ssl_cert, &unused); /* unused is incremented */ if (len < 0) { return NULL; } encoded_cert = apr_palloc(pool, apr_base64_encode_len(len)); apr_base64_encode(encoded_cert, binary_cert, len); return encoded_cert; } /* Disables compression for all SSL sessions. */ static void disable_compression(serf_ssl_context_t *ssl_ctx) { #ifdef SSL_OP_NO_COMPRESSION SSL_CTX_set_options(ssl_ctx->ctx, SSL_OP_NO_COMPRESSION); #endif } apr_status_t serf_ssl_use_compression(serf_ssl_context_t *ssl_ctx, int enabled) { if (enabled) { #ifdef SSL_OP_NO_COMPRESSION SSL_clear_options(ssl_ctx->ssl, SSL_OP_NO_COMPRESSION); return APR_SUCCESS; #endif } else { #ifdef SSL_OP_NO_COMPRESSION SSL_set_options(ssl_ctx->ssl, SSL_OP_NO_COMPRESSION); return APR_SUCCESS; #endif } return APR_EGENERAL; } static void serf_ssl_destroy_and_data(serf_bucket_t *bucket) { ssl_context_t *ctx = bucket->data; if (!--ctx->ssl_ctx->refcount) { ssl_free_context(ctx->ssl_ctx); } serf_default_destroy_and_data(bucket); } static void serf_ssl_decrypt_destroy_and_data(serf_bucket_t *bucket) { ssl_context_t *ctx = bucket->data; serf_bucket_destroy(*ctx->our_stream); serf_ssl_destroy_and_data(bucket); } static void serf_ssl_encrypt_destroy_and_data(serf_bucket_t *bucket) { ssl_context_t *ctx = bucket->data; serf_ssl_context_t *ssl_ctx = ctx->ssl_ctx; if (ssl_ctx->encrypt.stream == *ctx->our_stream) { serf_bucket_destroy(*ctx->our_stream); serf_bucket_destroy(ssl_ctx->encrypt.pending); /* Reset our encrypted status and databuf. */ ssl_ctx->encrypt.status = APR_SUCCESS; ssl_ctx->encrypt.databuf.status = APR_SUCCESS; /* Advance to the next stream - if we have one. */ if (ssl_ctx->encrypt.stream_next == NULL) { ssl_ctx->encrypt.stream = NULL; ssl_ctx->encrypt.pending = NULL; } else { bucket_list_t *cur; cur = ssl_ctx->encrypt.stream_next; ssl_ctx->encrypt.stream = cur->bucket; ssl_ctx->encrypt.pending = serf_bucket_aggregate_create(cur->bucket->allocator); ssl_ctx->encrypt.stream_next = cur->next; serf_bucket_mem_free(ssl_ctx->allocator, cur); } } else { /* Ah, darn. We haven't sent this one along yet. */ return; } serf_ssl_destroy_and_data(bucket); } static apr_status_t serf_ssl_read(serf_bucket_t *bucket, apr_size_t requested, const char **data, apr_size_t *len) { ssl_context_t *ctx = bucket->data; return serf_databuf_read(ctx->databuf, requested, data, len); } static apr_status_t serf_ssl_readline(serf_bucket_t *bucket, int acceptable, int *found, const char **data, apr_size_t *len) { ssl_context_t *ctx = bucket->data; return serf_databuf_readline(ctx->databuf, acceptable, found, data, len); } static apr_status_t serf_ssl_peek(serf_bucket_t *bucket, const char **data, apr_size_t *len) { ssl_context_t *ctx = bucket->data; return serf_databuf_peek(ctx->databuf, data, len); } const serf_bucket_type_t serf_bucket_type_ssl_encrypt = { "SSLENCRYPT", serf_ssl_read, serf_ssl_readline, serf_default_read_iovec, serf_default_read_for_sendfile, serf_default_read_bucket, serf_ssl_peek, serf_ssl_encrypt_destroy_and_data, }; const serf_bucket_type_t serf_bucket_type_ssl_decrypt = { "SSLDECRYPT", serf_ssl_read, serf_ssl_readline, serf_default_read_iovec, serf_default_read_for_sendfile, serf_default_read_bucket, serf_ssl_peek, serf_ssl_decrypt_destroy_and_data, }; Index: vendor/serf/dist/serf.h =================================================================== --- vendor/serf/dist/serf.h (revision 286498) +++ vendor/serf/dist/serf.h (revision 286499) @@ -1,1117 +1,1117 @@ /* Copyright 2002-2004 Justin Erenkrantz and Greg Stein * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ #ifndef SERF_H #define SERF_H /** * @file serf.h * @brief Main serf header file */ #include #include #include #include #include #include #include #include #ifdef __cplusplus extern "C" { #endif /* Forward declare some structures */ typedef struct serf_context_t serf_context_t; typedef struct serf_bucket_t serf_bucket_t; typedef struct serf_bucket_type_t serf_bucket_type_t; typedef struct serf_bucket_alloc_t serf_bucket_alloc_t; typedef struct serf_connection_t serf_connection_t; typedef struct serf_listener_t serf_listener_t; typedef struct serf_incoming_t serf_incoming_t; typedef struct serf_incoming_request_t serf_incoming_request_t; typedef struct serf_request_t serf_request_t; /** * @defgroup serf high-level constructs * @ingroup serf * @{ */ /** * Serf-specific error codes */ #define SERF_ERROR_RANGE 100 #define SERF_ERROR_START (APR_OS_START_USERERR + SERF_ERROR_RANGE) /* This code is for when this is the last response on this connection: * i.e. do not send any more requests on this connection or expect * any more responses. */ #define SERF_ERROR_CLOSING (SERF_ERROR_START + 1) /* This code is for when the connection terminated before the request * could be processed on the other side. */ #define SERF_ERROR_REQUEST_LOST (SERF_ERROR_START + 2) /* This code is for when the connection is blocked - we can not proceed * until something happens - generally due to SSL negotiation-like behavior * where a write() is blocked until a read() is processed. */ #define SERF_ERROR_WAIT_CONN (SERF_ERROR_START + 3) /* This code is for when something went wrong during deflating compressed * data e.g. a CRC error. */ #define SERF_ERROR_DECOMPRESSION_FAILED (SERF_ERROR_START + 4) /* This code is for when a response received from a http server is not in * http-compliant syntax. */ #define SERF_ERROR_BAD_HTTP_RESPONSE (SERF_ERROR_START + 5) /* The server sent less data than what was announced. */ #define SERF_ERROR_TRUNCATED_HTTP_RESPONSE (SERF_ERROR_START + 6) /* The proxy server returned an error while setting up the SSL tunnel. */ #define SERF_ERROR_SSLTUNNEL_SETUP_FAILED (SERF_ERROR_START + 7) /* The server unexpectedly closed the connection prematurely. */ #define SERF_ERROR_ABORTED_CONNECTION (SERF_ERROR_START + 8) /* SSL certificates related errors */ #define SERF_ERROR_SSL_CERT_FAILED (SERF_ERROR_START + 70) /* SSL communications related errors */ #define SERF_ERROR_SSL_COMM_FAILED (SERF_ERROR_START + 71) /* General authentication related errors */ #define SERF_ERROR_AUTHN_FAILED (SERF_ERROR_START + 90) /* None of the available authn mechanisms for the request are supported */ #define SERF_ERROR_AUTHN_NOT_SUPPORTED (SERF_ERROR_START + 91) /* Authn was requested by the server but the header lacked some attribute */ #define SERF_ERROR_AUTHN_MISSING_ATTRIBUTE (SERF_ERROR_START + 92) /* Authentication handler initialization related errors */ #define SERF_ERROR_AUTHN_INITALIZATION_FAILED (SERF_ERROR_START + 93) /* Error code reserved for use in the test suite. */ #define SERF_ERROR_ISSUE_IN_TESTSUITE (SERF_ERROR_START + 99) /* This macro groups errors potentially raised when reading a http response. */ #define SERF_BAD_RESPONSE_ERROR(status) ((status) \ && ((SERF_ERROR_DECOMPRESSION_FAILED == (status)) \ ||(SERF_ERROR_BAD_HTTP_RESPONSE == (status)) \ ||(SERF_ERROR_TRUNCATED_HTTP_RESPONSE == (status)))) /** * Return a string that describes the specified error code. * * If the error code is not one of the above Serf error codes, then * NULL will be returned. * * Note regarding lifetime: the string is a statically-allocated constant */ const char *serf_error_string(apr_status_t errcode); /** * Create a new context for serf operations. * * A serf context defines a control loop which processes multiple * connections simultaneously. * * The context will be allocated within @a pool. */ serf_context_t *serf_context_create( apr_pool_t *pool); /** * Callback function. Add a socket to the externally managed poll set. * * Both @a pfd and @a serf_baton should be used when calling serf_event_trigger * later. */ typedef apr_status_t (*serf_socket_add_t)( void *user_baton, apr_pollfd_t *pfd, void *serf_baton); /** * Callback function. Remove the socket, identified by both @a pfd and * @a serf_baton from the externally managed poll set. */ typedef apr_status_t (*serf_socket_remove_t)( void *user_baton, apr_pollfd_t *pfd, void *serf_baton); /* Create a new context for serf operations. * * Use this function to make serf not use its internal control loop, but * instead rely on an external event loop. Serf will use the @a addf and @a rmf * callbacks to notify of any event on a connection. The @a user_baton will be * passed through the addf and rmf callbacks. * * The context will be allocated within @a pool. */ serf_context_t *serf_context_create_ex( void *user_baton, serf_socket_add_t addf, serf_socket_remove_t rmf, apr_pool_t *pool); /** * Make serf process events on a connection, identified by both @a pfd and * @a serf_baton. * * Any outbound data is delivered, and incoming data is made available to * the associated response handlers and their buckets. * * If any data is processed (incoming or outgoing), then this function will * return with APR_SUCCESS. */ apr_status_t serf_event_trigger( serf_context_t *s, void *serf_baton, const apr_pollfd_t *pfd); /** @see serf_context_run should not block at all. */ #define SERF_DURATION_NOBLOCK 0 /** @see serf_context_run should run for (nearly) "forever". */ #define SERF_DURATION_FOREVER 2000000000 /* approx 1^31 */ /** * Run the main networking control loop. * * The set of connections defined by the serf context @a ctx are processed. * Any outbound data is delivered, and incoming data is made available to * the associated response handlers and their buckets. This function will * block on the network for no longer than @a duration microseconds. * * If any data is processed (incoming or outgoing), then this function will * return with APR_SUCCESS. Typically, the caller will just want to call it * again to continue processing data. * * If no activity occurs within the specified timeout duration, then * APR_TIMEUP is returned. * * All temporary allocations will be made in @a pool. */ apr_status_t serf_context_run( serf_context_t *ctx, apr_short_interval_time_t duration, apr_pool_t *pool); apr_status_t serf_context_prerun( serf_context_t *ctx); /** * Callback function for progress information. @a progress indicates cumulative * number of bytes read or written, for the whole context. */ typedef void (*serf_progress_t)( void *progress_baton, apr_off_t read, apr_off_t write); /** * Sets the progress callback function. @a progress_func will be called every * time bytes are read of or written on a socket. */ void serf_context_set_progress_cb( serf_context_t *ctx, const serf_progress_t progress_func, void *progress_baton); /** @} */ /** * @defgroup serf connections and requests * @ingroup serf * @{ */ /** * When a connection is established, the application needs to wrap some * buckets around @a skt to enable serf to process incoming responses. This * is the control point for assembling connection-level processing logic * around the given socket. * * The @a setup_baton is the baton established at connection creation time. * * This callback corresponds to reading from the server. Since this is an * on-demand activity, we use a callback. The corresponding write operation * is based on the @see serf_request_deliver function, where the application * can assemble the appropriate bucket(s) before delivery. * * The returned bucket should live at least as long as the connection itself. * It is assumed that an appropriate allocator is passed in @a setup_baton. * ### we may want to create a connection-level allocator and pass that * ### along. however, that allocator would *only* be used for this * ### callback. it may be wasteful to create a per-conn allocator, so this * ### baton-based, app-responsible form might be best. * * Responsibility for the buckets is passed to the serf library. They will be * destroyed when the connection is closed. * * All temporary allocations should be made in @a pool. */ typedef apr_status_t (*serf_connection_setup_t)( apr_socket_t *skt, serf_bucket_t **read_bkt, serf_bucket_t **write_bkt, void *setup_baton, apr_pool_t *pool); /** * ### need to update docco w.r.t socket. became "stream" recently. * ### the stream does not have a barrier, this callback should generally * ### add a barrier around the stream before incorporating it into a * ### response bucket stack. * ### should serf add the barrier automatically to protect its data * ### structure? i.e. the passed bucket becomes owned rather than * ### borrowed. that might suit overall semantics better. * Accept an incoming response for @a request, and its @a socket. A bucket * for the response should be constructed and returned. This is the control * point for assembling the appropriate wrapper buckets around the socket to * enable processing of the incoming response. * * The @a acceptor_baton is the baton provided when the specified request * was created. * * The request's pool and bucket allocator should be used for any allocations * that need to live for the duration of the response. Care should be taken * to bound the amount of memory stored in this pool -- to ensure that * allocations are not proportional to the amount of data in the response. * * Responsibility for the bucket is passed to the serf library. It will be * destroyed when the response has been fully read (the bucket returns an * APR_EOF status from its read functions). * * All temporary allocations should be made in @a pool. */ /* ### do we need to return an error? */ typedef serf_bucket_t * (*serf_response_acceptor_t)( serf_request_t *request, serf_bucket_t *stream, void *acceptor_baton, apr_pool_t *pool); /** * Notification callback for when a connection closes. * * This callback is used to inform an application that the @a conn * connection has been (abnormally) closed. The @a closed_baton is the * baton provided when the connection was first opened. The reason for * closure is given in @a why, and will be APR_SUCCESS if the application * requested closure (by clearing the pool used to allocate this * connection or calling serf_connection_close). * * All temporary allocations should be made in @a pool. */ typedef void (*serf_connection_closed_t)( serf_connection_t *conn, void *closed_baton, apr_status_t why, apr_pool_t *pool); /** * Response data has arrived and should be processed. * * Whenever response data for @a request arrives (initially, or continued data * arrival), this handler is invoked. The response data is available in the * @a response bucket. The @a handler_baton is passed along from the baton * provided by the request setup callback (@see serf_request_setup_t). * * The handler MUST process data from the @a response bucket until the * bucket's read function states it would block (see APR_STATUS_IS_EAGAIN). * The handler is invoked only when new data arrives. If no further data * arrives, and the handler does not process all available data, then the * system can result in a deadlock around the unprocessed, but read, data. * * The handler should return APR_EOF when the response has been fully read. * If calling the handler again would block, APR_EAGAIN should be returned. * If the handler should be invoked again, simply return APR_SUCCESS. * * Note: if the connection closed (at the request of the application, or * because of an (abnormal) termination) while a request is being delivered, * or before a response arrives, then @a response will be NULL. This is the * signal that the request was not delivered properly, and no further * response should be expected (this callback will not be invoked again). * If a request is injected into the connection (during this callback's * execution, or otherwise), then the connection will be reopened. * * All temporary allocations should be made in @a pool. */ typedef apr_status_t (*serf_response_handler_t)( serf_request_t *request, serf_bucket_t *response, void *handler_baton, apr_pool_t *pool); /** * Callback function to be implemented by the application, so that serf * can handle server and proxy authentication. * code = 401 (server) or 407 (proxy). * baton = the baton passed to serf_context_run. * authn_type = one of "Basic", "Digest". */ typedef apr_status_t (*serf_credentials_callback_t)( char **username, char **password, serf_request_t *request, void *baton, int code, const char *authn_type, const char *realm, apr_pool_t *pool); /** * Create a new connection associated with the @a ctx serf context. * * If no proxy server is configured, a connection will be created to * (eventually) connect to the address specified by @a address. The address must * live at least as long as @a pool (thus, as long as the connection object). * If a proxy server is configured, @address will be ignored. * * The connection object will be allocated within @a pool. Clearing or * destroying this pool will close the connection, and terminate any * outstanding requests or responses. * * When the connection is closed (upon request or because of an error), * then the @a closed callback is invoked, and @a closed_baton is passed. * * ### doc on setup(_baton). tweak below comment re: acceptor. * NULL may be passed for @a acceptor and @a closed; default implementations * will be used. * * Note: the connection is not made immediately. It will be opened on * the next call to @see serf_context_run. */ serf_connection_t *serf_connection_create( serf_context_t *ctx, apr_sockaddr_t *address, serf_connection_setup_t setup, void *setup_baton, serf_connection_closed_t closed, void *closed_baton, apr_pool_t *pool); /** * Create a new connection associated with the @a ctx serf context. * * A connection will be created to (eventually) connect to the address * specified by @a address. The address must live at least as long as * @a pool (thus, as long as the connection object). * * The host address will be looked up based on the hostname in @a host_info. * * The connection object will be allocated within @a pool. Clearing or * destroying this pool will close the connection, and terminate any * outstanding requests or responses. * * When the connection is closed (upon request or because of an error), * then the @a closed callback is invoked, and @a closed_baton is passed. * * ### doc on setup(_baton). tweak below comment re: acceptor. * NULL may be passed for @a acceptor and @a closed; default implementations * will be used. * * Note: the connection is not made immediately. It will be opened on * the next call to @see serf_context_run. */ apr_status_t serf_connection_create2( serf_connection_t **conn, serf_context_t *ctx, apr_uri_t host_info, serf_connection_setup_t setup, void *setup_baton, serf_connection_closed_t closed, void *closed_baton, apr_pool_t *pool); typedef apr_status_t (*serf_accept_client_t)( serf_context_t *ctx, serf_listener_t *l, void *accept_baton, apr_socket_t *insock, apr_pool_t *pool); apr_status_t serf_listener_create( serf_listener_t **listener, serf_context_t *ctx, const char *host, apr_uint16_t port, void *accept_baton, serf_accept_client_t accept_func, apr_pool_t *pool); typedef apr_status_t (*serf_incoming_request_cb_t)( serf_context_t *ctx, serf_incoming_request_t *req, void *request_baton, apr_pool_t *pool); apr_status_t serf_incoming_create( serf_incoming_t **client, serf_context_t *ctx, apr_socket_t *insock, void *request_baton, serf_incoming_request_cb_t request, apr_pool_t *pool); /** * Reset the connection, but re-open the socket again. */ apr_status_t serf_connection_reset( serf_connection_t *conn); /** * Close the connection associated with @a conn and cancel all pending requests. * * The closed callback passed to serf_connection_create() will be invoked * with APR_SUCCESS. */ apr_status_t serf_connection_close( serf_connection_t *conn); /** * Sets the maximum number of outstanding requests @a max_requests on the * connection @a conn. Setting max_requests to 0 means unlimited (the default). * Ex.: setting max_requests to 1 means a request is sent when a response on the * previous request was received and handled. * * In general, serf tends to take around 16KB per outstanding request. */ void serf_connection_set_max_outstanding_requests( serf_connection_t *conn, unsigned int max_requests); void serf_connection_set_async_responses( serf_connection_t *conn, serf_response_acceptor_t acceptor, void *acceptor_baton, serf_response_handler_t handler, void *handler_baton); /** * Setup the @a request for delivery on its connection. * * Right before this is invoked, @a pool will be built within the * connection's pool for the request to use. The associated response will * be allocated within that subpool. An associated bucket allocator will * be built. These items may be fetched from the request object through * @see serf_request_get_pool or @see serf_request_get_alloc. * * The content of the request is specified by the @a req_bkt bucket. When * a response arrives, the @a acceptor callback will be invoked (along with * the @a acceptor_baton) to produce a response bucket. That bucket will then * be passed to @a handler, along with the @a handler_baton. * * The responsibility for the request bucket is passed to the request * object. When the request is done with the bucket, it will be destroyed. */ typedef apr_status_t (*serf_request_setup_t)( serf_request_t *request, void *setup_baton, serf_bucket_t **req_bkt, serf_response_acceptor_t *acceptor, void **acceptor_baton, serf_response_handler_t *handler, void **handler_baton, apr_pool_t *pool); /** * Construct a request object for the @a conn connection. * * When it is time to deliver the request, the @a setup callback will * be invoked with the @a setup_baton passed into it to complete the * construction of the request object. * * If the request has not (yet) been delivered, then it may be canceled * with @see serf_request_cancel. * * Invoking any calls other than @see serf_request_cancel before the setup * callback executes is not supported. */ serf_request_t *serf_connection_request_create( serf_connection_t *conn, serf_request_setup_t setup, void *setup_baton); /** * Construct a request object for the @a conn connection, add it in the * list as the next to-be-written request before all unwritten requests. * * When it is time to deliver the request, the @a setup callback will * be invoked with the @a setup_baton passed into it to complete the * construction of the request object. * * If the request has not (yet) been delivered, then it may be canceled * with @see serf_request_cancel. * * Invoking any calls other than @see serf_request_cancel before the setup * callback executes is not supported. */ serf_request_t *serf_connection_priority_request_create( serf_connection_t *conn, serf_request_setup_t setup, void *setup_baton); /** Returns detected network latency for the @a conn connection. Negative * value means that latency is unknwon. */ apr_interval_time_t serf_connection_get_latency(serf_connection_t *conn); /** Check if a @a request has been completely written. * * Returns APR_SUCCESS if the request was written completely on the connection. * Returns APR_EBUSY if the request is not yet or partially written. */ apr_status_t serf_request_is_written( serf_request_t *request); /** * Cancel the request specified by the @a request object. * * If the request has been scheduled for delivery, then its response * handler will be run, passing NULL for the response bucket. * * If the request has already been (partially or fully) delivered, then * APR_EBUSY is returned and the request is *NOT* canceled. To properly * cancel the request, the connection must be closed (by clearing or * destroying its associated pool). */ apr_status_t serf_request_cancel( serf_request_t *request); /** * Return the pool associated with @a request. * * WARNING: be very careful about the kinds of things placed into this * pool. In particular, all allocation should be bounded in size, rather * than proportional to any data stream. */ apr_pool_t *serf_request_get_pool( const serf_request_t *request); /** * Return the bucket allocator associated with @a request. */ serf_bucket_alloc_t *serf_request_get_alloc( const serf_request_t *request); /** * Return the connection associated with @a request. */ serf_connection_t *serf_request_get_conn( const serf_request_t *request); /** * Update the @a handler and @a handler_baton for this @a request. * * This can be called after the request has started processing - * subsequent data will be delivered to this new handler. */ void serf_request_set_handler( serf_request_t *request, const serf_response_handler_t handler, const void **handler_baton); /** * Configure proxy server settings, to be used by all connections associated * with the @a ctx serf context. * * The next connection will be created to connect to the proxy server * specified by @a address. The address must live at least as long as the * serf context. */ void serf_config_proxy( serf_context_t *ctx, apr_sockaddr_t *address); /* Supported authentication types. */ #define SERF_AUTHN_NONE 0x00 #define SERF_AUTHN_BASIC 0x01 #define SERF_AUTHN_DIGEST 0x02 #define SERF_AUTHN_NTLM 0x04 #define SERF_AUTHN_NEGOTIATE 0x08 #define SERF_AUTHN_ALL 0xFF /** * Define the authentication handlers that serf will try on incoming requests. */ void serf_config_authn_types( serf_context_t *ctx, int authn_types); /** * Set the credentials callback handler. */ void serf_config_credentials_callback( serf_context_t *ctx, serf_credentials_callback_t cred_cb); /* ### maybe some connection control functions for flood? */ /*** Special bucket creation functions ***/ /** * Create a bucket of type 'socket bucket'. * This is basically a wrapper around @a serf_bucket_socket_create, which * initializes the bucket using connection and/or context specific settings. */ serf_bucket_t *serf_context_bucket_socket_create( serf_context_t *ctx, apr_socket_t *skt, serf_bucket_alloc_t *allocator); /** * Create a bucket of type 'request bucket'. * This is basically a wrapper around @a serf_bucket_request_create, which * initializes the bucket using request, connection and/or context specific * settings. * * This function will set following header(s): * - Host: if the connection was created with @a serf_connection_create2. */ serf_bucket_t *serf_request_bucket_request_create( serf_request_t *request, const char *method, const char *uri, serf_bucket_t *body, serf_bucket_alloc_t *allocator); /** @} */ /** * @defgroup serf buckets * @ingroup serf * @{ */ /** Pass as REQUESTED to the read function of a bucket to read, consume, * and return all available data. */ #define SERF_READ_ALL_AVAIL ((apr_size_t)-1) /** Acceptable newline types for bucket->readline(). */ #define SERF_NEWLINE_CR 0x0001 #define SERF_NEWLINE_CRLF 0x0002 #define SERF_NEWLINE_LF 0x0004 #define SERF_NEWLINE_ANY 0x0007 /** Used to indicate that a newline is not present in the data buffer. */ /* ### should we make this zero? */ #define SERF_NEWLINE_NONE 0x0008 /** Used to indicate that a CR was found at the end of a buffer, and CRLF * was acceptable. It may be that the LF is present, but it needs to be * read first. * * Note: an alternative to using this symbol would be for callers to see * the SERF_NEWLINE_CR return value, and know that some "end of buffer" was * reached. While this works well for @see serf_util_readline, it does not * necessary work as well for buckets (there is no obvious "end of buffer", * although there is an "end of bucket"). The other problem with that * alternative is that developers might miss the condition. This symbol * calls out the possibility and ensures that callers will watch for it. */ #define SERF_NEWLINE_CRLF_SPLIT 0x0010 struct serf_bucket_type_t { /** name of this bucket type */ const char *name; /** * Read (and consume) up to @a requested bytes from @a bucket. * * A pointer to the data will be returned in @a data, and its length * is specified by @a len. * * The data will exist until one of two conditions occur: * * 1) this bucket is destroyed * 2) another call to any read function or to peek() * * If an application needs the data to exist for a longer duration, * then it must make a copy. */ apr_status_t (*read)(serf_bucket_t *bucket, apr_size_t requested, const char **data, apr_size_t *len); /** * Read (and consume) a line of data from @a bucket. * * The acceptable forms of a newline are given by @a acceptable, and * the type found is returned in @a found. If a newline is not present * in the returned data, then SERF_NEWLINE_NONE is stored into @a found. * * A pointer to the data is returned in @a data, and its length is * specified by @a len. The data will include the newline, if present. * * Note that there is no way to limit the amount of data returned * by this function. * * The lifetime of the data is the same as that of the @see read * function above. */ apr_status_t (*readline)(serf_bucket_t *bucket, int acceptable, int *found, const char **data, apr_size_t *len); /** * Read a set of pointer/length pairs from the bucket. * * The size of the @a vecs array is specified by @a vecs_size. The * bucket should fill in elements of the array, and return the number * used in @a vecs_used. * * Each element of @a vecs should specify a pointer to a block of * data and a length of that data. * * The total length of all data elements should not exceed the * amount specified in @a requested. * * The lifetime of the data is the same as that of the @see read * function above. */ apr_status_t (*read_iovec)(serf_bucket_t *bucket, apr_size_t requested, int vecs_size, struct iovec *vecs, int *vecs_used); /** * Read data from the bucket in a form suitable for apr_socket_sendfile() * * On input, hdtr->numheaders and hdtr->numtrailers specify the size * of the hdtr->headers and hdtr->trailers arrays, respectively. The * bucket should fill in the headers and trailers, up to the specified * limits, and set numheaders and numtrailers to the number of iovecs * filled in for each item. * * @a file should be filled in with a file that can be read. If a file * is not available or appropriate, then NULL should be stored. The * file offset for the data should be stored in @a offset, and the * length of that data should be stored in @a len. If a file is not * returned, then @a offset and @a len should be ignored. * * The file position is not required to correspond to @a offset, and * the caller may manipulate it at will. * * The total length of all data elements, and the portion of the * file should not exceed the amount specified in @a requested. * * The lifetime of the data is the same as that of the @see read * function above. */ apr_status_t (*read_for_sendfile)(serf_bucket_t *bucket, apr_size_t requested, apr_hdtr_t *hdtr, apr_file_t **file, apr_off_t *offset, apr_size_t *len); /** * Look within @a bucket for a bucket of the given @a type. The bucket * must be the "initial" data because it will be consumed by this * function. If the given bucket type is available, then read and consume * it, and return it to the caller. * * This function is usually used by readers that have custom handling * for specific bucket types (e.g. looking for a file bucket to pass * to apr_socket_sendfile). * * If a bucket of the given type is not found, then NULL is returned. * * The returned bucket becomes the responsibility of the caller. When * the caller is done with the bucket, it should be destroyed. */ serf_bucket_t * (*read_bucket)(serf_bucket_t *bucket, const serf_bucket_type_t *type); /** * Peek, but don't consume, the data in @a bucket. * * Since this function is non-destructive, the implicit read size is * SERF_READ_ALL_AVAIL. The caller can then use whatever amount is * appropriate. * * The @a data parameter will point to the data, and @a len will * specify how much data is available. The lifetime of the data follows * the same rules as the @see read function above. * * Note: if the peek does not return enough data for your particular * use, then you must read/consume some first, then peek again. * * If the returned data represents all available data, then APR_EOF * will be returned. Since this function does not consume data, it * can return the same data repeatedly rather than blocking; thus, * APR_EAGAIN will never be returned. */ apr_status_t (*peek)(serf_bucket_t *bucket, const char **data, apr_size_t *len); /** * Destroy @a bucket, along with any associated resources. */ void (*destroy)(serf_bucket_t *bucket); /* ### apr buckets have 'copy', 'split', and 'setaside' functions. ### not sure whether those will be needed in this bucket model. */ }; /** * Should the use and lifecycle of buckets be tracked? * * When tracking, the system will ensure several semantic requirements * of bucket use: * * - if a bucket returns APR_EAGAIN, one of its read functions should * not be called immediately. the context's run loop should be called. * ### and for APR_EOF, too? * - all buckets must be drained of input before returning to the * context's run loop. * - buckets should not be destroyed before they return APR_EOF unless * the connection is closed for some reason. * * Undefine this symbol to avoid the tracking (and a performance gain). * * ### we may want to examine when/how we provide this. should it always * ### be compiled in? and apps select it before including this header? */ /* #define SERF_DEBUG_BUCKET_USE */ /* Internal macros for tracking bucket use. */ #ifdef SERF_DEBUG_BUCKET_USE #define SERF__RECREAD(b,s) serf_debug__record_read(b,s) #else #define SERF__RECREAD(b,s) (s) #endif #define serf_bucket_read(b,r,d,l) SERF__RECREAD(b, (b)->type->read(b,r,d,l)) #define serf_bucket_readline(b,a,f,d,l) \ SERF__RECREAD(b, (b)->type->readline(b,a,f,d,l)) #define serf_bucket_read_iovec(b,r,s,v,u) \ SERF__RECREAD(b, (b)->type->read_iovec(b,r,s,v,u)) #define serf_bucket_read_for_sendfile(b,r,h,f,o,l) \ SERF__RECREAD(b, (b)->type->read_for_sendfile(b,r,h,f,o,l)) #define serf_bucket_read_bucket(b,t) ((b)->type->read_bucket(b,t)) #define serf_bucket_peek(b,d,l) ((b)->type->peek(b,d,l)) #define serf_bucket_destroy(b) ((b)->type->destroy(b)) /** * Check whether a real error occurred. Note that bucket read functions * can return EOF and EAGAIN as part of their "normal" operation, so they * should not be considered an error. */ #define SERF_BUCKET_READ_ERROR(status) ((status) \ && !APR_STATUS_IS_EOF(status) \ && !APR_STATUS_IS_EAGAIN(status) \ && (SERF_ERROR_WAIT_CONN != status)) struct serf_bucket_t { /** the type of this bucket */ const serf_bucket_type_t *type; /** bucket-private data */ void *data; /** the allocator used for this bucket (needed at destroy time) */ serf_bucket_alloc_t *allocator; }; /** * Generic macro to construct "is TYPE" macros. */ #define SERF_BUCKET_CHECK(b, btype) ((b)->type == &serf_bucket_type_ ## btype) /** * Notification callback for a block that was not returned to the bucket * allocator when its pool was destroyed. * * The block of memory is given by @a block. The baton provided when the * allocator was constructed is passed as @a unfreed_baton. */ typedef void (*serf_unfreed_func_t)( void *unfreed_baton, void *block); /** * Create a new allocator for buckets. * * All buckets are associated with a serf bucket allocator. This allocator * will be created within @a pool and will be destroyed when that pool is * cleared or destroyed. * * When the allocator is destroyed, if any allocations were not explicitly * returned (by calling serf_bucket_mem_free), then the @a unfreed callback * will be invoked for each block. @a unfreed_baton will be passed to the * callback. * * If @a unfreed is NULL, then the library will invoke the abort() stdlib * call. Any failure to return memory is a bug in the application, and an * abort can assist with determining what kinds of memory were not freed. */ serf_bucket_alloc_t *serf_bucket_allocator_create( apr_pool_t *pool, serf_unfreed_func_t unfreed, void *unfreed_baton); /** * Return the pool that was used for this @a allocator. * * WARNING: the use of this pool for allocations requires a very * detailed understanding of pool behaviors, the bucket system, * and knowledge of the bucket's use within the overall pattern * of request/response behavior. * * See design-guide.txt for more information about pool usage. */ apr_pool_t *serf_bucket_allocator_get_pool( const serf_bucket_alloc_t *allocator); /** * Utility structure for reading a complete line of input from a bucket. * * Since it is entirely possible for a line to be broken by APR_EAGAIN, * this structure can be used to accumulate the data until a complete line * has been read from a bucket. */ /* This limit applies to the line buffer functions. If an application needs * longer lines, then they will need to manually handle line buffering. */ #define SERF_LINEBUF_LIMIT 8000 typedef struct { /* Current state of the buffer. */ enum { SERF_LINEBUF_EMPTY, SERF_LINEBUF_READY, SERF_LINEBUF_PARTIAL, SERF_LINEBUF_CRLF_SPLIT } state; /* How much of the buffer have we used? */ apr_size_t used; /* The line is read into this buffer, minus CR/LF */ char line[SERF_LINEBUF_LIMIT]; } serf_linebuf_t; /** * Initialize the @a linebuf structure. */ void serf_linebuf_init(serf_linebuf_t *linebuf); /** * Fetch a line of text from @a bucket, accumulating the line into * @a linebuf. @a acceptable specifies the types of newlines which are * acceptable for this fetch. * * ### we should return a data/len pair so that we can avoid a copy, * ### rather than having callers look into our state and line buffer. */ apr_status_t serf_linebuf_fetch( serf_linebuf_t *linebuf, serf_bucket_t *bucket, int acceptable); /** @} */ /* Internal functions for bucket use and lifecycle tracking */ apr_status_t serf_debug__record_read( const serf_bucket_t *bucket, apr_status_t status); void serf_debug__entered_loop( serf_bucket_alloc_t *allocator); void serf_debug__closed_conn( serf_bucket_alloc_t *allocator); void serf_debug__bucket_destroy( const serf_bucket_t *bucket); void serf_debug__bucket_alloc_check( serf_bucket_alloc_t *allocator); /* Version info */ #define SERF_MAJOR_VERSION 1 #define SERF_MINOR_VERSION 3 -#define SERF_PATCH_VERSION 7 +#define SERF_PATCH_VERSION 8 /* Version number string */ #define SERF_VERSION_STRING APR_STRINGIFY(SERF_MAJOR_VERSION) "." \ APR_STRINGIFY(SERF_MINOR_VERSION) "." \ APR_STRINGIFY(SERF_PATCH_VERSION) /** * Check at compile time if the Serf version is at least a certain * level. * @param major The major version component of the version checked * for (e.g., the "1" of "1.3.0"). * @param minor The minor version component of the version checked * for (e.g., the "3" of "1.3.0"). * @param patch The patch level component of the version checked * for (e.g., the "0" of "1.3.0"). */ #define SERF_VERSION_AT_LEAST(major,minor,patch) \ (((major) < SERF_MAJOR_VERSION) \ || ((major) == SERF_MAJOR_VERSION && (minor) < SERF_MINOR_VERSION) \ || ((major) == SERF_MAJOR_VERSION && (minor) == SERF_MINOR_VERSION && \ (patch) <= SERF_PATCH_VERSION)) /** * Returns the version of the library the application has linked/loaded. * Values are returned in @a major, @a minor, and @a patch. * * Applications will want to use this function to verify compatibility, * expecially while serf has not reached a 1.0 milestone. APIs and * semantics may change drastically until the library hits 1.0. */ void serf_lib_version( int *major, int *minor, int *patch); #ifdef __cplusplus } #endif /* * Every user of serf will want to deal with our various bucket types. * Go ahead and include that header right now. * * Note: make sure this occurs outside of the C++ namespace block */ #include "serf_bucket_types.h" #endif /* !SERF_H */