Bulletproof TLS Newsletter #28
Let's Encrypt downtime
31 May 2017
Author: Hanno Böck

This issue was distributed to 37,126 email subscribers.

Bulletproof TLS Newsletter is a free periodic newsletter bringing you commentary and news surrounding SSL/TLS and Internet PKI, designed to keep you informed about the latest developments in this space.

In this issue:

  1. Let's Encrypt downtime
  2. Symantec dispute nearing an end
  3. Short news

Let's Encrypt downtime

On May 18th and 19th several services operated by Let's Encrypt were unavailable. This most notably affected ACME and the OCSP server.

According to a postmortem by Josh Aas from Let's Encrypt, the reason was a change in the handling of slashes in URLs. OCSP requests are base64 encoded, and base64 contains slashes as one of its encoding characters. This subsequently led to an error in the caching of the CDN and thus an overload of the systems.

The incident shed light on some of the problems that arise when a certificate authority goes offline for short periods of time. Ideally, protocols and applications should be robust enough to handle such a situation, but in reality this is often not the case. The author of this newsletter summarized the poor state of OCSP stapling implementations in two major web servers, Apache and Nginx. Neither is capable of handling OCSP outages properly.

The Caddy web server wouldn't start if the ACME server was unavailable, which led to a longer discussion in a bug report. Subsequently, Caddy changed its behavior. Caddy developer Matt Holt described the issue in a blog post.

Symantec dispute nearing an end

After several incidents in which Symantec issued illegitimate certificates, Google planned to take some harsh actions against the certificate authority, as we reported in our March newsletter. A detailed overview of the issues can be found in the Mozilla Wiki.

It seems now that Google and Symantec may come to an agreement with significantly less impact for existing Symantec customers. The core of the agreement is that Symantec will create new certificate roots that will be cross-signed by their existing ones and another certificate authority. Therefore, over the long term, the trust in the existing roots can be removed. Statements from Mozilla representatives indicate that they'll agree to the proposed plan, so Mozilla and Chrome will align their actions. Vincent Lynch has posted a summary at the SSL Store blog.

Short news