For those of you who have been following my blog, or my work on the SSL Labs web site, it won't come as a surprise that my next book is about SSL/TLS and PKI, and all the related things you need to know if you want to use these technologies.

Bulletproof SSL/TLS and PKI is the book I wish I had back when I was starting to get involved with SSL. I don't remember when exactly that was, but I do remember how, when I was writing my first book, Apache Security, I began to appreciate cryptography. I began to like it, even. Before, SSL seemed simple to me; but then I realised how vast the world of cryptography actually is.

In 2009 I began to work on SSL Labs, and, for me, that world began to unravel. Fast-forward a couple of years, and in 2013 I still feel like I am only starting. Cryptography is a unique field where the more you learn the less you know.

In supporting the SSL Labs users over the years, I've realised that there is a lot of documentation on SSL/TLS and PKI, but that it suffers from two problems: (1) it's not documented in one place, and so the little bits and pieces (e.g., RFCs) are difficult to find and (2) it tends to be very detailed and low-level. I needed years of effort to begin to understand the everything fits together.

Bulletproof SSL/TLS and PKI aims to address the documentation gap, offering a very practical book that paints the whole picture and then proceeds to discuss the bits and pieces that you need in daily work, going as deep as needed to explain what you need to know.

At this point, the manuscript is about 120 pages long, which I estimate to be between 30% and 50% of the finished book. I expect the book will be available in Autumn. Until then, I have a little treat for you; if you head over to the Feisty Duck web site, I am realeasing the main OpenSSL chapter as a standalone free ebook called OpenSSL Cookbook. It's about 50 pages at the moment, but I hope that grows longer over time. Qualys has graciously allowed me to include the SSL/TLS Deployment Best Practices document in the appendix.

RC4 has long been considered problematic, but until very recently there was no known way to exploit the weaknesses. After the BEAST attack was disclosed in 2011, we—grudgingly—started using RC4 in order to avoid the vulnerable CBC suites in TLS 1.0 and earlier. This caused the usage of RC4 to increase, and some say that it now accounts for about 50% of all TLS traffic.

Last week, a group of researchers (Nadhem AlFardan, Dan Bernstein, Kenny Paterson, Bertram Poettering and Jacob Schuldt) announced significant advancements in the attacks against RC4, unveiling new weaknesses as well as new methods to exploit them. Matthew Green has a great overview on his blog, and here are the slides from the talk where the new issues were announced.

At the moment, the attack is not yet practical because it requires access to millions and possibly billions of copies of the same data encrypted using different keys. A browser would have to make that many connections to a server to give the attacker enough data. A possible exploitation path is to somehow instrument the browser to make a large number of connections, while a man in the middle is observing and recording the traffic.

We are still safe at the moment, but there is a tremendous incentive for researchers to improve the attacks on RC4, which means that we need to act swiftly.

What We (SSL Labs) Will Do

• Start warning our users about RC4 weaknesses. RC4 is demonstrably broken and unsafe to use in TLS as currently implemented. The difficulty is that, for public web sites that need to support a wide user base, there is practically nothing 100% secure they can use to replace RC4. We now have no choice but to accept that, no matter what settings we use, some segment of the user base will be at risk.
• If Apple were to implement 1/n-1 record splitting in their stacks (the only major browser vendor that hasn’t done that yet*), we’d likely consider BEAST sufficiently mitigated client-side, and that would allow us to start recommending CBC suites over RC4.
• Start recommending the use of GCM suites. Browsers will no doubt provide better support for TLS 1.2 and GCM suites at an accelerated schedule, and site operators should be ready to take advantage of that.
• Update SSL/TLS Deployment Best Practices with new information.
• At some point in the near future, update the rating algorithm to take the RC4 weaknesses into account.

Recommendations

SSL/TLS library developers

• Harden the stack against the Lucky 13 attack.
• Support TLS 1.2 and GCM suites as soon as possible.

Browser vendors

• Support TLS 1.2 and GCM suites as soon as possible.
• Implement 1/n-1 record splitting to make CBC suites safe in TLS 1.0 and earlier. As far as we are aware, Apple is the only remaining vendor that has not patched their browsers, either on OSX or iOS.

System administrators

• Disable TLS compression. This attack is similar in nature to the recent RC4 attacks, but practical.
• Support TLS 1.2 and GCM as soon as possible.

TLS Working Group

• Restore algorithm agility and diversity in TLS. AES GCM suites are now the only truly secure option in TLS, but we shouldn’t count on them to stay like that forever.
• Consider introducing other stream ciphers to the standard. Algorithm agility, which TLS already provides, is not sufficient if there is only one choice for a component.
• Consider changing how CBC is implemented in order to address the timing issues.

Application developers

• Harden session management to support reliable and frequent rotation of session cookies, triggered by elapsed time or the number of requests observed. Recent years have seen a rise in attacks that require attackers to control the client end of a TLS connection in some fashion. Most such attacks focus on extracting small bits of information, typically credentials. Session cookies are now the most popular target. Given how many requests are needed for the best attacks to succeed, rotating session cookies frequently is a good defense in depth measure.

---

(*) Apple appears to have shipped the 1/n-1 split in Mountain Lion, but it is disabled by default. I was also unable to observe any splitting after the functionality had been manually enabled. iOS devices should support TLS 1.2, which means that they might be secure provided TLS 1.2 is available server-side.

The SSL threat landscape has changed significantly since the rating guide was first published, back in 2009. The original text is thus no longer entirely adequate to deal with the threats of today. In addition, we now have several years of experience using the criteria in real life, and we know what works well and what not so much.

Our plan is to update the rating criteria to take all this new knowledge into account. However, because that process will take several months to complete and because an update to the guide is long overdue, we have decided to release a small patch release, labelling it 2009c. The year in the version number indicates that this is conceptually the same guide as previously published, with improvements.

Essentially, the purpose of this release is to keep the rating criteria relevant for a little while longer, until the next major version is ready. The text of the original document remains as is, but the following changes apply:

• SSL 2.0 is not allowed (F).
• Insecure renegotiation is not allowed (F).
• Vulnerability to the BEAST attack caps the grade to B.
• Vulnerability to the CRIME attack caps the grade to B.
• The score (0-100) is not shown any more, in order to focus on the grade, which is more useful. Future versions of the guide will probably remove the score altogether.

In addition, we've taken the opportunity to remove the old configuration advice, directing the readers to our SSL/TLS Deployment Best Practices document instead.

The International Computer Science Institute (ICSI) recently announced a new certificate notary project. Although I find the notary aspect of the project interesting, I think that their work is much more important because of the potential of their approach, which is based on large-scale passive monitoring of SSL connections. In the last couple of years I spent a lot of time looking at the server-side aspect of SSL (most recently, in the SSL Pulse project), but there's a large gap in our understanding when it comes to client-side capabilities, and actual real-life usage. The gap can be explained by ICSI's data.

Sadly, ICSI is unable to share their data because of various privacy concerns. However, they can still mine the data themselves. I decided to put together a list of unanswered SSL usage questions, as a way of contributing to the effort.

Focusing on client capabilities is a good way to start. Some of the most interesting attributes to track might be:

• Highest TLS version
• Cipher suites (and their order)
• TLS extensions
• Compression methods
• BEAST mitigation
• SNI (virtual SSL hosting)
• Secure renegotiation (via extension or SCSV)
• Session ticket support
• Request for OCSP stapling
• SPDY support (via the NPN extension)

The most exciting aspect of the project is that it might be able to finally give us the answer to the effective security of SSL. Some key data points include:

• Negotiated protocol version
• Negotiated cipher suite/strength
• Was the suite actively selected by the server
• Authentication strength (certificate key size)
• DH params/strength (for applicable cipher suites)
• Compression usage
• Session resumption status (tracked via either mechanism)
• Seen stapled OCSP response
• Server certificate information
  • Key size
  • Signature algorithm
  • Is it expired?
  • Is it self-signed?
• Certificate chain correctness
• Is the trust root a well-known CA?
• Vulnerabilities:
  • Renegotiation (do both sides support secure renegotiation)
  • BEAST (no browser mitigation and a CBC suite using TLS 1.0 or earlier)
  • CRIME (use of TLS compression for the connection)

I've had some available time recently to work on sslhaf, my passive SSL fingerprinting tool. I made the following improvements:

• Detect browser BEAST mitigation (1/n-1 splitting)
• Extract compression support
• Extract TLS extensions
• Change licence from GPLv2 to BSD
• Bug fixes

I think that the detection of BEAST mitigation measures is particularly useful, because it will help understand the risk coming from this attack. Most, but not all, major browsers have mitigations in place. In addition, we don't know how many vulnerable older clients there are. I hope to publish some measurements soon.

The tool is stable but technically still experimental, so use at your own risk. I'd be willing to make it production ready if there's anyone interested in deploying it in a large web site.

It seems that it is that time of year again, when Juliano and Thai present their most recent attack against crypto system. Last year, it was BEAST. This year, it's CRIME, a practical attack against how TLS is used in browsers. In a wider sense, the same attack conceptually applies to any encrypted protocol where the attacker controls what is being communicated.

Initially, it was only known that the attack builds on top of an information leakage weakness, and the full results were going to be revealed at the talk at Ekoparty on September 21. Funnily enough, the details that were revealed themselves "leaked" and were sufficient for the experts to understand what is going on: On StackExchange, Thomas Pornin speculated that it was about compression. An academic paper from 2002 (PDF) was revealed. A proof of concept was submitted by xorninja, and improved by Krzysztof Kotowicz. Dan Goodin wrote a great summary of what was known at the time, and included a video of the demonstration. Finally, Threat Post published a confirmation from Juliano and Thai that was indeed compression.

What is the problem?

The root cause of the problem is information leakage that occurs when data is compressed prior to encryption. If someone can repeatedly inject and mix arbitrary content with some sensitive and relatively predictable data, and observe the resulting encrypted stream, then she will be able to extract the unknown data from it.

In practice, the attack works best against session cookies. If a man-in-the-middle (MITM) attacker 1) can observe network traffic, and 2) manipulate the victim's browser to submit requests to the target site, she can, as a result, steal the site's cookies, and thus hijack the victim's session. In the current form, the exploit uses JavaScript and needs 6 requests to extract one byte of data. The use of JavaScript is desired, but not required: simple <img> tags can do the job just as well, although with a performance penalty.

How bad is it?

Several aspects need to come together for CRIME to be widely exploited. First of all, there has to be server-side support for compression of request data before encryption. TLS supports DEFLATE compression (not to be confused with HTTP response compression, which is very popular, but not vulnerable to CRIME), but not all servers implement it. SSL Labs tests across the SSL Pulse data set indicate that about 42% of the servers support TLS compression. The servers include some of the most popular sites in the world.

Another possibility is that the newer protocol, SPDY, is also vulnerable, because it has a separate mechanism for request header compression. In that case, all servers that support SPDY would be potentially vulnerable. SPDY is a young protocol and not very widely supported, but the sites that do support it are typically larger properties. SSL Labs is currently half way in measuring the support for SPDY across the SSL Pulse data set, but we're currently seeing SPDY support at 0.82%.

The second requirement is that client-side also supports compression, and here we will find that the situation is much better. The only browser to properly support TLS compression is Chrome. As for SPDY, both Chrome and Firefox support it. However, it is understood that both Chrome and Firefox have removed support for TLS compression in their most recent updates. (For Firefox, it's even not clear if it was ever enabled.) SSL Labs runs a passive SSL fingerprinting tool called sslhaf. Using it we were able to estimate that only about 7% of browsers support compression at this point. This is on our site, which gets a higher portion of the traffic from Chrome and Firefox users. Analysing the logs, we could see traces of Chrome, and, it seems, some mobile browsers. It is not clear if there were any changes to the SPDY implementation in Chrome and Firefox, but it is reasonable to expect that there will be.

The third requirement for wide exploitation is an easy to use exploitation tool. Unlike BEAST, CRIME seems much easier to exploit; we've already seen a simple proof of concept, and so it's likely that we will see a complete tool soon, maybe as a fork of sslstrip.

Taking all of the above in the account, CRIME will be easy to exploit, but it will be harder to find vulnerable clients. Internet Explorer, Safari, and Opera do not seem to be affected. Chrome and Firefox seem to be, but they use automatic updates, so I expect that the majority of the user base will upgrade to the patched versions. The vulnerable clients will thus be restricted to those using older clients.

Further, unlike BEAST, which requires a manual intervention to mitigate, CRIME will be easier to patch. I expect most vendors will simply disable TLS compression.

What to do?

First of all, if you're using an older Chrome or Firefox, upgrade to the most recent version. If you're operating a web site, use the SSL Labs assessment tool to determine if your site supports TLS compression and SPDY (look for Compression and Next Protocol Support on the results page, towards the bottom). If you think the risk is too high, disable compression if your web software allows you to do it. (If they don't today, they will soon.)

Update (15 Sep 2012) In Details on the "CRIME" attack, iSec Partners give a very good overview of which browsers are vulnerable, and how to disable compression server side.

Web application firewalls have come a long way from their modest beginnings more than a decade ago. They are now an accepted security best practice and have a significant role in compliance. But there is still a lot left to do before they can unlock their full potential.

There is one aspect in particular that interests me a great deal, and that is the ability of end users to verify the operation of WAFs and measure their technical quality. Understandably, vendors are reluctant to talk about the weaknesses in their products. However, understanding the weak points is critical for effective deployments. We cannot claim to have achieved any level of security otherwise. As always with these things, we should assume that our adversaries already know about those weaknesses; but how can we know too? Simple, by forcing the issue out in the open.

Today at Black Hat we (Qualys) are announcing a new research project on protocol-level evasion of web application firewalls. This type of evasion focuses on the low level operation of WAFs, aiming to exploit little differences in how WAFs see traffic and how backend web servers and applications see it. If you get the WAF to see something different from what the backend is seeing, you have an evasion opportunity that could possibly be used to execute any attack type, without detection.

I spent a great deal of effort on protocol-level evasion in my years of working on ModSecurity (an open source web application firewall I started in 2002, and worked on until 2009). I imagine all WAF manufacturers spend a lot of effort in this area, yet this topic is seldom discussed in public. It is our aim to change this. Our focus on protocol-level evasion is part of our work on IronBee, a new open source web application firewall we are building at Qualys.

Attached to this post is our research paper that focuses on request path, parameter, and multipart/form-data evasion. Also attached are the Black Hat talk slides that introduce the research. The testing suite (a sort of a research toolkit) is in the IronBee WAF Research repository on GitHub.

• Protocol-Level Evasion of Web Application Firewalls v1.1 (18 July 2012).pdf (254.4 K)
• Protocol-Level Evasion of Web Application Firewalls (Ivan Ristic, Qualys, Black Hat USA 2012) SLIDES.pdf (1.7 MB)

When it comes to the mitigation of the BEAST attack against SSL/TLS, the usual advice is to prioritize RC4 cipher suites in order to avoid CBC suites, which are vulnerable. In some cases, companies may even have to use RC4 suites exclusively. For example, I've recently heard of a case where a company was failed on its PCI compliance test because they had non-RC4 cipher suites enabled in their configuration.

When RC4 prioritization is discussed, people often seek assurance that RC4 is widely supported. Obviously, if you are going to disable a large number of cipher suites, you start to worry if the remaining suites are going to be sufficient for your user base. Until recently, I had no answer to those questions. Even though all major browsers support RC4 by default, there was no way to quantify the support. Further, some browsers allow users to turn off certain cipher suites, and we just didn't know what the situation on the ground was.

About two months ago I re-enabled mod_sslhaf on the SSL Labs web server. This Apache module, a project of SSL Labs, records all cipher suites offered by clients. Today I looked at the results, curious to find out exactly how many of the SSL Labs clients supported RC4. I choose to compare the total number of unique IP addresses against the number of IP addresses that did not support TLS_RSA_WITH_RC4_128_MD5 (0x04) or TLS_RSA_WITH_RC4_128_SHA (0x05).

In about two months, SSL Labs saw 48,481 unique IP addresses. Only 45 of those -- or 0.09% -- did not support RC4. Looking at the user agent information, the clients not supporting RC4 seem to be largely those whose users decided to explicitly disable RC4 for one reason or another.

The usual disclaimer applies here: the sample is taken from the SSL Labs web site and may not apply to other sites. However, if the assumption that the only non RC4 clients are those where this cipher suite was disabled by their owners, you could argue that an average site will see an even smaller number of non-compatible clients.

I am looking for a Lead Application Security Researcher to join the WAF team at Qualys, and work with us to build the best application security monitoring, detection, and protection product (our cloud-based web application firewall). We want someone who knows the application security field inside-out, and has tons of practical experience with the right mix of offense and defense skills.

The focus of this role is to own, design, and implement the security aspects of our product. This is a rare opportunity to break new ground, work in an exciting field, focus on application security research, and work with great people. We want someone who is passionate and focused, works independently and leaves no stone unturned. Above all, we want someone who will deliver.

To make things even more interesting, we've been building our core engine and libraries as open source from day one. See them now at https://github.com/ironbee/. We expect our open source and community involvement to intensify over time. There's also ample opportunity for writing articles, research papers, and presenting at conferences.

This is a full time position in Redwood Shores, CA or Madison, WI. For the candidate that is just right, I am willing to consider other options.

Skill checklist:

1. You know how the Internet works (networking, protocols, etc)
2. You know how web applications are built (and may have built some yourself)
3. You know how to build secure web applications (the emphasis here is on understanding how to avoid vulnerabilities; strong programming experience is not required, but it's useful if you are able to do some scripting to build a research lab and automate testing).
4. You know how browsers work and interact with applications
5. You know application security and can use your skills to attack vulnerable applications
6. You can evade web application firewalls
7. You understand exactly why your attacks and evasion techniques work
8. You can use all of the above to build a better mouse trap

If you fit the above description we would love to talk to you, employ you, and give you what you need to make you successful in our environment. Please contact me directly (the email is iristic at qualys.com) with your resume, examples of the work you are proud of, and a short cover letter. No agencies, please.

During our research of web application firewall evasion issues, we uncovered a flaw in ModSecurity that may lead to complete bypass of the installed rules, in the cases when ModSecurity is deployed to protect the backends where impedance mismatch is not mitigated. Additionally, a separate flaw in ModSecurity CRS makes the content type checks ineffective, allowing for bypass attacks, when deployed to protect the backends where impedance mismatch is not mitigated.

ModSecurity is a popular and widely deployed open source web application firewall engine. By design, ModSecurity does not include any security logic. The recommended configuration contains only a couple of rules that are tightly coupled with the performance of the engine itself. A comprehensive set of rules for ModSecurity can be obtained from a separate project called ModSecurity Core Rules.

One of the main goals of web application firewalls (WAFs) is to detect attacks against the web applications they are protecting. In the most commonly used deployment modes (e.g., when operating as a reverse proxy), a WAF will terminate the higher layers of the traffic stream (e.g., the HTTP protocol), but only inspect and pass-through the remainder of the data. In the latter case, WAFs are vulnerable to impedance mismatch issues, where they interpret traffic in one way and the backend application interprets it in another way. When an impedance mismatch issue exists, the WAF may be vulnerable to evasion attacks.

Impedance mismatch issues are possibly the most difficult aspect of WAF implementation. In many cases, correct deployment requires not only correct implementation in the WAF, but correct configuration and handling of the reported problems by administrators. Our hope is that this short document, as well as our future research in this area, will shed some light onto this rarely discussed topic of WAF design. We would be delighted if this information is used to raise the effective security in real-life WAF deployments.

Vulnerable versions

• Confirmed in ModSecurity 2.6.5; earlier versions likely to be vulnerable
• Confirmed in ModSecurity Core Rule Set 2.2.4; earlier versions likely to be vulnerable

Users are advised to upgrade to ModSecurity 2.6.6 and ModSecurity Core Rule Set 2.2.5, which are thought to fix the issues documented here. Further, those who are not deploying the CRS should check that in their configuration they have rules that check REQBODY_ERROR and MULTIPART_STRICT_ERROR, configured to block requests that have either of these flags set. For guidance on how to write such rules refer to the recommended default configuration file, which is included with ModSecurity.>

Problem #1: Multipart bypass in ModSecurity with PHP in the backend

A mismatch between how multipart content is parsed in ModSecurity and PHP enables an attacker to perform a full rule set bypass.

PHP has a very lax multipart/form-data parser. Traditionally, securing ModSecurity against evasion in this parser is where a lot of development time was spent. In 2009, Stefan Esser published an evasion technique that relies on the use of single quotes—which are supported by PHP but were not supported by ModSecurity at the time—to trick ModSecurity into treating a request parameter as a file. This results in a bypass because ModSecurity uses separate mechanisms for the inspection of request parameter and file content, and files are often left uninspected.

This issue was addressed in November 2009 in ModSecurity 2.5.11, which started to accept single quotes for quoting. However, upon further examination of the PHP source code, we determined that the fix was not sufficient. PHP will not only allow a single quote to be used at the beginning of a string, but also at any other position within the string. ModSecurity, on the other hand, expects quote characters only at the first position. With some creativity, the impedance mismatch issue can be exploited to perform a bypass of the rules.

The issue was confirmed against PHP 5.4.3, but it's very likely that earlier versions can be used too. We are not releasing a proof of concept at this time, but the vulnerability is easy to exploit.

The users of ModSecurity CRS may be protected from this attack, depending on the exact deployment configuration. After the original issue had been reported, a defence-in-depth rule was added to CRS to detect side effects of a bypass attempt. This rule is effective when CRS is deployed in the traditional blocking mode, but not when anomaly scoring mode is used.

This issue should be addressed in ModSecurity's multipart parser. In addition, we recommend the following:

• Short term, improve the recommended default configuration to include the same defense in depth rule as the CRS.
• Long term, implement full request body rewriting. If the multipart payloads are fully rewritten according to how ModSecurity understands them, then any missed attack payloads will not be passed through to the backend. Such approach may require more processing, but we do not believe that this improvement would cause any practical performance issues because multipart content types are infrequent on average.

Problem #2: Content type bypass in ModSecurity CRS with certain backends

When the ModSecurity CRS is used to protect certain permissive backend applications, supplying an invalid content type can be used for a complete bypass.

To address unknown content type bypass issues, ModSecurity CRS employs rules that allow only known content types to be used. However, these rules are not strict enough. By default, the CRS will check if the MIME type can be found within the following string (all one continuous line):

application/x-www-form-urlencoded multipart/form-data text/xml application/xml application/x-amf

This check will indeed reject many unknown and invalid MIME types, but it will also accept any substrings that can be found within the above string. In most cases, such invalid MIME types can be used only against a small number of applications. The only situation where this can be exploited is when attacking applications that expect only certain MIME types known to them (e.g., application/x-www-form-urlencoded) and don't check what actual MIME type is indicated in the Content-Type request header.

The attack was confirmed against Apache Commons FileUpload 1.2.1, but earlier versions are equally likely to contribute to the bypass. Starting with the Servlet 3.0 specification, file uploads are supported natively, without the need to use a separate library. The Tomcat web server bundles the FileUpload library to implement file uploads, so even applications that do not explicitly use upload libraries may be vulnerable. The problem likely affects other web servers that are built on the Tomcat code base. Outside Java, at least one other server-side framework is thought to be vulnerable to the same problem.

The attack was confirmed against Apache Commons FileUpload 1.2.1, but earlier versions are equally likely to contribute to the bypass in this way.

Credit

The new vulnerabilities discussed here were discovered by Ivan Ristic, from Qualys Vulnerability & Malware Research Labs (VMRL).

Notices

• Ivan Ristic is the original author of ModSecurity. He started the project in 2002, and led it until January 2009. His last code contribution was in 2010. He remains involved through his ongoing work on ModSecurity Handbook, which is the definitive guide on ModSecurity.
• At Qualys, Ivan Ristic is part of a team working on IronBee, which is also an open source web application firewall.
• ModSecurity and mod_security are trademarks or registered trademarks of Trustwave Holdings, Inc.

Ivan Ristić is an entrepreneur, software engineer, writer, and application security researcher.

Apache  Apache Security  BEAST  Bulletproof SSL/TLS and PKI  Business  Canoe  Computer Misuse Act  DocBook  Feisty Duck  Firefox  Humour  IronBee  IronBee Blog  Lua  ModSecurity  ModSecurity Blog  ModSecurity Handbook  MySQL  Open Source  OpenSSL Cookbook  PCI  Qualys  Qualys Security Labs  RC4  SSL  Secure programming  Security  UK  Web Application Firewalls  Writing  XSS