The OpenSSL encryption flaw, known as the Heartbleed bug, is being called one of the biggest security flaws ever seen on the Internet. One security analyst called it “catastrophic” and said that on a scale of 1 to 10, the vulnerability was an 11. The newly discovered vulnerability isn’t “big news” because of its complexity, but for the fact that the amazingly simple bug existed in an extremely popular SSL/TLS implementation for two years before anyone noticed, allowing millions of servers to remain vulnerable and open to hacker attacks.
What Heartbleed does
Heartbleed allows a hacker to retrieve chunks of the server memory – up to 64K at a time. What is returned to the hacker is somewhat random but there is virtually no trace of the attack so the hacker can simply run the exploit over and over, accumulating 64K chunks of server memory with each pass which can then be analyzed, and mined, offline. The hacker grabs the server memory simply by sending a malformed heartbeat request to the server (technical details below) which in return, will serve up a memory dump to the hacker.
What’s at risk
The Heartbleed vulnerability allows attackers to snag large chunks of data from the server’s allocated memory within the OpenSSL process. Yeah, OpenSSL – the part that encrypts the network stream. That means the data that can be snagged from the server’s memory is likely the most important stuff – passwords, credit card numbers, bank account numbers, confidential email, etc. This alone makes it a pretty ugly vulnerability.
OpenSSL is typically used on Unix/Linux servers, primarily Apache and nginx web servers. This particular vulnerability affects OpenSSL version 1.0.1 through 1.01.f. OpenSSL is used beyond web server processes though – Email is one that example that’s also widespread but SSL/TLS is also used in instant messaging (IM) and virtual private networks (VPNs). In short, the Heartbleed bug exists on the majority of external-facing servers on the Internet.
Malware and security analyst Mark Loman demonstrated the vulnerability by running an attack against Yahoo mail servers. As you can see in the screenshots, the memory served back through the bug included email accounts and passwords. It’s a very easily-exploited vulnerability and as stated, the data returned is often critically sensitive information.
How do you fix the problem?
Patching the server to remove the Heartbleed vulnerability is only the first step. Since the vulnerable server has likely ran the exploitable code for years, there’s no telling what critically sensitive data has been mined by attackers and this includes the server’s private key (which can be used to decrypt the encrypted streams sent to and from the server), TSL session keys, passwords, and session ticket keys. As such, it is recommended that after the server is patched, the X.509 certificates be replaced (revoke the old ones, create a new private key, deal with the Certificate Authority, etc.) and all users and administrators change their passwords.
Did the NSA left us hanging?
When word of the Heartbleed vulnerability hit the streets, the news was coupled with another potential revelation – that the NSA may have been using the bug to its advantage. I don’t think many will argue that the NSA scandal has been disconcerting. But if the NSA used the Heartbleed vulnerability to its advantage, it demonstrates that they knew about the bug and did not inform the American public about it. Rather than participate in the protection of the citizens, they left us vulnerable to attack. That makes news of the Heartbleed vulnerability even more disturbing.
Wired Magazine voiced their suspicions:
“Security experts have speculated about whether the NSA cracked SSL communications and if so how the agency might have accomplished the feat. Now, Heartbleed raises the possibility that in some cases the NSA might not have needed to crack SSL. Instead, it’s possible the agency used the vulnerability to obtain the private keys of companies to decrypt their traffic.”
Who was vulnerable to Heartbleed?
Practically every major website operator that runs *Nix systems was vulnerable at some point. The following websites are known to have been vulnerable to Heartbleed. This is the list that will scare the crap out of you.
- Tumblr
- Yahoo
- Gmail
- Yahoo Mail
- GoDaddy
- Intuit Turbo Tax
- Dropbox
- Minecraft
- OkCupid
Heartbleed itself it mind-blowingly simple
The ugly details
The Heartbeat Extension is a TLS/DTLS protocol that provides keep-alive functionality allowing the communicating parties to continue without renegotiation. The bug is basically a memory read overrun allowed because the code did not do a missing bounds check. Without the bounds check, an attacker can request that a TLS server hand over a large slice (up to 64KB) of its private memory space.
In easy-to-understand terms, the hacker tells the server they are sending 64K worth of data in the payload but instead, they only send 1K worth of data in the payload. The server thinks it received 64K in the payload (because the hacker said so – yeah, the server doesn’t bother to check), and so it echoes back 64K worth of data to the hacker (in what is essentially a ping response) – that includes 1K of the payload the hacker really sent and 63K worth of system process memory to pad out the rest of the request.
Breakdown of Heartbleed OpenSSl code
Let’s break it down and start with this bit of Heartbeat code:
/* Read type and payload length first */
hbtype = *p++;
n2s(p, payload);
pl = p;
In the first line, a byte of the SSL record is read into hbtype, the heartbeat type. Line 2 uses a macro, n2s(), which reads two bytes from p and places them into payload. These two bytes are the length of the payload. Finally, in line 3, the payload (p) is copied to pl. At this point, we have the payload type stored in hbtype, the payload length stored in payload, and the payload itself stored in pl.
Here’s where things start to get ugly.
unsigned char *buffer, *bp;
int r;
/* Allocate memory for the response, size is 1 byte
* message type, plus 2 bytes payload length, plus
* payload, plus padding
*/
buffer = OPENSSL_malloc(1 + 2 + payload + padding);
bp = buffer;
In this code, buffer space has been allocated in buffer. The amount of space allocated through the OPENSSL_malloc() call includes 1 byte for the message type, 2 bytes for the payload length, the payload itself, and a bit of padding. All of this is moved into bp.
And here’s the Heartbleed vulnerability itself.
/* Enter response type, length and copy payload */
*bp++ = TLS1_HB_RESPONSE;
s2n(payload, bp);
memcpy(bp, pl, payload);
The s2n() macro is similar to the n2s() one seen above except it does its thing in reverse. It packs a two-byte field with the length of the payload. The memcpy() function is the key to the exploit. pl in the function call is a pointer to the payload that the hacker supplied. payload indicates the length, or how much memory should be copied. Notice that payload is trusted without a bounds check. If the hacker said 64KB should be copied, that’s what gets copied. bp is basically the recipient of the copy function and will contain the payload (more accurately, a pointer to it).
The problem with this line arises when the hacker *tells* the server that it sent 64K worth of payload but instead, sends a lesser amount (let’s say 1K as an example). If only 1K worth of data was sent in the payload and the memcpy() function thinks it’s supposed to copy 64K worth of data, the extra 63K comes from, you guessed it, an overrun into the processes memory (basically whatever memory happens to lie close to the SSL record within the process).
Here’s the code in its entirety:
2412 /* Read type and payload length first */
2413 hbtype = *p++;
2414 n2s(p, payload);
2415 pl = p;
2416
2417 if (s->msg_callback)
2418 s->msg_callback(0, s->version, TLS1_RT_HEARTBEAT,
2419 &s->s3->rrec.data[0], s->s3->rrec.length,
2420 s, s->msg_callback_arg);
2421
2422 if (hbtype == TLS1_HB_REQUEST)
2423 {
2424 unsigned char *buffer, *bp;
2425 int r;
2426
2427 /* Allocate memory for the response, size is 1 bytes
2428 * message type, plus 2 bytes payload length, plus
2429 * payload, plus padding
2430 */
2431 buffer = OPENSSL_malloc(1 + 2 + payload + padding);
2432 bp = buffer;
2433
2434 /* Enter response type, length and copy payload */
2435 *bp++ = TLS1_HB_RESPONSE;
2436 s2n(payload, bp);
2437 memcpy(bp, pl, payload);
2438
2439 r = ssl3_write_bytes(s, TLS1_RT_HEARTBEAT, buffer, 3 + payload + padding);
Hey, C is a great language, one that you have pretty much total control to do whatever you want. That same power however, makes it easy to make a critical mistake like this. In no way do I blame the OpenSSL programmers – I’ve probably made many mistakes like this in my code too. It’s very easy to do when your thought processes are focused on the application’s *logic* and not necessarily on the discrete fundamentals of the code.
The patch for the code, seen below, is pretty simple. It merely adds a bounds check that validates the length of the payload and makes sure nothing fishy is going on.
/* Read type and payload length first */
if (1 + 2 + 16 > s->s3->rrec.length)
return 0; /* silently discard */
hbtype = *p++;
n2s(p, payload);
if (1 + 2 + payload + 16 > s->s3->rrec.length)
return 0; /* silently discard per RFC 6520 sec. 4 */
pl = p;
Additional information about Heartbleed
You can test to see if your server is vulnerable here: http://filippo.io/Heartbleed/
Below is a FAQ supplied by heartbleed.com:
Heartbleed Q&A
What is the CVE-2014-0160?
CVE-2014-0160 is the official reference to this bug. CVE (Common Vulnerabilities and Exposures) is the Standard for Information Security Vulnerability Names maintained by MITRE. Due to co-incident discovery a duplicate CVE, CVE-2014-0346, which was assigned to us, should not be used, since others independently went public with the CVE-2014-0160 identifier.
Why it is called the Heartbleed Bug?
Bug is in the OpenSSL’s implementation of the TLS/DTLS (transport layer security protocols) heartbeat extension (RFC6520). When it is exploited it leads to the leak of memory contents from the server to the client and from the client to the server.
What makes the Heartbleed Bug unique?
Bugs in single software or library come and go and are fixed by new versions. However this bug has left large amount of private keys and other secrets exposed to the Internet. Considering the long exposure, ease of exploitation and attacks leaving no trace this exposure should be taken seriously.
Is this a design flaw in SSL/TLS protocol specification?
No. This is implementation problem, i.e. programming mistake in popular OpenSSL library that provides cryptographic services such as SSL/TLS to the applications and services.
What is being leaked?
Encryption is used to protect secrets that may harm your privacy or security if they leak. In order to coordinate recovery from this bug we have classified the compromised secrets to four categories: 1) primary key material, 2) secondary key material and 3) protected content and 4) collateral.
What is leaked primary key material and how to recover?
These are the crown jewels, the encryption keys themselves. Leaked secret keys allows the attacker to decrypt any past and future traffic to the protected services and to impersonate the service at will. Any protection given by the encryption and the signatures in the X.509 certificates can be bypassed. Recovery from this leak requires patching the vulnerability, revocation of the compromised keys and reissuing and redistributing new keys. Even doing all this will still leave any traffic intercepted by the attacker in the past still vulnerable to decryption. All this has to be done by the owners of the services.
What is leaked secondary key material and how to recover?
These are for example the user credentials (user names and passwords) used in the vulnerable services. Recovery from this leaks requires owners of the service first to restore trust to the service according to steps described above. After this users can start changing their passwords and possible encryption keys according to the instructions from the owners of the services that have been compromised. All session keys and session cookies should be invalided and considered compromised.
What is leaked protected content and how to recover?
This is the actual content handled by the vulnerable services. It may be personal or financial details, private communication such as emails or instant messages, documents or anything seen worth protecting by encryption. Only owners of the services will be able to estimate the likelihood what has been leaked and they should notify their users accordingly. Most important thing is to restore trust to the primary and secondary key material as described above. Only this enables safe use of the compromised services in the future.
What is leaked collateral and how to recover?
Leaked collateral are other details that have been exposed to the attacker in the leaked memory content. These may contain technical details such as memory addresses and security measures such as canaries used to protect against overflow attacks. These have only contemporary value and will lose their value to the attacker when OpenSSL has been upgraded to a fixed version.
Recovery sounds laborious, is there a short cut?
After seeing what we saw by “attacking” ourselves, with ease, we decided to take this very seriously. We have gone laboriously through patching our own critical services and are in progress of dealing with possible compromise of our primary and secondary key material. All this just in case we were not first ones to discover this and this could have been exploited in the wild already.
How revocation and reissuing of certificates works in practice?
If you are a service provider you have signed your certificates with a Certificate Authority (CA). You need to check your CA how compromised keys can be revoked and new certificate reissued for the new keys. Some CAs do this for free, some may take a fee.
Am I affected by the bug?
You are likely to be affected either directly or indirectly. OpenSSL is the most popular open source cryptographic library and TLS (transport layer security) implementation used to encrypt traffic on the Internet. Your popular social site, your company’s site, commerce site, hobby site, site you install software from or even sites run by your government might be using vulnerable OpenSSL. Many of online services use TLS to both to identify themselves to you and to protect your privacy and transactions. You might have networked appliances with logins secured by this buggy implementation of the TLS. Furthermore you might have client side software on your computer that could expose the data from your computer if you connect to compromised services.
How widespread is this?
Most notable software using OpenSSL are the open source web servers like Apache and nginx. The combined market share of just those two out of the active sites on the Internet was over 66% according to Netcraft’s April 2014 Web Server Survey. Furthermore OpenSSL is used to protect for example email servers (SMTP, POP and IMAP protocols), chat servers (XMPP protocol), virtual private networks (SSL VPNs), network appliances and wide variety of client side software. Fortunately many large consumer sites are saved by their conservative choice of SSL/TLS termination equipment and software. Ironically smaller and more progressive services or those who have upgraded to latest and best encryption will be affected most. Furthermore OpenSSL is very popular in client software and somewhat popular in networked appliances which have most inertia in getting updates.
What versions of the OpenSSL are affected?
Status of different versions:
OpenSSL 1.0.1 through 1.0.1f (inclusive) are vulnerable
OpenSSL 1.0.1g is NOT vulnerable
OpenSSL 1.0.0 branch is NOT vulnerable
OpenSSL 0.9.8 branch is NOT vulnerable
Bug was introduced to OpenSSL in December 2011 and has been out in the wild since OpenSSL release 1.0.1 on 14th of March 2012. OpenSSL 1.0.1g released on 7th of April 2014 fixes the bug.
How common are the vulnerable OpenSSL versions?
The vulnerable versions have been out there for over two years now and they have been rapidly adopted by modern operating systems. A major contributing factor has been that TLS versions 1.1 and 1.2 came available with the first vulnerable OpenSSL version (1.0.1) and security community has been pushing the TLS 1.2 due to earlier attacks against TLS (such as the BEAST).
How about operating systems?
Some operating system distributions that have shipped with potentially vulnerable OpenSSL version:
Debian Wheezy (stable), OpenSSL 1.0.1e-2+deb7u4
Ubuntu 12.04.4 LTS, OpenSSL 1.0.1-4ubuntu5.11
CentOS 6.5, OpenSSL 1.0.1e-15
Fedora 18, OpenSSL 1.0.1e-4
OpenBSD 5.3 (OpenSSL 1.0.1c 10 May 2012) and 5.4 (OpenSSL 1.0.1c 10 May 2012)
FreeBSD 10.0 – OpenSSL 1.0.1e 11 Feb 2013
NetBSD 5.0.2 (OpenSSL 1.0.1e)
OpenSUSE 12.2 (OpenSSL 1.0.1c)
Operating system distribution with versions that are not vulnerable:
Debian Squeeze (oldstable), OpenSSL 0.9.8o-4squeeze14
SUSE Linux Enterprise Server
FreeBSD 8.4 – OpenSSL 0.9.8y 5 Feb 2013
FreeBSD 9.2 – OpenSSL 0.9.8y 5 Feb 2013
FreeBSD Ports – OpenSSL 1.0.1g (At 7 Apr 21:46:40 2014 UTC)
How can OpenSSL be fixed?
Even though the actual code fix may appear trivial, OpenSSL team is the expert in fixing it properly so latest fixed version 1.0.1g or newer should be used. If this is not possible software developers can recompile OpenSSL with the handshake removed from the code by compile time option -DOPENSSL_NO_HEARTBEATS.
Should heartbeat be removed to aid in detection of vulnerable services?
Recovery from this bug could benefit if the new version of the OpenSSL would both fix the bug and disable heartbeat temporarily until some future version. It appears that majority if not almost all TLS implementations that respond to the heartbeat request today are vulnerable versions of OpenSSL. If only vulnerable versions of OpenSSL would continue to respond to the heartbeat for next few months then large scale coordinated response to reach owners of vulnerable services would become more feasible.
Can I detect if someone has exploited this against me?
Exploitation of this bug leaves no traces of anything abnormal happening to the logs.
Can IDS/IPS detect or block this attack?
Although the content of the heartbeat request is encrypted it has its own record type in the protocol. This should allow intrusion detection and prevention systems (IDS/IPS) to be trained to detect use of the heartbeat request. Due to encryption differentiating between legitimate use and attack can not be based on the content of the request, but the attack may be detected by comparing the size of the request against the size of the reply. This seems to imply that IDS/IPS can be programmed to detect the attack but not to block it unless heartbeat requests are blocked altogether.
Has this been abused in the wild?
We don’t know. Security community should deploy TLS/DTLS honeypots that entrap attackers and to alert about exploitation attempts.
Can attacker access only 64k of the memory?
There is no total of 64 kilobytes limitation to the attack, that limit applies only to a single heartbeat. Attacker can either keep reconnecting or during an active TLS connection keep requesting arbitrary number of 64 kilobyte chunks of memory content until enough secrets are revealed.
Is this a MITM bug like Apple’s goto fail bug was?
No this doesn’t require a man in the middle attack (MITM). Attacker can directly contact the vulnerable service or attack any user connecting to a malicious service. However in addition to direct threat the theft of the key material allows man in the middle attackers to impersonate compromised services.
Does TLS client certificate authentication mitigate this?
No, heartbeat request can be sent and is replied to during the handshake phase of the protocol. This occurs prior to client certificate authentication.
Does OpenSSL’s FIPS mode mitigate this?
No, OpenSSL Federal Information Processing Standard (FIPS) mode has no effect on the vulnerable heartbeat functionality.
Does Perfect Forward Secrecy (PFS) mitigate this?
Use of Perfect Forward Secrecy (PFS), which is unfortunately rare but powerful, should protect past communications from retrospective decryption. Please see https://twitter.com/ivanristic/status/453280081897467905 how leaked tickets may affect this.
Can heartbeat extension be disabled during the TLS handshake?
No, vulnerable heartbeat extension code is activated regardless of the results of the handshake phase negotiations. Only way to protect yourself is to upgrade to fixed version of OpenSSL or to recompile OpenSSL with the handshake removed from the code.
Who found the Heartbleed Bug?
This bug was independently discovered by a team of security engineers (Riku, Antti and Matti) at Codenomicon and Neel Mehta of Google Security, who first reported it to the OpenSSL team. Codenomicon team found heartbleed bug while improving the SafeGuard feature in Codenomicon’s Defensics security testing tools and reported this bug to the NCSC-FI for vulnerability coordination and reporting to OpenSSL team.
What is the Defensics SafeGuard?
The SafeGuard feature of the Codenomicon’s Defensics security testtools automatically tests the target system for weaknesses that compromise the integrity, privacy or safety. The SafeGuard is systematic solution to expose failed cryptographic certificate checks, privacy leaks or authentication bypass weaknesses that have exposed the Internet users to man in the middle attacks and eavesdropping. In addition to the Heartbleed bug the new Defensics TLS Safeguard feature can detect for instance the exploitable security flaw in widely used GnuTLS open source software implementing SSL/TLS functionality and the “goto fail;” bug in Apple’s TLS/SSL implementation that was patched in February 2014.
Who coordinates response to this vulnerability?
NCSC-FI took up the task of reaching out to the authors of OpenSSL, software, operating system and appliance vendors, which were potentially affected. However, this vulnerability was found and details released independently by others before this work was completed. Vendors should be notifying their users and service providers. Internet service providers should be notifying their end users where and when potential action is required.
Is there a bright side to all this?
For those service providers who are affected this is a good opportunity to upgrade security strength of the secret keys used. A lot of software gets updates which otherwise would have not been urgent. Although this is painful for the security community, we can rest assured that infrastructure of the cyber criminals and their secrets have been exposed as well.
Where to find more information?
This Q&A was published as a follow-up to the OpenSSL advisory, since this vulnerability became public on 7th of April 2014. The OpenSSL project has made a statement at https://www.openssl.org/news/secadv_20140407.txt. NCSC-FI published an advisory at https://www.cert.fi/en/reports/2014/vulnerability788210.html. Individual vendors of operating system distributions, affected owners of Internet services, software packages and appliance vendors may issue their own advisories.
Update 4/16/14: The National Journal is reporting that Google knew about Heartbleed beforehand but did not tell the NSA about it. Apparently Google is still pretty upset about the NSA hacking into their data center. According to National Journal::
“… asked whether Google discussed Heartbleed with the government, a company spokeswoman said only that the “security of our users’ information is a top priority” and that Google users do not need to change their passwords.”
