The widely anticipated security fix included in OpenSSL 3.0.7 is for two buffer overflows that occur while the library is performing certificate validation, one of which may allow an attacker to gain remote code execution on a target system. The main threat is to clients, rather than servers, and exploitation is not a simple process.

The vulnerabilities (CVE-2022-3786 and CVE-2022-3602) are in the way that OpenSSL handles Punycode-encoded names. The latter issue, which is the one that OpenSSL initially described as critical in a pre-announcement last week, is rated as high in the final advisory after some further research found mitigating factors for exploitation.

“A buffer overrun can be triggered in X.509 certificate verification, specifically in name constraint checking. Note that this occurs after certificate chain signature verification and requires either a CA to have signed the malicious certificate or for the application to continue certificate verification despite failure to construct a path to a trusted issuer. An attacker can craft a malicious email address to overflow four attacker-controlled bytes on the stack,” the advisory for CVE-2022-3602 says.

“This buffer overflow could result in a crash (causing a denial of service) or potentially remote code execution. Many platforms implement stack overflow protections which would mitigate against the risk of remote code execution. The risk may be further mitigated based on stack layout for any given platform/compiler.”

The OpenSSL team said that the downgrade from critical to high for CVE-2022-3602 was the result of testing from third-party organizations that were able to identify mitigations.

“Firstly, we had reports that on certain Linux distributions the stack layout was such that the 4 bytes overwrote an adjacent buffer that was yet to be used and therefore there was no crash or ability to cause remote code execution,” the OpenSSL team said in a blog post.

“Secondly, many modern platforms implement stack overflow protections which would mitigate against the risk of remote code execution and usually lead to a crash instead.”

The other vulnerability is similar and related, but does not appear to include the possibility of remote code execution.

“An attacker can craft a malicious email address in a certificate to overflow an arbitrary number of bytes containing the '.' character (decimal 46) on the stack. This buffer overflow could result in a crash (causing a denial of service). In a TLS client, this can be triggered by connecting to a malicious server. In a TLS server, this can be triggered if the server requests client authentication and a malicious client connects,” the description of CVE-2022-3786 says.

Johannes Ullrich of the SANS Technology Institute said in an analysis of the flaws that gaining remote code execution using CVE-2022-3602 is not a simple matter.

“The certificate needs to contain a malicious Punycode encoded name, and the vulnerability is only triggered AFTER the certificate chain is verified. An attacker first needs to be able to have a malicious certificate signed by a certificate authority the client trusts. This does not appear to be exploitable against servers,” Ullrish said in an analysis.

“In short: While this is a potential remote code execution vulnerability, the requirements to trigger the vulnerability are not trivial, and I do not see this as a "Heartbleed Emergency". Patch quickly as updated packages become available, but beyond this, no immediate action is needed.”

The OpenSSL library is embedded in a large number of apps and operating systems, including many Linux distributions, as well as in mail servers, VPNs, and other systems. This vulnerability only affects version 3.0.x of OpenSSL, and not the older 1.1.1 branch, so the effects may be limited by that factor. Version 3.0.x has only been out since September 2021, so it is not as widely deployed as some of the older versions. Among the Linux distributions known to be vulnerable are some versions of Ubuntu, Fedora, Kali, OpenMandriva, OpenSUSE, and Red Hat Enterprise Linux.

The challenge now for enterprise security teams is identifying systems that are running vulnerable versions of OpenSSL. One way of doing this for web servers is to check the HTTP header that a server returns, which will often include version information for OpenSSL. Researchers at Censys, which compiles data on attack surfaces across the Internet, found about 7,000 web servers running vulnerable versions of the library as of Oct. 30.