CDI’s Statement on the Evolving Log4J Vulnerability
By Josh More, Chief Information Security Officer, CDI
*Last Updated December 30, 2021, 3:45 PM EST*
Summary of Issue:
On December 9, 2021, it was publicly announced that a vulnerability in the Log4J component of many systems allowed for remote code execution for a wide variety of systems across the world. This vulnerability is being actively exploited in several ways, and the full scope of the issue is still being determined.
Additional information released on December 14th and 18th, 2021 shows that, as anticipated, the initial patches did not address some risk scenarios and another updated patch was released. This change does not invalidate the work done previously, as that patch did address the immediate threat. It is likely that there will be more patches in the near future, however, we are now in the “long tail” of this issue, and the criticality of the newer patches is dropping. We expect the heightened attention on this component to identify more obscure and difficult to exploit issues over time.
UPDATE: In recent days, there have been two minor events. The first is yet another update for log4j itself. The issue, identified as CVE-2021-44832 is a much lower priority than the other such updates, as exploiting the issue requires configuration level access. Unless the system is set up such that people configuring logging do not have administrative access and those people are untrusted, there is no risk from this vulnerability. The other issues involve two unrelated patches from Apache project. Apache oversees many projects, not just log4j, and it is important to recognize that these other issues have no impact on the continuing log4j situation.
While it is believed that this vulnerability could have been exploited against specific targets as early as December 1, 2021, mass exploitation did not begin until shortly after the release of the vulnerability, on December 10. There are indications that attackers were deploying Bitcoin miners, but not that attackers have been targeting business sensitive data. At this time, there is no evidence that CDI has been targeted or compromised.
Impact on CDI and CDI’s Customers:
CDI has not developed any code that leverages the Log4J component. Additionally, while some of CDI’s vendor-supplied tools were vulnerable, the majority of those tools have been patched with vendor-released patches and there are no indicators of compromise. Many CDI customers have vendor-supplied software that is vulnerable, however CDI’s network connections to customers are highly segmented, and any compromise at one CDI customer cannot impact another across CDI’s network. At this time, no compromise at any customer has been found.
CDI will continue to update internal systems as vendors incorporate the new patches. CDI recommends that customers do the same. However, we must stress that the criticality of the initial issue is addressed with the early patches available from vendors. The details of how each vendor is impacted by the newer discoveries varies drastically by vendor and not all vendors will be issuing or will need to issue patches beyond their initial release.
We must stress that the presence of the vulnerable software does not necessarily mean that the issue is critical. The nature of the vulnerability is such that exploitation varies drastically between vendors and products – with some products requiring significantly greater technical skill for exploitation than others. Additionally, some vulnerable software is cloud-based and others are centrally-managed, so they are patched automatically by the vendors. Still others are locally-installed or part of deployed appliances and will require direct action. As such, CDI is allowing cloud-based and centrally managed vendors to address this issue through their existing processes. CDI is prioritizing – for both CDI and client-owned environments – the more easily compromised externally-facing software for remediation and will proceed from there following a risk-based approach.
CDI can address this issue based on lists of known-vulnerable software as well as performing discovery and testing activities. In recent days, we have seen significant maturity increases in the tools used to scan for vulnerable components and to actively test applications. However, it is important to understand that – while the tools are improving quickly – we are still in early days on this vulnerability and a “pass” should not be interpreted as a “clean” environment. Tools will continue to improve for weeks or months to come. CDI recommends that all customers engage in daily to weekly vulnerability scanning on all potentially-impacted systems to more easily identify when vendor-released patches are available for each system.
The timeline from vulnerability release to weaponization continues to shrink, allowing for rapid mass exploitation when a wide-ranging vulnerability is discovered. Also, as this issue allows for arbitrary remote execution of code, the best detection of exploitation will be behavioral, not signature-based – and the anti-malware and EDR vendors are still developing behavioral detections.
Testing for this issue can be challenging. While simple scanning and form field testing will cover the majority of real-world risks, testing for API weakness and testing for copied code will require significant manual effort to detect and exploit.
The list of vendors vulnerable to this issue that require on-site patching or reconfiguration for remediation continues to grow and may never be fully complete. Many of these vendors are placing notices on their sites but are not necessarily proactively reaching out to customers. This is particularly true for vendors that largely operate as a “channel” and are placing the notification burden on their partners.
Additionally, as this vulnerable software is used in many systems, we expect that the information technology industry will be dealing with this issue for quite some time.
While Log4J is used in many systems, it is used in very different ways, and the ease of exploitation varies accordingly. For most systems, exploitation requires an external query, making it much more difficult to exploit systems that are egress-filtered. Additionally, the vulnerable module is included with some products and only referenced if specifically configured to do so, so there are many potentially vulnerable systems that are not exploitable due simply to configuration details.
Additionally, there are workarounds as well as patches available to many systems. The modular nature of Java helped to make this issue widespread, but it also makes it very easy to address to patches are being released much more quickly than is usual in the industry.
Finally, this issue is getting an unprecedented amount of media attention, so the likelihood that vendors are proactively investigating and addressing this issue is significantly higher than we saw with similar issues in the past.
If you were running a vulnerable version of Log4J, either directly or as a component of another product, you should apply all available patches. If patches are not available, then implementing and verifying a workaround should be performed. Additionally, as should be done to combat all supply chain attacks, all network egress from third party systems should be monitored and filtered wherever possible.
If you happen to have full logging data going back to November 30, 2021, it would be wise to review those logs for outbound connections from vulnerable systems to identify whether incident response may be needed.
Finally, this is an actively evolving situation and facts, and recommendations are expected to be clarified over time. There have already been some instances of vendors claiming they were not vulnerable only to retract that claim the following day. Please check this page for updates as time goes by.
As this issue is highly environmentally specific, if you have questions, please first reach out to your existing CDI contact.
If you believe that you have been compromised and need immediate security help, please also contact us at CDI.Security@cdillc.com.
Appendix – Vendor Statements:
The following is a list of links to public statements from CDI’s key vendors on the issue:
- CarbonBlack: https://community.carbonblack.com/t5/Knowledge-Base/All-Products-Where-Can-I-find-Information-on-CVE-2021-44228/ta-p/109162
- Cisco: https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-apache-log4j-qRuKNEbd
- Citrix: https://support.citrix.com/article/CTX33570
- Dell (login required): https://www.dell.com/support/kbdoc/en-uk/000194414/dell-response-to-apache-log4j-remote-code-execution-vulnerability
- Fortigate: https://www.fortiguard.com/psirt/FG-IR-21-245
- IBM WebSphere: https://www.ibm.com/support/pages/security-bulletin-vulnerability-apache-log4j-affects-websphere-application-server-cve-2021-44228
- Juniper: https://kb.juniper.net/InfoCenter/index?page=content&id=JSA11259&actp=RSS
- ManageEngine: https://pitstop.manageengine.com/portal/en/community/topic/apache-log4j-vulnerability-cve-2021-44228-1
- Microsoft: https://msrc-blog.microsoft.com/2021/12/11/microsofts-response-to-cve-2021-44228-apache-log4j2/
- Mitel: https://www.mitel.com/support/security-advisories/mitel-product-security-advisory-21-0010
- New Relic: https://newrelic.com/blog/nerdlog/security-guidance-for-log4j
- Okta: https://sec.okta.com/articles/2021/12/log4shell
- Oracle: https://support.oracle.com/rs?type=doc&id=2827611.1
- Palo Alto: https://security.paloaltonetworks.com/CVE-2021-44228
- Sailpoint: https://community.sailpoint.com/t5/Community-Announcements/Log4j-Vulnerability/ba-p/206702
- Sophos: https://www.sophos.com/en-us/security-advisories/sophos-sa-20211210-log4j-rce
- Splunk: https://www.splunk.com/en_us/blog/bulletins/splunk-security-advisory-for-apache-log4j-cve-2021-44228.html
- VMware: https://kb.vmware.com/s/article/87068?lang=en_US
- Varonis (login required): https://help.varonis.com/s/article/Apache-Log4j-Zero-Day-Vulnerability-CVE-2021-44228
- Veeam: https://www.veeam.com/kb4254
- Veritas: https://www.veritas.com/content/support/en_US/article.100052058
- Zerto: https://help.zerto.com/kb/000004822