Much has been written about the Log4j vulnerabilities since the first was reported on Dec.9, 2021, and not nearly enough about how they effect our cloud environments. This CVE required immediate action from both cloud providers and cloud users to update and patch their infrastructure. However, as with every zero-day, there was a significant amount of time until it was exposed, allowing attackers to exploit this vulnerability. Most attacks seen in the wild usually ended with mining activity or ransomware on the vulnerable host. In the context of cloud, an attack may result in initial access to the cloud environment, potentially allowing an attacker to gain access to sensitive data and resources.
In this blog we will share additional insights on how the log4j vulnerabilities may be exploited against cloud environments and the impact this may cause, and then provide information on Cortex XDR detection and hunting capabilities to handle such scenarios.
Log4j in the Clouds
The major cloud providers have published patches for the Log4j vulnerabilities, including Amazon Web Services (AWS), Google Cloud, and Microsoft Azure. As expected, some of the most common cloud services were vulnerable to the Log4j CVE, including compute services, storage services, databases, and more. Therefore, it is possible that organizations’ cloud environments were breached during that time before patches were released or applied, therefore actions such as detection, threat hunting, and investigation and response should be performed.
Log4j Cloud Exploitation
By taking into account how widespread the Log4j library usage is, and the fact that the primary CVE is a Remote Code Execution (RCE) zero-day with critical severity, we can try to understand the variety of options available to a malicious actor looking to exploit the vulnerability in cloud environments.
The techniques shown here would work against unpatched systems – and could have been carried out by attackers before organizations had the chance to patch. We recommend patching against log4j vulnerabilities as soon as possible, while also following best practices for detection and hunting to identify any existing breaches.
Exploitation attempts seen in the wild fall into two categories:
1. Direct exploitation attempt to the vulnerable asset, a malicious request to a vulnerable asset with the expectation for the exploitation to execute opun that specific asset.
2. Indirect exploitation attempt to a vulnerable asset, a malicious request to an asset which is not vulnerable, however, the request will most likely be logged on a different service which may be vulnerable.
Direct exploitation attempts in the wild
Attackers sometimes attempted to use the vulnerabilities to get hold of secrets that could grant initial or expanded access.
An example of a direct exploitation attempt seen in the wild were trying to retrieve AWS ENV variables:
While not every compute instance holds secrets in its ENV variables, for AWS Lambda this is not the case. This means that exploitation attempts upon a vulnerable Lambda function will result in a successful exfiltration of the Lambda function’s token, allowing the attacker to execute Cloud API calls on behalf of the Lambda. We would recommend following the mitigation steps and recommendations provided by AWS.
Indirect exploitation attempts in the wild
A different approach is to try to affect an organization's logging server/service and not the attacked resource. For example, see this malicious API call for cloud storage objects:
In such cases the executed attack will not be shown against any customer-owned servers, but the actual infrastructure that they’re installed on, makes it harder or impossible to detect on the customer logs side.
Having said that, once an attacker compromises the actual infrastructure other attacks can be carried against the customer’s assets with high probability of avoiding most mitigation out there.
Seeking Tokens to Gain Initial Access
Metadata Abuse across the clouds
As the exploitation can be used for Remote Code Execution (RCE), it can be exploited in many vectors, such a vector is the cloud Metadata service, handling data regarding your compute instance that you can use to configure or manage the running instance.
A successful exploitation enables attackers to access the Metadata service, which can only be accessed within the machine through a loopback. As seen below, once Metadata access was achieved, a malicious actor may manipulate it to its own advantage.
AWS Instance Metadata Abuse
The attacker can exploit the vulnerable customer's service using an HTTP request with a crafted log4j payload, which can be implemented in various manners, e.g headers, URI, etc. The payload retrieves the metadata service via the machine's loopback interface and sends the output to the attacker's controlled server.
Payload flow -
- Queries metadata service using an HTTP GET request to
- Encodes the output with base64 and sends it back to the attacker HTTP server.
Using this method, attackers can exfiltrate all the information available to the Metadata service which includes the instance’s token, network information, SSH key, etc . Below, we demonstrate how it could steal SSH keys:
Google Cloud Instance Metadata Abuse
A similar scenario applies to unpatched instances hosted on Google Cloud.
We would recommend following the mitigation steps and recommendations provided by GCP.
A successful attack could be used to exfiltrate the Service-Account (SA) access token that is attached to the machine, as well as additional metadata about the machine, including the associated project ID, instance name, and instance zone.
Exilitrate a Google Cloud Instance Access Token
Once the SA token and the instance details are exfiltrated, the attacker can leverage them to invoke additional cloud API calls on the SA’s behalf. This may include the compute API, which enables gaining the current metadata fingerprint and modifying the instance metadata by adding new SSH keys to it. The metadata SSH keys are automatically processed and added to the local machine.
As a result, the attacker has backdoor privileged SSH access to the instance.
New Local User SSH Access
In Google Cloud, by default, the SA account which is attached to new instances contains the IAM editor role with a scope of all instances (which is a very permissive permission), meaning an attacker would be able to launch as many instances as desired after successfully compromising it (and set an army of miners).
To avoid such cases, we would recommend following limiting service account privileges guidelines by Google Cloud.
Detecting and Investigating With Cortex XDR For Cloud
As we can see, there are many attack techniques that can be carried out in the cloud. Cortex XDR has been improving coverage for such attack scenarios to assist customers in detection, hunting, and mitigation.
Log4j exploitation via cloud SDK shown in Cortex’s new Cloud causality card
The ability to detect exfiltration of cloud tokens is one of the crucial Cortex XDR for cloud capabilities for this situation.
Below we can see an example of such an alert: “Suspicious usage of EC2 token.“
Cortex XDR for cloud is able to learn the behavior of cloud environments and detect when a specific EC2 token (a token generated by and for an EC2 instance) is being used outside of its dedicated instance.
In addition, Cortex XDR for cloud is able to learn typical behavior in your cloud and alert on unusual activities – for example, unusual geolocation activity.
Specifically for cloud assets, unusual geolocation activity may indicate a breach, meaning that
If XDR for cloud has learned a typical cloud instance activity comes from US regions (as it is deployed there), this means that we don't expect to see any type of API call execution from any other geolocation.
Analytics profiles learning the usual behavior in the environment
While monitoring, detecting and hunting the attack's exploitation attempts is necessary, in many cases, additional actions are required in order to determine the impact on cloud infrastructure.
Let’s try to investigate the attack described above against the Google Cloud hosted instance.
As mentioned, an attacker could gain initial access by exfiltrating the Service Account (SA) token remotely and use it to gain SSH access to the instance.
As an initial step, we can try to hunt for unusual activity. We can do so by imitating the “unusual geolocation activity” alert with a minor change. To do so, we will use the following query (while the below query filters for Google Cloud, the filter cloud_provider = "GCP" condition can be removed, and this query will be executed upon all cloud providers):
By using filter identity_type != ENUM.Users we are filtering only cloud identities which are not of the type User. As mentioned above, a cloud asset should not have multiple geo locations, and as we can see in the results, the compromised SA pops up with three different geolocations.
Hunting for cloud identities (non-users) with multiple geolocation activity
Once we have identified the compromised SA, we can investigate and look for all the operations executed by it by running the following XQL query:
Investigating operation of a specific SA
We can see above, two API calls (one successful and one that failed) were executed to update the Metadata service. We can also find additional indicators of compromise (IoCs) with the attack IP and ASN it was launched from.
Summary
The primary Log4j vulnerability is extremely critical, potentially allowing anonymous attackers from the internet to gain initial access to cloud environments. Cortex XDR for cloud provides a useful suite of tools to quickly detect and investigate such attacks – from out of the box detectors that detect different patterns of the attack to incidents and alerts visualization and the ability to run hunting queries against audit logs to investigate the cloud breach and its affected users and resources.
Learn More
To learn how you can use Cortex XDR to investigate and stop Log4Shell attacks, investigate alerts and find vulnerable Log4j software in your environment, see our Log4j incident response simulation:
