A critical security vulnerability in Microsoft’s Azure cloud database platform – Azure Cosmos DB Critical Vulnerability – could have allowed the remote takeover of accounts, with admin rights to read, write and delete any information to a database instance.
According to researchers at Wiz, any Azure customer could access another customer’s account without authentication. According to researchers, the bug, dubbed ChaosDB, could be trivially exploited and impact thousands of organizations, including numerous Fortune 500 companies.
This blog post provides an overview of the Azure Cosmos DB and the ChaosDB vulnerability.

Azure Cosmos DB & Jupyter Notebooks

Azure Cosmos DB is a fully managed NoSQL database for modern app development. Single-digit millisecond response times, and automatic and instant scalability, guarantee the speed at any scale. Business continuity is assured with SLA-backed availability and enterprise-grade security.
App development is faster and more productive thanks to turnkey multi-region data distribution globally, open-source APIs, and SDKs for popular languages. Azure Cosmos DB takes database administration off your hands with automatic management, updates, and patching as a fully managed service. It also handles capacity management with cost-effective serverless and automatic scaling options that respond to application needs to match capacity with demand.
Azure Cosmos DB built-in Jupyter Notebooks are directly integrated into the Azure portal, and your Azure Cosmos DB accounts, making them convenient and easy to use. Developers, data scientists, engineers, and analysts can use the familiar Jupyter Notebooks experience to explore data cleaning, transformations, numerical simulations, statistical modeling, data visualization, and machine learning.
Built-in Jupyter notebooks in Azure Cosmos DB enable you to analyze and visualize your data from the Azure portal. This article describes how to enable this feature for your Azure Cosmos DB account.
Inside the notebook, you can take advantage of built-in commands and features that make it easy to create Azure Cosmos DB resources, upload data, and query and visualize your data in Azure Cosmos DB.
Jupyter Notebooks can include several types of components, each organized into discrete blocks or cells:

• Text and HTML
• Code and output
• Visualizations
• Multimedia
• Data

The following diagram shows the generalized hierarchy of elements that belong to a Cosmos DB account. 👇

Built-in Jupyter notebooks in Azure Cosmos DB enable you to analyze and visualize your data from the Azure portal. Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text.
Starting February 10, 2021, new Azure Cosmos accounts created in one of the supported regions will automatically have notebooks enabled. There is no additional configuration needed to enable notebooks.

What is ChaosDB?

ChaosDB is an unprecedented critical vulnerability in the Azure cloud platform that allows for remote account takeover of Azure’s flagship database – Cosmos DB. The vulnerability disclosed to Microsoft in August 2021 by Wiz Research Team gives any Azure user full admin access (read, write, delete) to another customer’s Cosmos DB instances without authorization. The vulnerability has a trivial exploit that doesn’t require previous access to the target environment and impacts thousands of organizations, including Fortune 500 companies.

Vulnerability Overview

By exploiting a chain of vulnerabilities in the Jupyter Notebook feature of Cosmos DB, a malicious actor can query information about the target Cosmos DB Jupyter Notebook. By doing so, the attacker will obtain a set of credentials related to the target Cosmos DB account, the Jupyter Notebook compute, and the Jupyter Notebook Storage account, including the Primary Key. Using these credentials, it is possible to view, modify, and delete data in the target Cosmos DB account via multiple channels. Below is a diagram that illustrates the attack.

CHAOSDB PoC

Was the issue Fixed?

On Aug 26, 2021, Microsoft notified over 30% of Cosmos DB customers about the potential security breach. Microsoft’s security teams immediately fixed the problem and disabled the vulnerable feature within 48 hours of the report. However, the vulnerability has been exploitable for months, and every Cosmos DB customer should assume they’ve been exposed. To mitigate the risk, Microsoft advises customers to regenerate the Cosmos DB Primary Keys. We believe the actual number of customers affected by ChaosDB is higher and recommend that all customers follow this guidance.

Mitigation & Remediation

Currently, there are no impacted customers.

Microsoft’s Statement

Microsoft released the following statement to impacted customers: “Microsoft has recently become aware of a vulnerability in Azure Cosmos DB that could potentially allow a user to gain access to another customer’s resources by using the account’s primary read-write key. This vulnerability was reported to us in confidence by an external security researcher. Once we became aware of this issue on 12 August 2021, we mitigated the vulnerability immediately.

We do not indicate that external entities outside the researcher had access to the primary read-write key associated with your Azure Cosmos DB account. In addition, we are not aware of any data access because of this vulnerability. Azure Cosmos DB accounts with a vNET or firewall enabled are protected by additional security mechanisms that prevent the risk of unauthorized access. Out of an abundance of caution, we are notifying you to take the following actions as a precautionary measure.”

Update on the vulnerability in the Azure Cosmos DB Jupyter Notebook Feature

This vulnerability only affects a subset of customers who had the Jupyter Notebook feature enabled. Notifications have been sent to all customers that could be potentially affected due to researcher activity, advising they regenerate their primary read-write key. Other keys, including the secondary read-write key, primary read-only key, and secondary read-only key, were not vulnerable.  
If you did not receive an email or in-portal notification, there is no evidence any other external parties had access to your primary read-write account key. The suggestion is to enable Diagnostic Logging and Azure Defender to be available and periodically rotate your keys. If you have diagnostic logs enabled, you can also review the logs for unusual IP addresses. 

Am I exposed

You can check quickly if your Azure environment is vulnerable, and you can check if you can minimize the vulnerability with a quick response with the flow below.

Note: If you’re gonna apply Azure Cosmos DB hardening with Firewall, Private Endpoint, and other security controls, bear in mind you only minimize the risk. 

Minimize the Risk

Keep in mind that you can take advantage of the fact that there are several built-in security options, the fact that you need to make sure to align your architecture and use security controls that can lower the risks. This is just a first step.
Note: in some scenarios, you could not allow regenerating the primary key. Therefore you can work with Azure Cosmos DB hardening to educate the attack surface are. The list below provides many recommendations that can be applied to your Azure Cosmos DB environment:

• Network security and firewall settings
• User authentication and fine-grained user controls
• Ability to replicate data globally for regional failures
• Ability to failover from one data center to another
• Local data replication within a data center
• Restoration of deleted data from backups
• Protect and isolate sensitive data
• Monitoring for attacks
• Responding to attacks
• Ability to geofence data to adhere to data governance restrictions
• Physical protection of servers in protected data centers
• Automatic data backups
• Certifications
• Azure Purview governance

A few recommendations and highlights from the Azure Cosmos DB portal security controls include Firewall, Private Endpoint, and Advanced Security.



The Azure Cosmos DB accounts should have firewall rules alert on Azure Security Center.

Keys Mitigation

Primary and secondary keys provide access to all the administrative resources for the database account, and each account consists of a primary key and a secondary key. The purpose of dual keys is to let you regenerate or roll keys, providing continuous access to your account and data.
Primary/secondary keys:

• Provide access to accounts, databases, users, and permissions.
• It cannot be used to provide granular access to containers and documents.
• Primary/secondary keys provide access to all the administrative resources for the database account.
• It can be regenerated at any time.

Primary/secondary keys come in two versions: read-write and read-only. The read-only keys only allow read operations on the account but do not provide access to read permissions resources.

Reset Keys via PowerShell – You can reset/regenerate all Azure Cosmos DB Keys, including the primary key and secondary key via PowerShell. Ps1 – Run this script carefully!!! Download the PowerShell reset script from GitHub – Account connection strings and Regenerate a key.

Azure Sentinel to the Rescue

As always, it comes to my favorite platform – Azure Sentinel. Assume you’ve got Azure Sentinel, and if yes, you can forward the Azure activity logs to the Azure Sentinel and create an incident to raise alerts if someone touches the Azure Cosmos DB keys.
First, make sure to enable the diagnostic setting with at least the following data: Administrative, Security, dataplanerequests, alert, policy. If you can allow all of this data, it will be excellent.


Once the diagnostic setting is configured, make some testing and reset keys. Make sure to reset on a test environment and NOT ON THE PRODUCTION ENVIRONMENT!

At this point, you’ve got the information, and you can play with KQL and create a query for an incident to raise an alert if someone touches the Keys. Below, the query lets each action extends from specific properties and projects several values.
TIP: you cannot pullout data only from OperationName value because it won’t provide the best value

Then, take the query to the analytics rule. Remember to add entities.

The simple query and analytics rule is available on my GitHub 👇
Azure-Sentinel-4-SecOps/Hunting/DocumentDb at master · eshlomo1/Azure-Sentinel-4-SecOps (github.com)

References

ChaosDB: Unauthorized Privileged Access to Microsoft Azure Cosmos DB
ChaosDB: How we hacked thousands of Azure customers’ databases | Wiz Blog
Update on the vulnerability in the Azure Cosmos DB Jupyter Notebook Feature – Microsoft Security Response Center
Introduction to Azure Cosmos DB | Microsoft Docs
Introduction to the built-in Jupyter Notebooks support in Azure Cosmos DB (Preview) | Microsoft Docs
Azure Cosmos DB resource model | Microsoft Docs
Database security – Azure Cosmos DB | Microsoft Docs
Learn how to secure access to data in Azure Cosmos DB | Microsoft Docs
Microsoft Azure Cosmos DB Guidance | CISA
Security Archives – Elli Shlomo (eshlomo.us)