Advanced Hunting 4 Hero’s

While working with security incidents, the primary key is information. Things like important information, logs, data sources, a correlation between all data, and the signals – All of these are valuable when hunting and searching for specific information, for example, when searching for a patient zero.

Since many Microsoft 365 security services (ATP) come together and fit into one central place, it’s easier to search for data and perform unified hunting across all security services.

The current Microsoft security service is Microsoft Cloud App Security, Office 365 ATP, Azure ATP, and Defender ATP.

Note: This blog-post is part of a series about Microsoft 365 Defender. 

To work with all Microsoft 365 security stack, you need first to activate (opt-in) the Microsoft 365 Defender (previous MTP). Once it is activated, you can start to investigate incidents and search for security information.

The information is available on the Microsoft 365 Defender (previous MTP) portal and available on the Advanced Hunting page across Office and device data.

Before we start to look for some data, we must know how the Advanced hunting Schema is working, and once you understand how the Schema is working is better to search for data.

Few words about Threat hunting. Threat hunting is the practice of proactively searching for threats that are lurking undetected in a network. Threat hunting digs deep to find malicious actors in your environment that have slipped past your initial endpoint security defenses.

After sneaking in, an attacker can stealthily remain in a network for a long time as they quietly collect data, look for confidential material, or obtain login credentials that will allow them to move laterally across the environment.

Threat hunting is becoming increasingly important as companies seek to stay ahead of the latest cyber threats and rapidly respond to any potential attacks.

Advanced Hunting Dashboard

First, go to the Advanced Hunting portal >>> https://security.microsoft.com/advanced-hunting

Once you’ve landed on the Advanced Hunting dashboard, you will have the following windows that are divided into few categories:

The Schema includes multiple tables that provide either event information or information about devices, alerts, identities, and other entity types.

Functions are an enrichment function in advanced hunting and allow you to write more accurate queries.

Query providing the input field with all commands and queries itself

Export providing the output queries with all results

Filters provide a quick way to add filters to the query.

Create detection rules that let you proactively monitor various events and system states

Tip: When trying new queries, always use a limit to avoid substantial result sets. You can also initially assess the size of the result set using counts.

Advanced Hunting Breaking Down

Behind the scene, Advanced Hunting working with many components, tables, features, data sources, and various platforms and options the way its work is exciting, so let’s break down.

Data Sources

Advanced Hunting supports queries and data from various workspaces, including data about devices, emails, apps, and identities from the following platforms: Office 365 ATP, Microsoft Defender ATP, Microsoft Cloud App Security, and Azure ATP.

Data source Type Table Info (example)
Office 365 ATP Email EmailEvents
Microsoft Defender ATP Device DeviceInfo
Azure ATP App AppFileEvents
Microsoft Cloud App Security App AppFileEvents

Data Types

Advanced Hunting data has two main types:

Event or activity includes tables such as alerts, security events, system events, and routine assessments. Data receiving immediately after the sensors that collect them successfully transmit them to the corresponding cloud services.

Entity includes tables such as users and devices. This data comes from both relatively static data sources, such as Active Directory entries, and dynamic sources, such as event logs.

To provide new data, tables are updated every 15 minutes with any further information, adding rows that might not be fully populated. Every 24 hours, data is consolidated to insert a record that contains the latest, most comprehensive data set about each entity.

Tip: When joining tables, specify the table with fewer rows first

Schema Tables

All data sources come into few Schema types, and each schema type represents events and activities.

Alerts Apps & Identities Email Devices TVM
AlertInfo IdentityInfo EmailEvents DeviceInfo DeviceTvmSoftwareInventoryVulnerabilities
AlertEvidence IdentityLogonEvents EmailAttachmentInfo DeviceNetworkInfo DeviceTvmSoftwareVulnerabilitiesKB
IdentityQueryEvents EmailUrlInfo DeviceProcessEvents DeviceTvmSecureConfigurationAssessment
AppFileEvents EmailPostDeliveryEvents DeviceNetworkEvents DeviceTvmSecureConfigurationAssessmentKB
DeviceFileEvents
DeviceRegistryEvents
DeviceLogonEvents
DeviceImageLoadEvents
DeviceImageLoadEvents  
DeviceImageLoadEvents

Tip: Look in a specific column rather than running full-text searches across all columns

Schema Columns

Each table includes data such as the Column name Data type, and each one of them contains the actual information.

The Column name contains the information type for each table, and each table and schema type has its Column Name, and there’s a difference between the Email and Device tables.

The Data type contains each data’s value with shared values such as strings, boolean, int, DateTime, long.

For example, The DeviceNetworkInfo table contains information about the networking configuration, including network adapters, IP and MAC addresses, and connected networks or domains.

Tip: each table contains a specific column name, so you cannot compare Email to Identity

In short, every table has different columns, but,  in many situations, they overlap for the most common information types.

For example, the following query finds the ten latest logins made by email recipients within 30 minutes after they received known malicious emails.

Functions

The Function allows you to find more accurate information and, in some cases, to speed up analysis in Advanced Hunting. Functions are reusable queries or query parts and support a specific function: FileProfile.

The FileProfile() function is an enrichment function in advanced hunting that adds the following data to files found by the query.

Queries

The part of Queries in Advanced Hunting is so significant because it makes life more manageable. There are hundreds of Advanced Hunting queries, for example, Delivery, Execution, C2, and so much more.

Besides, you can save all your queries to My Queries and run them as you want.

Tip: When joining tables, specify the table with fewer rows first

Custom Detection

With custom detections, you can proactively monitor and respond to various events and system states, including suspected activities and misconfigured devices. This is made possible by customizable detection rules that automatically trigger alerts as well as response actions.

In general, Custom detections provide:

  • Alerts for rule-based detections built within advanced hunting queries
  • Automatic response actions and remediations that apply to files and devices

You can create a custom detection rule with a specific response by the following actions:

  • Prepare the query
  • Create a rule and provide alert details
  • Specify actions

Custom Detection Highlights

When creating the rules, you need to provide details with one or more values.

  • Detection name
  • Frequency
  • Alert title
  • Severity
  • Category
  • Description
  • Recommended actions

Remediation can be for Email, Device, and other tables.

  • Isolate device
  • Collect investigation package
  • Run antivirus scan
  • Allow or Block files.
  • Quarantine file

KQL

Kusto is the PowerShell of Query Languages.

KQL, the Kusto Query Language, is used in many Microsoft services, including the Azure platform and Microsoft 365 Security platform such as the ATP family.

Kusto is a service for storing and running interactive analytics over Big Data.

It is based on relational database management systems, supporting entities such as databases, tables, and columns. It provides complex analytics query operators, such as calculated columns, searching and filtering or rows, group by-aggregates, and joins.

Kusto offers excellent data ingestion and query performance by “sacrificing” the ability to perform in-place updates of individual rows and cross-table constraints or transactions. Therefore, it supplants, rather than replaces, traditional RDBMS systems for OLTP and data warehousing scenarios.

As a Big Data service, Kusto handles structured, semi-structured (JSON-like nested types), and unstructured, including free-text data equally well.

For more information about Kusto – The PowerShell of Query Language or Kusto Query Language Introduction

Use Cases

Advanced Hunting and KQL provide many ways to protect your asset, and there are many use cases.

  • Monitor Windows Firewall configuration
  • Monitor local administrators and administrative logons
  • Monitor Email activities for C-Level accounts and Prirvlidege Accounts

A good example of remote devices. Let’s assume you receive IoC’s for an ongoing attack with known files or IP’s. From this point, you can query these IoC’s on both on-premises devices and on the remote devices which exist on the internet and never in the office.

In Conclusion, Information is robust, and nothing explains it better than Advanced Hunting in Microsoft Threat Protection. Once you know how Schema works and the relevant table, you can start and work with KQL to run queries and receive useful information from your data sources and signals.

More information about Advanced Hunting

Advanced Hunting 4 Hero’s

While working with security incidents, the primary key is information. Things like important information, logs, data sources, a correlation between all data, and the signals – All of these are valuable when hunting and searching for specific information, for example, when searching for a patient zero.
Since many Microsoft 365 security services (ATP) come together and fit into one central place, it’s easier to search for data and perform unified hunting across all security services.
The current Microsoft security service is Microsoft Cloud App Security, Office 365 ATP, Azure ATP, and Defender ATP.
Note: This blog-post is part of a series about Microsoft 365 Defender. 
To work with all Microsoft 365 security stack, you need first to activate (opt-in) the Microsoft 365 Defender (previous MTP). Once it is activated, you can start to investigate incidents and search for security information.
The information is available on the Microsoft 365 Defender (previous MTP) portal and available on the Advanced Hunting page across Office and device data.
Before we start to look for some data, we must know how the Advanced hunting Schema is working, and once you understand how the Schema is working is better to search for data.
Few words about Threat hunting. Threat hunting is the practice of proactively searching for threats that are lurking undetected in a network. Threat hunting digs deep to find malicious actors in your environment that have slipped past your initial endpoint security defenses.
After sneaking in, an attacker can stealthily remain in a network for a long time as they quietly collect data, look for confidential material, or obtain login credentials that will allow them to move laterally across the environment.
Threat hunting is becoming increasingly important as companies seek to stay ahead of the latest cyber threats and rapidly respond to any potential attacks.

Advanced Hunting Dashboard

First, go to the Advanced Hunting portal >>> https://security.microsoft.com/advanced-hunting
Once you’ve landed on the Advanced Hunting dashboard, you will have the following windows that are divided into few categories:
The Schema includes multiple tables that provide either event information or information about devices, alerts, identities, and other entity types.
Functions are an enrichment function in advanced hunting and allow you to write more accurate queries.
Query providing the input field with all commands and queries itself
Export providing the output queries with all results
Filters provide a quick way to add filters to the query.
Create detection rules that let you proactively monitor various events and system states

Tip: When trying new queries, always use a limit to avoid substantial result sets. You can also initially assess the size of the result set using counts.

Advanced Hunting Breaking Down

Behind the scene, Advanced Hunting working with many components, tables, features, data sources, and various platforms and options the way its work is exciting, so let’s break down.

Data Sources

Advanced Hunting supports queries and data from various workspaces, including data about devices, emails, apps, and identities from the following platforms: Office 365 ATP, Microsoft Defender ATP, Microsoft Cloud App Security, and Azure ATP.

Data source Type Table Info (example)
Office 365 ATP Email EmailEvents
Microsoft Defender ATP Device DeviceInfo
Azure ATP App AppFileEvents
Microsoft Cloud App Security App AppFileEvents

Data Types

Advanced Hunting data has two main types:
Event or activity includes tables such as alerts, security events, system events, and routine assessments. Data receiving immediately after the sensors that collect them successfully transmit them to the corresponding cloud services.
Entity includes tables such as users and devices. This data comes from both relatively static data sources, such as Active Directory entries, and dynamic sources, such as event logs.
To provide new data, tables are updated every 15 minutes with any further information, adding rows that might not be fully populated. Every 24 hours, data is consolidated to insert a record that contains the latest, most comprehensive data set about each entity.
Tip: When joining tables, specify the table with fewer rows first

Schema Tables

All data sources come into few Schema types, and each schema type represents events and activities.

Alerts Apps & Identities Email Devices TVM
AlertInfo IdentityInfo EmailEvents DeviceInfo DeviceTvmSoftwareInventoryVulnerabilities
AlertEvidence IdentityLogonEvents EmailAttachmentInfo DeviceNetworkInfo DeviceTvmSoftwareVulnerabilitiesKB
IdentityQueryEvents EmailUrlInfo DeviceProcessEvents DeviceTvmSecureConfigurationAssessment
AppFileEvents EmailPostDeliveryEvents DeviceNetworkEvents DeviceTvmSecureConfigurationAssessmentKB
DeviceFileEvents
DeviceRegistryEvents
DeviceLogonEvents
DeviceImageLoadEvents
DeviceImageLoadEvents  
DeviceImageLoadEvents

Tip: Look in a specific column rather than running full-text searches across all columns

Schema Columns

Each table includes data such as the Column name Data type, and each one of them contains the actual information.
The Column name contains the information type for each table, and each table and schema type has its Column Name, and there’s a difference between the Email and Device tables.
The Data type contains each data’s value with shared values such as strings, boolean, int, DateTime, long.
For example, The DeviceNetworkInfo table contains information about the networking configuration, including network adapters, IP and MAC addresses, and connected networks or domains.
Tip: each table contains a specific column name, so you cannot compare Email to Identity
In short, every table has different columns, but,  in many situations, they overlap for the most common information types.
For example, the following query finds the ten latest logins made by email recipients within 30 minutes after they received known malicious emails.

Functions

The Function allows you to find more accurate information and, in some cases, to speed up analysis in Advanced Hunting. Functions are reusable queries or query parts and support a specific function: FileProfile.
The FileProfile() function is an enrichment function in advanced hunting that adds the following data to files found by the query.

Queries

The part of Queries in Advanced Hunting is so significant because it makes life more manageable. There are hundreds of Advanced Hunting queries, for example, Delivery, Execution, C2, and so much more.
Besides, you can save all your queries to My Queries and run them as you want.

Tip: When joining tables, specify the table with fewer rows first

Custom Detection

With custom detections, you can proactively monitor and respond to various events and system states, including suspected activities and misconfigured devices. This is made possible by customizable detection rules that automatically trigger alerts as well as response actions.
In general, Custom detections provide:

  • Alerts for rule-based detections built within advanced hunting queries
  • Automatic response actions and remediations that apply to files and devices

You can create a custom detection rule with a specific response by the following actions:

  • Prepare the query
  • Create a rule and provide alert details
  • Specify actions

Custom Detection Highlights

When creating the rules, you need to provide details with one or more values.

  • Detection name
  • Frequency
  • Alert title
  • Severity
  • Category
  • Description
  • Recommended actions

Remediation can be for Email, Device, and other tables.

  • Isolate device
  • Collect investigation package
  • Run antivirus scan
  • Allow or Block files.
  • Quarantine file

KQL

Kusto is the PowerShell of Query Languages.
KQL, the Kusto Query Language, is used in many Microsoft services, including the Azure platform and Microsoft 365 Security platform such as the ATP family.
Kusto is a service for storing and running interactive analytics over Big Data.
It is based on relational database management systems, supporting entities such as databases, tables, and columns. It provides complex analytics query operators, such as calculated columns, searching and filtering or rows, group by-aggregates, and joins.
Kusto offers excellent data ingestion and query performance by “sacrificing” the ability to perform in-place updates of individual rows and cross-table constraints or transactions. Therefore, it supplants, rather than replaces, traditional RDBMS systems for OLTP and data warehousing scenarios.
As a Big Data service, Kusto handles structured, semi-structured (JSON-like nested types), and unstructured, including free-text data equally well.
For more information about Kusto – The PowerShell of Query Language or Kusto Query Language Introduction

Use Cases

Advanced Hunting and KQL provide many ways to protect your asset, and there are many use cases.

  • Monitor Windows Firewall configuration
  • Monitor local administrators and administrative logons
  • Monitor Email activities for C-Level accounts and Prirvlidege Accounts

A good example of remote devices. Let’s assume you receive IoC’s for an ongoing attack with known files or IP’s. From this point, you can query these IoC’s on both on-premises devices and on the remote devices which exist on the internet and never in the office.
In Conclusion, Information is robust, and nothing explains it better than Advanced Hunting in Microsoft Threat Protection. Once you know how Schema works and the relevant table, you can start and work with KQL to run queries and receive useful information from your data sources and signals.
More information about Advanced Hunting

You may also like...

3 Responses

  1. June 12, 2022

    […] Advanced Hunting 4 Hero’s […]

  2. March 6, 2023

    […] Advanced Hunting 4 Hero’s […]

  3. March 11, 2023

    […] More queries and tips on Twitter and the Advanced Hunting 4 Hero’s. […]

Leave a Reply

error: Content is Protected !!
%d bloggers like this: