Skip to content

Query Style Guide

shainw edited this page Oct 15, 2019 · 73 revisions

Goal

This document aims to create a uniform style for Sentinel content provided to and by Microsoft. We encourage external contributors to follow this same guidance, but this is not enforced. Microsoft will review and update any query that is pulled into the Azure Sentinel UX with the requirements below as needed.

Detections and Hunting Queries

Format

File format is YAML and should be validated with any YAML syntax validator, there are many online.

Name

Required for Detections and Hunting Queries

A short description of the detection. This should include what the detection is about without reading the full description.

  • It should be clear which entities are performing the suspicious activities on which datatype.
  • Use Sentence case capitalization
  • Do NOT end with a period
  • Length SHOULD NOT exceed 50 chars when possible
  • Terms to avoid (applies also to Description):
Examples
Avoid Use
IP IPAddress
Execute Run
Suspicious,Suspect Unexpected, Anomalous, Rare

Description

Required for Detections and Hunting Queries

Details the purpose of the query and any references such as EventID explanations or URL references

  • Starts with - "This query searches for" or "Identifies"
  • Should attempt to be a max of 5 sentences.
  • Do NOT Describe the Data source (connector or datatype).
  • Do NOT provide a Technical explanation for the query language used.
  • Standard English capitalization for description section.
What NOT to do Instead, do this
clients with a high reverse DNS count could be carrying out scanning activity. Alert is generated if the IP performing such reverse DNS lookups was not seen doing so in the preceding 7-day period. This query identifies IP addresses performing a high rate of reverse DNS lookups and has not been seen doing this lookup in the previous 7 days.

Severity

Required for Detections only

Indicates the confidence level of the detections

  • Low – Could potentially cause noise or is a detection that would need additional detections to raise the overall severity.
  • Medium – These are generally quiet but may require some additional investigation to verify the impact of the attack.
  • High – These are high confidence, rare detections that are generally guaranteed to indicate compromise or a high level impact of an attack.

ID

Required for Detections and Hunting Queries

This is a standard GUID. You can generate from just about any development tool or from PowerShell via the New-GUID cmdlet.

  • Must NOT collide with other GUIDs.
  • Be cautious when reusing files.

DataConnector

Not Required (Microsoft will add as needed)

One or more from the list of Data Connectors.

DataType

Not Required (Microsoft will add as needed)

One or more from the list of Data Types in your workspace. GitHub contains a general list based on folder structure.

QueryPeriod

Required for Detections only

The time frame that the query will run across, such as the last 3 days.

  • Expressed in Kusto Query Language (KQL) TimeSpan Format
  • Any learning or reference period MUST be included within this time.
  • Maximal Value Supported (technical limitation): 14d

QueryFrequency

Required for Detections only

How often the query runs against the data.

  • Expressed in Kusto Query Language (KQL) TimeSpan Format
  • If the QueryPeriod is greater than or equal to 2 days (2d), the QueryFrequncy value MUST NOT be less than 1 hour (1h)

AlertTriggerOperator

Required for Detections only

Indicates the mechanism that triggers the alert, such as greater than a count of 6.

Supported values:

  • gt – Greater Than
  • lt – Less Than
  • eq – Equal To

AlertTriggerThreshold

Required for Detections only

Indicates the threshold count related to the mechanism that triggers the alert, such as equal to 1

Supported Values:

  • Int – Any integer starting with 0.

If the AlertTriggerOperator is set to Greater Than and the AlertTriggerThreshold is set to 1, then the alert will only trigger if there is more than 1 of these alerts.

Tactics

Required for Detections only

Relevant MITRE Tactics

  • The name MUST NOT have any spaces
  • Example – InitialAccess or LateralMovement

RelevantTechniques

Recommended for Detections only

Relevant MITRE Techniques ID

  • MUST match MITRE Tactics
  • Example: T1100 or T1120

Query

Required for Detections and Hunting Queries

Kusto Query Language (KQL)

  • Define the human readable names for explicit constants:
    • let FailedLoginEventID = 4625;
    • let countThreshold = 6;
  • Use of comments to clarify the query is highly recommended.
    • Comments must be on a separate line, not at the end of a query statement line
    • // Removing noisy processes for an environment, adjust as needed
  • At least return every available entity field for mapping. See Entity Mapping below.
  • Sanitize the returned table so that it provides only the necessary properties to investigate further

Entity Mapping

Required for Detections and Hunting Queries

Mapping is the process of extracting entities from query’s results. In current version only one entity of each type can be extracted (per result entry). The extraction is based on the following:

  • timestamp
    • This is generally TimeGenerated
  • AccountCustomEntity
    • This is generally the Account, AccountName, UserPrincipalName, UserId property depending on the datatype
    • When those are not available, you can substitute SID, AADUserId or other strong identifiers
    • Extracted field is mapped to Account.Name in the standardized Alert Schema
  • HostCustomEntity
    • This is generally the Host, Computer, System, Device property depending on the datatype
    • Extracted field is mapped to Host.HostName in the standardized Alert Schema
  • IPCustomEntity
    • This is generally the IP, IPAddress, ClientIP, RemoteIP, DestinationIP property depending on the datatype
    • Extracted field is mapped as IP.Address

At the end of your query, use the following syntax to map the entities for easy Rule creation, example:

| extend timestamp = TimeGenerated, HostCustomEntity = Computer, AccountCustomEntity = UserPrincipalName, IPCustomEntity = RemoteIP

Full Examples

https://github.com/Azure/Azure-Sentinel/blob/master/Detections/SigninLogs/FailedLogonToAzurePortal.yaml https://github.com/Azure/Azure-Sentinel/blob/master/Hunting%20Queries/SecurityEvent/ProcessEntropy.yaml

Clone this wiki locally