How To Write Detections

There isn’t a single correct way to write detections, but following these guidelines will help you create more efficient and effective detections.

Query Detections (SQL)

When using the explore page and creating a detection, you’ll notice that the generated query has {from:DateTime} and {to:DateTime} in it. These are parameters, and they are a ClickHouse-specific syntax that you can read more about in their documentation:

https://clickhouse.com/docs/en/sql-reference/syntax#defining-and-using-query-parameters

RunReveal passes the following parameters to each of your detections every time they run to help with windowing your data:

from - The start time of the current data window
to - The end time of the current data window

By including these parameters in your detection, you can support effective windowing without needing to worry about lost data, delayed data, etc. By using the receivedAt field from the RunReveal log schema, you automatically handle situations where your services generate logs that arrive with a delay (for example, CloudTrail) - in addition, this is an indexed field, making searches faster.

For example:

SELECT 
    *
FROM aws_cloudtrail_logs
WHERE
    eventName='Decrypt' AND
    receivedAt >= {from:DateTime} AND
    receivedAt < {to:DateTime}

Defining Your Own Parameters

You can also define your own parameters for useful omissions, inclusions, etc. Several of RunReveal’s pre-created detections contain custom parameters. For example, to create a detection that excludes your list of office IP addresses, you can create a SQL detection like this:

SELECT * from runreveal_logs
WHERE eventName = 'thing-that-should-only-be-done-in-office'
AND srcIP not in ({officeIPs:Array(String)})

Then, define the officeIPs parameter to include each of your office IPs:

1.2.3.4, 2.3.4.5, 4.3.2.1

Context

It is recommended to return as much context as possible in your query detections. Although returning small portions may seem like a good idea, returning as much context as you can from the log makes it easier to perform a full analysis on the alerts.

We recommend your SELECT statements to look something like:

SELECT *,
    JSONExtractString(property, 'sub-property') as my_property
    ...
...

This will pull all normalized fields from the event, as well as fields from nested properties that you want to surface to the top level.

Additionally, if you are returning any custom data in your detection, ensure you are returning the fields values instead of objects.

SELECT *,
        actor,          # --> Don't
        actor.email     # --> Do
...

Streaming Detections (Sigma)

Specify SourceTypes

Even though the engine will accept detections without a list of sourcetypes or logsource, detections should list whichever sources they are covering.

Detections without a list of sources will be less performant, will run against more than necessary events and will be harder to reproduce using the built-in validators.

Your detection configuration should contain one of:

sourcetypes:
- okta

logsource:
  product: aws
  service: guardduty

Detections with GJSON

Although Sigma does not support this out-of-the-box, we added GJSON support in query selectors.

This allows complex queries, logical selectors, array selectors, and much more. Here are some examples of selectors you can write for the following sample input:

{
  "field1": "value1",
  "simpleArray": ["a", "b", "c"],
  "complexArray": [
    {
      "caKey": "key1",
      "caInt": 10,
      "caField": "caValue1"
    },
    {
      "caKey": "key2",
      "caInt": 20,
      "caField": "caValue2"
    }
  ]
}

detection:
    selection:
        rawLog.complexArray.#(caKey=="key1").caField=="caValue1": true
        rawLog.complexArray.#.caField: "caValue2"
        rawLog.simpleArray: "a"
        complexArray.#(caInt<15): true

Targeting Values

There are multiple fields that can be targeted in a detection. The following is a non-exhaustive list of possibilities:

Note: If you are using legacy fields prefixed with normalized, there is backwards compatibility to still find those fields.

Event fields

id
receivedAt
workspaceID
sourceType
sourceName
sourceID
eventID
eventName
eventTime
readOnly
serviceName

Actor

actor.id
actor.email
actor.username

Enrichments

enrichments.<provider>.<key>

[
  {
    "data": {
      "vpn-abuse": "true"
    },
    "name": "normalized.src.ip",
    "provider": "ip_threat_intel",
    "type": "custom",
    "value": "37.19.210.0/23"
  }
]

enrichments.ip_threat_intel.vpn-abuse: true

RawLog

All other fields that are not normalized in the processing phase will end up in the rawLog, which can also be used to query data.

rawLog.<field>
<field>

Resources

resources.<index>.<field>

Important Fields

RunReveal supports a property in the detection configs called fields which allows highlighting fields that are relevant to the analysts triaging the alerts.

These will get pulled to the top of the alert table view, as long as there is a match.

fields:
    - actor.name
    - events.#.parameters(name=="value")
    - srcIP

The above example demonstrates a few of the possibilities that this property allows. It supports GJSON as well, meaning you can write complex logic in the important fields to surface deeply nested, non-normalized data from the logs.

Fields Sigma Demo

AI-Powered Detection Writing

RunReveal’s AI chat can help you write and optimize detections. Use the native AI chat in the UI to:

Generate SQL queries from natural language descriptions
Convert Sigma rules to RunReveal’s streaming detection format
Optimize existing detections for better performance
Debug detection logic and troubleshoot issues
Suggest field mappings and parameter configurations

Simply describe what you want to detect in plain English, drop in existing detections for review, or ask for for the chat to review logs from x source and suggest relevant detections. Learn more about using our Native AI Chat or the Model Context Protocol (MCP) integration for advanced AI assistance with your detection development workflow.

MITRE ATT&CK Coverage

You can use AI chat to generate detections for specific MITRE ATT&CK tactics and techniques. When you provide context that you’re looking to add MITRE tactic coverage, the AI can suggest relevant detections based on your actual log data.

Example query:

Review logs from my github source for the past day and suggest detections for the most critical MITRE tactics

The AI will analyze your GitHub logs and suggest detections for high-priority MITRE tactics like:

Initial Access (T1078 - Valid Accounts)
Persistence (T1098 - Account Manipulation)
Privilege Escalation (T1078 - Valid Accounts)
Defense Evasion (T1070 - Indicator Removal)
Credential Access (T1552 - Unsecured Credentials)

AI Chat MITRE Detections

AI Chat MITRE Detections Results

Note: The suggested detections may need some tweaking and can be tested on the detections creation page or explore tab, but they provide a solid starting point when you’re looking to add coverage for specific sources and MITRE tactics.

Helpful Resources

Now that you understand how to write effective detections, explore these related guides:

Detections, Signals & Alerts Quick Start Guide - Complete setup guide for your detection workflow
Sigma Streaming - Use Sigma rules for standardized threat detection
Detection as Code - Manage detections through code and version control
Notifications - Set up alerting and notification channels
Native AI Chat - Use AI to help write and optimize detections
Model Context Protocol - Advanced AI integration for detection development

Sigma Streaming Enrichments