Log Transforms

Transforms convert raw log data into RunReveal's normalized schema. Each transform contains one or more rules. Each rule is a linear chain of processors that ends with Output to Field.

How processing works

Understanding what each step receives as input is the most important part of writing transforms.

1. Every rule starts from the raw log — the full event string as ingested (usually JSON).
2. Processors run in order — the output of step N becomes the input of step N+1.
3. The final processor is always Output to Field — it writes the pipeline result to one normalized field (actor.email, eventTime, tags, etc.).
4. Rules are independent — rule 2 does not see the output of rule 1; it also starts from the raw log.

Example: single-rule flow

Raw log (sample event):

{"userGivenName":"John","userFamilyName":"Doe","eventName":"user.login"}

Rule: map eventName → eventName

Available processors

The tables below use a shared sample where helpful. Step input is what the processor receives; Step output is what it passes to the next step.

Extract JSON Field (gjson)

Extracts a value from JSON using GJSON path syntax.

Config: GJSON Path (for example user.email, data.items.0.id, users.#.name)

Extract Delimited Field (field)

Splits delimited text and returns the field at a zero-based index. Unlike Split String, this processor can respect quoted fields that contain the delimiter.

Config: Delimiter, Field Index, Quote Handling (none, single ', or double ")

Extract with Regex (regex)

Matches the input against a regex pattern and returns one capture group (or the full match if no group is selected).

Config: Regex Pattern, Capture Group (number or name), optional Match index for multiple matches

Add Prefix (prefix)

Prepends a static string.

Config: Prefix

Add Suffix (suffix)

Appends a static string.

Config: Suffix

Convert to Lowercase (lowercase)

Config: none

Convert to Uppercase (uppercase)

Config: none

Regex Find/Replace (regexreplace)

Finds matches in the input and replaces them. Supports Go regexp capture references in the replacement string: $1, $2, etc.

Config: Find Pattern, Replace With

Join two JSON fields into one value (for example actor.username):

The delimiter between $1 and $2 is whatever you put in Replace With — a space, underscore, @, etc.

Strip Characters (strip)

Removes every occurrence of the specified characters from the input.

Config: Characters to Strip

Trim Whitespace (trim)

Removes leading and trailing whitespace (spaces, tabs, newlines).

Config: none

Extract Substring (substring)

Returns a portion of the string by start/end index. Indices can be negative (count from the end). If end is omitted, it defaults to the end of the string.

Config: Start Index (optional), End Index (optional)

Split String (split)

Splits on a delimiter and returns the segment at the given index. Supports negative indices.

Config: Delimiter, Index to Take

Format Date/Time (datetimeformat)

Parses a timestamp string and normalizes it to RFC3339 (for example 2021-01-01T00:00:00Z). Use before Output to Field → eventTime.

Config: Input Format

Custom format tokens: YYYY, MM, DD, HH, hh, mm, ss, SSS, Z.

Output to Field (output)

Final step in every rule. Passes the pipeline value through unchanged and writes it to a normalized field when the transform is applied.

Config: Field Path (and Tag Key when field is tags)

Available output fields: id, eventName, eventTime, readOnly, actor.id, actor.email, actor.username, src.ip, src.port, dst.ip, dst.port, service.name, resources, tags.

Each normalized field (except tags and resources) can only be targeted by one rule per transform. Use tags with different keys for additional custom values.

Full pipeline examples

Map a JSON field to actor.email

Raw log:

{"userEmail":"[email protected]","action":"login"}

Parse a CSV-style log line to eventName

Raw log:

2024-01-15,user.login,[email protected]

Concatenate userGivenName + userFamilyName → actor.username

Raw log:

{"userGivenName":"John","userFamilyName":"Doe"}

Unix timestamp → eventTime

Raw log:

{"ts":"1609459200","event":"login"}

Building an effective pipeline

1. Extract — Use Extract JSON Field, Extract with Regex, or Extract Delimited Field on the raw log.
2. Clean and format — Trim, case conversion, strip unwanted characters, substring/split.
3. Transform dates — Format Date/Time before writing to eventTime.
4. Map to schema — Output to Field.

Add and test one processor at a time using Test Rule in the transform editor.

Testing your transform

1. Paste a representative event into Sample Event.
2. Click Test Rule to run the current rule. Each step shows its output; errors appear in red (for example invalid JSON input).
3. Click View Normalized Event to see the full normalized event after all rules run.

Troubleshooting

Related documentation

• Custom Source field mapping — wizard-based mapping that creates transforms automatically
• Sources — attach transforms to sources during setup