Custom Views

Custom Views allow you to create virtual tables that extract specific fields from your log data using JSON path expressions. This feature enables you to organize and analyze your log data with custom columns tailored to your specific use cases, without modifying the underlying data structure.

⚠️

Feature Flag Required: Custom Views is currently gated behind a feature flag. Please reach out to RunReveal support to request access to this feature.

Getting Started: Navigate to Settings → Custom Views to create your first custom view, or use the CLI and API for programmatic access.

Custom View

Accessing Your Custom View: Once created, your custom view table will be available in the Search Explorer page where you can query it directly using SQL.

Key Features

Virtual Tables: Create queryable views without data transformation
JSON Path Extraction: Extract nested fields from your raw logs
Source Filtering: Views automatically filter to specific sources
Real-time Analysis: Query-time extraction for immediate insights
Flexible Schema: Add custom columns for any JSON field in your logs

Architecture

Creating Custom Views

Via Web UI

1) Navigate to Custom Views

Go to Settings → Custom Views and click “Create Custom View”.

2) Configure Basic Information

Fill in the basic information:

Name: Descriptive name for your view
Description: Optional description of the view’s purpose
Source: Select the source to filter logs from

3) Add Custom Columns

Add custom columns with:

Column Name: Name for the extracted field
Type: ClickHouse data type (String, Int64, Bool, etc.)
SQL Expression: JSON path to extract the field
Description: Optional description of the column

4) Test and Save

Test your view with sample data, then save your configuration.

Column Configuration

Supported Data Types

Type	Description	Example
`String`	Text data	`"[email protected]"`
`Int8/16/32/64`	Signed integers	`12345`
`UInt8/16/32/64`	Unsigned integers	`4294967295`
`Float32/64`	Floating point numbers	`3.14159`
`Bool`	Boolean value	`true`
`DateTime/DateTime64`	Date and time	`2025-01-15 10:30:00`
`Date/Date32`	Date only	`2025-01-15`
`Array(String)`	Array of strings	`["tag1", "tag2"]`
`Nullable(String)`	Nullable string	`null` or `"value"`
`LowCardinality(String)`	Memory-optimized string	`"frequent_value"`

JSON Path Expressions

Custom views support simple JSON path expressions to extract nested fields:

Basic Syntax

field.subfield.array[0].nested_field

Examples

{
  "user": {
    "profile": {
      "email": "[email protected]"
    }
  },
  "tags": ["security", "alert"],
  "metadata": {
    "source_ip": "192.168.1.100"
  }
}

Valid JSON Paths:

user.profile.email → "[email protected]"
tags[0] → "security"
metadata.source_ip → "192.168.1.100"

Limitations

Simple Paths: Direct field access and array indexing only
No Complex Logic: Cannot use conditional statements in path expressions
ClickHouse Functions: Can use ClickHouse functions in SQL expressions
Path Length: Maximum 256 characters

Example: Application Security Custom View

Use Case

Extract important security fields from application logs for analysis and monitoring.

Sample Raw Log

{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "INFO",
  "service": "auth-service",
  "user": {
    "id": "12345",
    "email": "[email protected]",
    "role": "admin"
  },
  "request": {
    "method": "POST",
    "path": "/api/v1/login",
    "ip": "192.168.1.100",
    "user_agent": "Mozilla/5.0..."
  },
  "response": {
    "status": 200,
    "duration_ms": 150
  },
  "security": {
    "risk_score": 0.2,
    "threats": ["none"],
    "location": {
      "country": "US",
      "city": "San Francisco"
    }
  }
}

Configuration

Basic Information:

Name: app_security_analysis
Description: Application security events analysis
Source: Your application source

Custom Columns:

[
  {
    "name": "user_email",
    "description": "User email address",
    "type": "String",
    "sql": "user.email"
  },
  {
    "name": "user_role",
    "description": "User role or permission level",
    "type": "String",
    "sql": "user.role"
  },
  {
    "name": "request_ip",
    "description": "Client IP address",
    "type": "String",
    "sql": "request.ip"
  },
  {
    "name": "response_status",
    "description": "HTTP response status code",
    "type": "Int64",
    "sql": "response.status"
  },
  {
    "name": "response_duration",
    "description": "Request duration in milliseconds",
    "type": "Int64",
    "sql": "response.duration_ms"
  },
  {
    "name": "risk_score",
    "description": "Security risk score (0-1)",
    "type": "Float64",
    "sql": "security.risk_score"
  },
  {
    "name": "is_high_risk",
    "description": "Whether the event is high risk",
    "type": "Bool",
    "sql": "security.risk_score > 0.7"
  },
  {
    "name": "country",
    "description": "User's country",
    "type": "String",
    "sql": "security.location.country"
  }
]

Querying Custom Views

Querying Custom View Columns: You do not have to target the rawLog.jsonfield name in your query. You can just target the jsonfield name that you created in your custom view. For example, if you created a column named user_email that extracts user.email from the raw log, you can query it directly as user_email without referencing rawLog or logJson.

View Naming Convention

The workspace name prefix is automatically added to your custom view name.

Example: If your workspace is named workspace and you create a view named app_security_analysis, it will be accessible as:

SELECT * FROM workspace_app_security_analysis LIMIT 10;

The full view name format is: {workspace_name}_{your_view_name}

Basic Queries

-- Get all records from the custom view
SELECT * FROM workspace_app_security_analysis LIMIT 10;
 
-- Get specific columns
SELECT 
  receivedAt,
  user_email,
  user_role,
  request_ip,
  response_status
FROM workspace_app_security_analysis
ORDER BY receivedAt DESC;

Security Analysis Queries

-- Find high-risk events
SELECT 
  receivedAt,
  user_email,
  user_role,
  request_ip,
  risk_score,
  country
FROM workspace_app_security_analysis
WHERE is_high_risk = true
ORDER BY receivedAt DESC
LIMIT 100;
 
-- Find failed authentication attempts
SELECT 
  receivedAt,
  user_email,
  request_ip,
  response_status,
  country
FROM workspace_app_security_analysis
WHERE response_status = 401
ORDER BY receivedAt DESC;
 
-- Find admin users from unusual locations
SELECT 
  receivedAt,
  user_email,
  request_ip,
  country,
  risk_score
FROM workspace_app_security_analysis
WHERE user_role = 'admin'
  AND country NOT IN ('US', 'CA', 'GB')
ORDER BY risk_score DESC;

Combining with Transforms

Best Practice: Hybrid Approach

Use transforms for standard field mapping and custom views for source-specific analysis:

Transform (Data Normalization)

{
  "processors": [
    {
      "type": "field_mapper",
      "config": {
        "mappings": [
          {
            "source": "user_email",
            "target": "actor.email"
          },
          {
            "source": "request_ip",
            "target": "srcIP"
          },
          {
            "source": "user_role",
            "target": "actor.role"
          },
          {
            "source": "response_status",
            "target": "eventName"
          }
        ]
      }
    }
  ]
}

Benefits

Standard Fields: Work with existing RunReveal detections
Custom Fields: Enable advanced source-specific analysis
Flexibility: Query either standard table or custom view as needed

Using Custom Views in Detections

Custom views can be used in SQL detections but not in Sigma streaming detections. When creating SQL detections, you can query your custom view directly.

SQL Detection Example

Here’s an example SQL detection that uses a custom view:

SELECT 
  receivedAt,
  user_email,
  user_role,
  request_ip,
  risk_score,
  country
FROM workspace_app_security_analysis
WHERE is_high_risk = true
  AND user_role = 'admin'
  AND country IN ('CN', 'RU', 'KP')
ORDER BY receivedAt DESC

Note: Replace workspace_app_security_analysis with your actual view name (workspace prefix + view name). See View Naming Convention for details.

Sigma Detection (Not Supported)

Due to the nature of Sigma (Streaming Detections) Custom views cannot be used in Sigma detections.

Troubleshooting

Common Issues

”Invalid JSON path” Error

Cause: JSON path syntax is incorrect
Solution: Use simple dot notation: field.subfield.array[0]

”No rows in result set”

Cause: No logs match the source filter or time range
Solution: Check source is active and has recent logs

”Column not found” Error

Cause: Column name doesn’t exist in the view
Solution: Verify column names match the custom view definition

Next Steps

Now that you understand custom views, explore these related features:

Transforms: For data normalization and field mapping
Detections: For alerting on custom view data
Writing Detections: Learn how to write detection rules
Dashboards: For visualizing your custom view data

Transforms Filtering