SourcesForwardersLogstash

Logstash Log Forwarder

Logstash can be configured to send data to RunReveal using the HTTP output plugin. It collects, parses, filters, and forwards logs to external systems—ideal for streaming security and application events into RunReveal.

Quick Start

This guide mirrors a typical production setup. It uses absolute paths, environment variables for secrets, and retry/timeouts. Optional persistent queues are included for durability.

Step 1: Create a Webhook Source in RunReveal

  1. Navigate to RunReveal: Go to your RunReveal dashboard
  2. Create Source: Click on “Sources” in the left sidebar
  3. Add Webhook: Create a source for “Structured Webhook or Generic Webhook” based on your data format.
  4. Configure: Give your webhook a name and description
  5. Copy URL: Copy the generated webhook URL to use in your configuration. (It’s always available after you save the source)

Step 2: Install Logstash

# Install (macOS)
brew install logstash
 
# Verify
logstash --version

Step 3: Create a Test Log (RunReveal Schema)

Create a JSON Lines file that matches RunReveal’s ** Webhooks** schema (fields like eventName, eventTime, actor, src, service).

sudo mkdir -p /var/log/app
sudo tee /var/log/app/application.log > /dev/null << 'EOF'
{"eventName":"UserLogin","eventTime":"2025-01-15T10:30:00Z","actor":{"email":"[email protected]","id":"USR001"},"service":{"name":"webapp"},"src":{"ip":"192.168.1.100"}}
{"eventName":"FileAccess","eventTime":"2025-01-15T10:31:00Z","actor":{"email":"[email protected]","id":"USR002"},"service":{"name":"webapp"},"src":{"ip":"192.168.1.101"}}
{"eventName":"DataExport","eventTime":"2025-01-15T10:32:00Z","actor":{"email":"[email protected]","id":"USR003"},"service":{"name":"webapp"},"src":{"ip":"192.168.1.102"}}
EOF
sudo chmod 644 /var/log/app/application.log

If you already have application logs, you can also transform them into this schema with Logstash filters. See the “Optional: Transform JSON to structured log format” section below.

Step 4: Configure Logstash

Use absolute paths, env vars for secrets, and retry/timeouts. Choose one of the following tabs.

Basic Configuration — No Authentication

# Set environment variable for your  Webhook URL
export YOUR_WEBHOOK_URL="https://api.runreveal.com/sources/webhook/YOUR__WEBHOOK_ID"
 
# Write the pipeline config
cat > ~/.config/logstash/logstash.conf << 'EOF'
input {
  file {
    path => "/var/log/app/application.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
 
filter {
  json { source => "message" }
}
 
output {
  http {
    url => "${YOUR_WEBHOOK_URL}"          # set as environment variable
    http_method => "post"
    format => "json"                      # sends the entire event as JSON
    content_type => "application/json"
    retry_non_idempotent => true          # retry on 5xx/network errors
    request_timeout => 30                 # seconds
    # connect_timeout => 10               # (optional) if supported by your plugin version
    # pool_max => 50                      # (optional) connection pooling
  }
}
EOF

Notes:

  • Use absolute paths for inputs (/var/log/...).
  • Never hardcode secrets; prefer environment variables ($\{YOUR_BEARER_TOKEN\}).
  • Retries and timeouts improve resilience in production.
  • If your environment proxies outbound traffic, set HTTP_PROXY/HTTPS_PROXY accordingly.

Step 5: (Optional) Enable Persistent Queues

Persistent queues protect against data loss during outages. Configure them in your Logstash settings file (not the pipeline). On many systems this is /etc/logstash/logstash.yml. For user-scoped installs, use $LS_SETTINGS_DIR/logstash.yml.

# /etc/logstash/logstash.yml
queue.type: persisted
path.queue: /var/lib/logstash/queue
queue.max_bytes: 1024mb
dead_letter_queue.enable: false

Restart the service after changing settings:

sudo systemctl restart logstash

If you run Logstash manually (not as a service), set LS_SETTINGS_DIR to point to a folder containing your logstash.yml.

Step 6: Validate & Start Logstash

Validate:

logstash -f ~/.config/logstash/logstash.conf --config.test_and_exit

Start:

logstash -f ~/.config/logstash/logstash.conf

The file input tails new lines. Append to the application log file to trigger ingest:

echo '{"eventName":"AdminAction","eventTime":"2025-01-15T10:33:00Z","actor":{"email":"[email protected]","id":"USR001"},"service":{"name":"webapp"},"src":{"ip":"192.168.1.100"}}' | sudo tee -a /var/log/app/application.log

Step 7: Verify Delivery in RunReveal

  1. Open Sources → your ** Webhook** source card
  2. Confirm the Last Event time updates
  3. Explore logs in Explorer. A starter query:
SELECT *
FROM {table:Identifier}
WHERE ({tsColumn:Identifier} >= now() - INTERVAL 1 HOUR)
  AND ({tsColumn:Identifier} < now())
  AND (sourceID = 'WEBHOOK_SOURCE_ID')
LIMIT 10

Optional: Transform JSON to structured log format

If your existing app logs don’t already match the schema, transform them in Logstash.

Example json log to remap

sudo tee /var/log/app/app.log > /dev/null << 'EOF'
{"action":"login","ts":"2025-08-27T19:02:00Z","user":"bob","user_email":"[email protected]","client_ip":"192.0.2.55","svc":"billing","ro":false}
{"action":"export","ts":"2025-08-27T19:03:00Z","user":"carol","user_email":"[email protected]","client_ip":"192.0.2.56","svc":"billing","ro":true}
EOF

Mapping pipeline → schema

cat > ~/.config/logstash/logstash.conf << 'EOF'
input {
  file {
    path => "/var/log/app/app.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  json { source => "message" }
 
  mutate {
    rename => {
      "[ts]"        "eventTime"
      "[action]"    "eventName"
    }
    add_field => {
      "[service][name]" "billing"
      "[tags][source]"  "logstash"
    }
  }
 
  mutate {
    add_field => {
      "[actor][username]" "%{[user]}"
      "[actor][email]"    "%{[user_email]}"
      "[src][ip]"         "%{[client_ip]}"
    }
  }
 
  mutate {
    rename => { "[ro]" "readOnly" }
    convert => { "readOnly" => "boolean" }
  }
 
  mutate {
    remove_field => ["message","host","@version","user","user_email","client_ip"]
  }
}
output {
  http {
    url => "${YOUR_WEBHOOK_URL}"
    http_method => "post"
    format => "json"
    content_type => "application/json"
    retry_non_idempotent => true
    request_timeout => 30
  }
}
EOF

Restart or re-run Logstash with this mapping to produce Webhook events for RunReveal.

Running as a Service (systemd)

For production, place your pipeline config in /etc/logstash/conf.d/runreveal.conf and manage via systemd:

sudo tee /etc/logstash/conf.d/runreveal.conf > /dev/null << 'EOF'
input {
  file {
    path => "/var/log/app/application.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter { json { source => "message" } }
output {
  http {
    url => "${YOUR_WEBHOOK_URL}"
    http_method => "post"
    format => "json"
    content_type => "application/json"
    retry_non_idempotent => true
    request_timeout => 30
  }
}
EOF
 
sudo systemctl restart logstash
sudo systemctl status logstash --no-pager

Export your environment variables in a file sourced by the service (e.g., /etc/sysconfig/logstash or /etc/default/logstash) depending on your distro/service wrapper, or use the native mechanism for passing env vars to the Logstash service.

Troubleshooting

No configuration found

  • Use absolute path with -f or cd into the directory containing logstash.conf
  • Avoid mixing -f and pipelines.yml unless intended

No new events read

  • The file input tails; append new lines or use sincedb_path => "/dev/null" for test reruns
  • Check file permissions and SELinux/AppArmor if applicable

HTTP/SSL errors

  • Verify the webhook URL and that outbound egress is allowed
  • For custom CAs, configure JVM truststore env or plugin SSL settings

Check pipeline health

curl -s http://localhost:9600/_node/pipelines?pretty

Debug run

logstash -f ~/.config/logstash/logstash.conf --log.level debug

Resources

DatadogTransforms