SPIFFE/mTLS Authentication

SPIFFE (Secure Production Identity Framework for Everyone) provides a standardized way to issue and manage service identities using X.509 certificates. RunReveal supports SPIFFE-based mutual TLS (mTLS) authentication for ClickHouse connections, enabling secure service-to-service communication without passwords.

Why Use SPIFFE/mTLS?

Enhanced Security: Certificates are more secure than static passwords and can be automatically rotated
Zero Trust Architecture: Every connection is authenticated with cryptographic proof of identity
Kubernetes Native: Works seamlessly with SPIFFE implementations like SPIRE in Kubernetes environments
Automatic Rotation: Certificates can be rotated without service restarts

SPIFFE/mTLS is recommended for production BYOC deployments where ClickHouse runs in the same network as RunReveal services.

Supported Destinations

SPIFFE authentication works with both:

Default ClickHouse - The global/shared ClickHouse cluster configured via environment
Custom Destinations (BYODB) - Per-workspace ClickHouse instances configured in destination settings

For BYODB destinations, enable SPIFFE by setting useSPIFFE: true in the destination configuration.

Configuration

Enable SPIFFE authentication by setting the following configuration options:

{
  "clickhouse": {
    "spiffe": {
      "defaultDestEnabled": true,
      "certPath": "/etc/identity/tls.crt",
      "keyPath": "/etc/identity/tls.key",
      "caCertPath": "/etc/identity/ca.crt",
      "refreshInterval": "10m"
    }
  }
}

Configuration Options

Option	Description	Default
`defaultDestEnabled`	Enable SPIFFE for the default/multi-tenant ClickHouse destination	`false`
`certPath`	Path to the client certificate file	`/etc/identity/tls.crt`
`keyPath`	Path to the client private key file	`/etc/identity/tls.key`
`caCertPath`	Path to the CA certificate for server verification	(none)
`refreshInterval`	How often to check for certificate updates	`10m`

When SPIFFE is enabled, the certificate Common Name (CN) or the X-ClickHouse-User header is used for ClickHouse authentication. Ensure your ClickHouse user configuration matches.

Certificate Requirements

Your SPIFFE certificates must meet these requirements:

Format: PEM-encoded X.509 certificates
Key Usage: Client authentication (clientAuth extended key usage)
Common Name: Must match the ClickHouse username you want to authenticate as
Validity: Certificate must be valid (not expired)

Example Certificate Structure

/etc/identity/
├── tls.crt    # Client certificate (PEM format)
└── tls.key    # Private key (PEM format)

ClickHouse Server Configuration

Configure your ClickHouse server to accept certificate-based authentication:

Enable HTTPS

Add SSL configuration to your ClickHouse config.xml:

<clickhouse>
    <https_port>8443</https_port>
    <openSSL>
        <server>
            <certificateFile>/etc/clickhouse-server/certs/server.crt</certificateFile>
            <privateKeyFile>/etc/clickhouse-server/certs/server.key</privateKeyFile>
            <caConfig>/etc/clickhouse-server/certs/ca.crt</caConfig>
            <verificationMode>relaxed</verificationMode>
            <loadDefaultCAFile>false</loadDefaultCAFile>
        </server>
    </openSSL>
</clickhouse>

Create Certificate-Authenticated User

Add a user that authenticates via certificate CN in users.xml:

<clickhouse>
    <users>
        <runreveal_service>
            <ssl_certificates>
                <common_name>runreveal-service</common_name>
            </ssl_certificates>
            <networks>
                <ip>::/0</ip>
            </networks>
            <profile>default</profile>
            <quota>default</quota>
            <access_management>0</access_management>
        </runreveal_service>
    </users>
</clickhouse>

Verify Configuration

Test the connection using curl:

curl --cert /etc/identity/tls.crt \
     --key /etc/identity/tls.key \
     --cacert /etc/clickhouse-server/certs/ca.crt \
     -H "X-ClickHouse-SSL-Certificate-Auth: on" \
     -H "X-ClickHouse-User: runreveal_service" \
     "https://clickhouse:8443/?query=SELECT%201"

Certificate Rotation

RunReveal automatically watches for certificate changes and reloads them without requiring a restart. The refreshInterval setting controls how often the certificate files are checked for updates.

When certificates are rotated:

Your SPIFFE provider (e.g., SPIRE) writes new certificates to the configured paths
RunReveal detects the file change within the refreshInterval period
New connections use the updated certificates automatically
Existing connections continue until they're recycled by the connection pool

Set refreshInterval to match your certificate rotation frequency. For short-lived certificates (common in SPIRE), use a shorter interval like 1m.

Kubernetes with SPIRE

If you're using SPIRE in Kubernetes, configure the SPIRE agent to project certificates to your RunReveal pods:

apiVersion: v1
kind: Pod
metadata:
  name: runreveal-api
spec:
  containers:
  - name: api
    volumeMounts:
    - name: spiffe-workload-api
      mountPath: /etc/identity
      readOnly: true
  volumes:
  - name: spiffe-workload-api
    csi:
      driver: "csi.spiffe.io"
      readOnly: true

Troubleshooting

Connection refused on HTTPS port

Verify ClickHouse is listening on the HTTPS port: netstat -tlnp | grep 8443
Check ClickHouse logs for SSL initialization errors
Ensure the server certificate and key are readable by ClickHouse

Debug Logging

Enable debug logging to troubleshoot certificate issues:

{
  "logging": {
    "level": "debug"
  }
}

Look for log messages containing "SPIFFE" or "watching SPIFFE certificates" to verify the certificate watcher is running.

Deployment Guide - Setting up RunReveal in your environment
ClickHouse Configuration - ClickHouse SSL documentation
SPIFFE Documentation - Learn more about SPIFFE

On this page