Google Cloud Platform (GCP) Logs
GCP logs provide comprehensive monitoring and logging across all Google Cloud services through Cloud Logging. These logs capture various types of data, including system events, API calls, network traffic, resource access, and performance metrics. GCP logs are essential for tracking user activity, monitoring infrastructure, troubleshooting issues, auditing security and compliance, and optimizing the performance of Google Cloud resources.
Ingest Methods
GCP Logs can be ingested using the GCS object storage method as well as setting up a webhook to receive events.
GCS buckets are inherently cheaper than using the webhook method but logs can be delayed by up to an hour. The webhook ingestion imports logs as soon as they are generated, but using pub/sub to forward every event can become more expensive if there are lots of logs.
- Google Cloud Storage
- RunReveal Webhook
After creating your cloud storage bucket and other resources needed to receive events, you will need to setup a GCP log router to forward logs to the bucket.
GCP Log Router Sink Setup
Navigate to the GCP Log router setup page to create a new sink.
You can setup a logging sink for a single project or your entire organization and forward logs to the bucket created.
Create a new logging sink giving it a name, choose Cloud Storage bucket as the sink service. Enter the bucket information that was created as the destination bucket.
Add an optional inclusion or exclusion filter to limit the logs that are forwarded by this logging sink.
When routing logs directly to cloud storage, Google will batch the logs and write them to a file every hour. It may take some time before you see any logs start to show up in the bucket.
Schema
The following columns are exposed for this source. RunReveal applies schema normalization across all sources, ensuring uniform field names and data types for cross-source queries and reusable detection logic.
Table: gcp_logs (60 columns)
gcp_logs (60 columns)| Column | Type |
|---|---|
workspaceID | String |
sourceID | String |
sourceType | LowCardinality(String) |
sourceTTL | UInt32 |
receivedAt | DateTime |
id | String |
eventTime | DateTime |
eventName | String |
eventID | String |
srcIP | String |
srcASCountryCode | String |
srcASNumber | UInt32 |
srcASOrganization | String |
srcCity | String |
srcConnectionType | String |
srcISP | String |
srcLatitude | Float64 |
srcLongitude | Float64 |
srcUserType | String |
dstIP | String |
dstASCountryCode | String |
dstASNumber | UInt32 |
dstASOrganization | String |
dstCity | String |
dstConnectionType | String |
dstISP | String |
dstLatitude | Float64 |
dstLongitude | Float64 |
dstUserType | String |
actor | Map(String, String) |
| Column | Type |
|---|---|
tags | Map(String, String) |
resources | Array(String) |
serviceName | String |
enrichments | Array(Tuple(data Map(String, String), name String, provider String, type String, value String)) |
readOnly | Bool |
rawLog | String |
insertId | String |
logName | String |
textPayload | String |
payloadType | String |
methodName | String |
resourceName | String |
payloadServiceName | String |
severity | String |
resourceType | String |
principalEmail | String |
principalSubject | String |
authorizationInfo | Array(String) |
projectID | String |
subscriptionID | String |
resourceLabels | Array(Tuple(String, String)) |
callerIp | String |
callerSuppliedUserAgent | String |
requestPayload | String |
responsePayload | String |
payloadMetadataType | String |
metadataEvents | Array(String) |
bindingDeltas | Array(String) |
jsonPayloadMessage | String |
jsonPayloadPID | String |