Exploring Google Cloud’s Log Analytics: A Must-Have Tool for Cloud Security Professionals

Asjad Nasir

·

Follow

4 min read

·

Mar 11, 2023

--

1

Listen

Share

If you’re a cloud security professional, you know how crucial it is to have access to detailed logs of events and activities within your cloud environment. Without proper logging and analysis, it’s challenging to identify potential security threats, troubleshoot issues, and ensure compliance with regulations. That’s where Log Analytics comes in.

Log Analytics is a powerful feature within Google Cloud’s Cloud Logging service that allows you to search, aggregate, and transform all types of log data, including application, network, and audit log data. By leveraging BigQuery, Log Analytics breaks down data silos, helping security teams run analytics using a single copy of data. Let’s look at how Log Analytics can help with Cloud Security.

When to use Log Analytics

For the most comprehensive security use cases, opting for out-of-the-box security solutions like Security Command Center or Palo Alto Prisma Cloud is recommended. A SIEM solution such as Chronicle or Splunk is ideal for effectively processing large security logs. However, Log Analytics can provide additional value in specific scenarios, such as identifying trends and patterns in log data that may not be detectable by traditional security tools and allowing for ad-hoc queries and analysis of log data for environment-specific use cases where deviations from expected behavior can indicate security threats.

How to use Log Analytics

There are two simple steps to get started with Log Analytics.

1. Upgrade Storage Bucket to use Log Analytics

You can upgrade log buckets to use Log Analytics. You can also use BigQuery to view the data stored in a log bucket when the log bucket is upgraded to use Log Analytics and when a linked BigQuery dataset exists. Creating a linked dataset lets you join your log data with other data stored in BigQuery and access data from tools like Looker Studio and Looker.

To create a log bucket and upgrade the log bucket to use Log Analytics, you must use the Google Cloud console, the Google Cloud CLI, or the Cloud Logging API. Not all regions are supported for Log Analytics. For more information, see Supported regions for Log Analytics.

To upgrade an existing log bucket to use Log Analytics, See Upgrade a bucket to use Log Analytics

2. Run queries using SQL

You can query logs in these buckets using SQL, which lets you filter and aggregate your logs. For more information, See Build queries using SQL

Here are five SQL queries that can be used to analyze log data in Log Analytics for cloud security

1. Permissions granted to impersonate a service account.

SELECT
  timestamp,
  proto_payload.audit_log.authentication_info.principal_email as grantor,
  JSON_VALUE(bindingDelta.member) as grantee,
  JSON_VALUE(bindingDelta.role) as role,
  proto_payload.audit_log.resource_name,
  proto_payload.audit_log.method_name
FROM
  `[MY_PROJECT_ID].[MY_DATASET_ID]._AllLogs`,
  UNNEST(JSON_QUERY_ARRAY(proto_payload.audit_log.service_data.policyDelta.bindingDeltas)) AS bindingDelta
WHERE
  timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 180 DAY)
  -- AND log_id = "cloudaudit.googleapis.com/activity"
  AND (
    (resource.type = "service_account"
    AND proto_payload.audit_log.method_name LIKE "google.iam.admin.%.SetIAMPolicy")
    OR
    (resource.type IN ("project", "folder", "organization")
    AND proto_payload.audit_log.method_name = "SetIamPolicy")
  )
  AND JSON_VALUE(bindingDelta.role) IN (
    'roles/iam.serviceAccountTokenCreator',
    'roles/iam.serviceAccountUser'
  )
  AND JSON_VALUE(bindingDelta.action) = 'ADD'
  -- Principal (grantee) exclusions
  AND JSON_VALUE(bindingDelta.member) NOT LIKE "%@example.com"
ORDER BY
  timestamp DESC

2. VPC Service Control configuration modification in last 24 hrs

SELECT
  timestamp,
  proto_payload.authenticationInfo.principalEmail,
  proto_payload.methodName,
  proto_payload.resourceName
FROM
  cloudaudit.googleapis.com/activity
WHERE
  timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY)
AND proto_payload.methodName IN (
  "v1.compute.networks.addPeering",
  "v1.compute.networks.removePeering",
  "v1.compute.networks.updatePeering",
  "v1.compute.networks.addPrivateIp",
  "v1.compute.networks.deletePrivateIp",
  "v1.compute.networks.updatePrivateIp",
  "v1.compute.networks.update",
  "v1.compute.networks.delete"
)

3. Large number of compute instances created in a specified period of time.

SELECT 
  COUNT(*) as num_instances_created,
  resource.labels.project_id as project_id
FROM 
  `PROJECT_ID`
WHERE 
  resource.type="gce_instance" 
  AND jsonPayload.event_subtype="compute.instances.insert" 
  AND jsonPayload.event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 5 MINUTE)
GROUP BY 
  resource.labels.project_id
HAVING 
  COUNT(*) >= 100

4. Access to sensitive data in Big Query.

SELECT
  proto_payload.audit_log.authentication_info.principal_email,
  COUNT(*) AS COUNTER
FROM 
  `[MY_PROJECT_ID].[MY_DATASET_ID]._AllLogs`,
  UNNEST(proto_payload.audit_log.authorization_info) authorization_info
WHERE
  (proto_payload.audit_log.method_name = "google.cloud.bigquery.v2.JobService.InsertJob" OR
   proto_payload.audit_log.method_name = "google.cloud.bigquery.v2.JobService.Query")
  AND authorization_info.permission = "bigquery.tables.getData"
  AND authorization_info.resource = "projects/[PROJECT_ID]/datasets/[DATASET_ID]/tables/accounts"
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY
  1
ORDER BY
  2 desc, 1
LIMIT
  100
Footer

5. Identify network reconnaissance activity such as one host trying to connect to multiple hosts

SELECT 
  jsonPayload.connection.src_ip as src_ip,
  jsonPayload.connection.dest_ip as dest_ip,
  COUNT(DISTINCT jsonPayload.connection.dest_ip) as num_unique_dest_ips
FROM 
  `PROJECT_ID.dataset_name.cloud_vpc_flowlogs_YYYYMMDD` 
WHERE 
  jsonPayload.connection.tcp_flags_reset = true 
  AND jsonPayload.connection.src_ip != jsonPayload.connection.dest_ip 
GROUP BY 
  jsonPayload.connection.src_ip, 
  jsonPayload.connection.dest_ip 
HAVING 
  COUNT(DISTINCT jsonPayload.connection.dest_ip) >= 5

Conclusion

In today’s digital world, cloud security has become a top priority for organizations, and security professionals need the right tools to manage security risks effectively. Log Analytics is a powerful feature that can help security professionals improve their security posture by providing detailed logs of events and activities within their cloud environment.