To get the codes directly https://github.com/tomwynn/gcp-stackdriver-terraform

Problem Statement: How do we ensure that we get all audit logs from all the accounts under all the GCP folders or GCP orgs in an automated fashion?

Let’s first review the GCP resources hierarchy, they are commonly organized as following:

We want to get audit logs for all the Projects to make sure we have the visibility into all of our GCP Projects. But before we dive into dissecting the terraform codes, let’s determine the logs of interest and what GCP audit logs provide:

“Google Cloud services write audit logs that record administrative activities and accesses within your Google Cloud resources. Audit logs help you answer “who did what, where, and when?” within your Google Cloud resources with the same level of transparency as in on-premises environments. Enabling audit logs helps your security, auditing, and compliance entities monitor Google Cloud data and systems for possible vulnerabilities or external data misuse.“

https://cloud.google.com/logging/docs/audit

Types of audit logs that we are going to obtain in this terraform:

Cloud Audit Logs provides the following audit logs for each Cloud project, folder, and organization:

• Admin Activity audit logs
• Data Access audit logs
• System Event audit logs
• Policy Denied audit logs

Our simplified architectural diagram

Now, time to dive into the terraforms:

Terraform structure:

main.tf – main terraform codes are here

backend.tf – where we setup our states, this is important, I will explain why later

outputs.tf – define outputs from main.tf, mostly used to troubleshoot

provider.tf – define our Cloud Service Provider and other related variables here

variables.tf – terraform parameters

backend.tf

This defines where do we want to store our terraform states. If you want to know more about terraform states, take a look here Why should we use a remote state? Remote state allows you to share output values which will allow teams to collaborate on infrastructure resources. Alleviate issues with unknown configurations or lack of documentations, handover, etc.

provider.tf

This simply depicts which provider (platforms), we will have google here. Also defined variables such as project and region.

variables.tf

Here I defined:

• pubsub_prefix: for organizational purposes, I like to append a prefix to my pubsubs with some meaning so I know where they are coming from. In this case, I will append a prefix with a fake Folder Name called “Operational_Security_”
• loggingsink_folder_id: this is the folder that we want to get all the logs of its children from – our bread and butter – you can essentially replace this folder with org id and get all the children on the org including all folders and all projects. But I like to have some levels of granularity. So I will perform this task at the main folder level.
• topic_names: define the names for the topics
• subscription_names: define the names for the subscriptions
• filters: this is an important step to know, filters are ways we build queries to get the events of interest in GCP Log Explorer. More in-depth information here
  • In this code, we will filter (or query) by
    • logName: activity for admin activity
    • logName: system_event for system_ events
    • logName: vpc_flows for vpc flow logs
    • logName: data_access for Data access events
  • We can be creative too with whatever we want to ingest from the logging sink, just have to find the right filter (query)

outputs.tf

Why use outputs.tf? This has many uses, you can manipulate to have the outputs and review what gets spit out as the results of the main.tf. Essentially acting as our logging mechanism. For example: if in main.tf you created the topics, then I would want to know their names or all the designated GCP urls to reuse later.

main.tf

Alright, it’s time for our main.tf, where we provision our infra. The main.tf file is a little long so I will explain it piece by piece

—————

resource “google_pubsub_topic” “topics”: create topics with variables defined earlier with for_each loop

resource “google_pubsub_subscription” “subscriptions”: create subscriptions against the topics created above. With depends_on to make sure that the topics were created. And we also looped through the topics with for each loop in each.value.name

—————

resource “google_logging_folder_sink” “log_sinks“: create the logging sinks and connect to the topics created

• loop through topics with for_each loop
• create logging sink names with replace
• add descriptions
• folder: this is where we specify the folder that we want to create the aggregated logging sink for all of the resources umbrellaed by this folder
• destination: these are the FQDN of the topics (this is there outputs would be handy to understand the values created by the terraform resources)
• include_children: this is important, we need this set to true to be able to obtain audit logs from all the sub resources of the folder

—————

Now we add the role pubsub_publisher to the writer_identity created from the resource “google_logging_folder_sink” “log_sinks” created above so that the log sinks can write to our pub/sub

—————

Extras:

If you don’t have a service account, here are the steps to create the service accounts and store the keys safely in GCP Secret Manager

Create service account:

then store the keys into GCP Secret Manager Safely

Results:

Note that logging sinks at the folder level can only be retrieved by this command

gcloud logging sinks list --folder [folder-id]

What next? Let’s build an Elastic environment to ingest these logs from these PubSubs!