There is a specific meeting that happens after every security incident. Someone asks a simple question, "who changed that, and when," and the room goes quiet, because the answer was never being recorded. The logs that would have held it were off by default, and nobody turned them on because nothing had gone wrong yet. The trail starts the day you go looking for it, which means it starts one day too late.
The whole point of a day-0 build is to refuse that sequence. The audit trail is not a reaction to an incident. It is a property of the project, present before the first real resource exists, so that if the bad day comes, the record of who-did-what is already three months deep instead of empty.
This is part five of building a GCP environment security-first. The encrypt-and-redact part drew the line on what not to write down. This part is the opposite decision: the small set of things you absolutely must, and the one category of them that Google leaves switched off until you ask.
Two audit trails, and only one is on
On GCP the audit trail is not one thing. It is split into categories, and they do not have the same default.
Admin Activity logs are always on, and you cannot turn them off. Every call that creates, modifies, or deletes a resource, every change to an IAM policy, every org-policy edit, lands here. It is free, it is immutable, and it is retained for 400 days without you lifting a finger. This is the good news, and it is genuinely good: on day 0, before you have configured anything, GCP is already recording the most dangerous class of action, the configuration change.
Data Access logs are the other half, and they are off by default. These are the reads: who called AccessSecretVersion on your one secret, who listed the objects in a bucket, who read a row. Google leaves them off for two honest reasons. They are high volume, because reads vastly outnumber writes, and they are billed, because that volume costs money to ingest and store. So the default is silence.
That default is the exact gap. In my infra the coordinator's one real secret is a Turnstile key, pulled from Secret Manager by a single minimal service account. Admin Activity would tell me if someone granted access to that secret. It would say nothing about someone using that access to read the secret itself. The read is the interesting event. The read is what off-by-default hides.
So the decision on a security-first build is: turn Data Access logging on, deliberately, scoped on purpose, on day 0. Not all of it, not everywhere, but on for the things whose reads actually matter.
What "turn it on" actually means
Data Access logging is controlled by an IAM audit config on the project. In Terraform it is a google_project_iam_audit_config resource, and the shape of the decision lives in which log types you enable per service.
There are three log types. ADMIN_READ records metadata reads, listing resources, reading configuration. DATA_READ records reads of the actual content. DATA_WRITE records writes to content. You enable them per service, so you are not forced into all-or-nothing. That granularity is the whole game, because the naive move, "turn on Data Access for allServices," is how you get a logging bill that dwarfs your compute bill and a trail so noisy nobody reads it.
My approach is to start narrow and name the reason for each inclusion:
- Secret Manager:
DATA_READ. Every read of every secret version is now on the record. This is the single highest-value line, because a secret read is the closest thing to a smoking gun in a credential incident. The volume is low, the value is high, it is an easy yes. - IAM and resource-management:
ADMIN_READ. Who has been looking at the permission structure. Reconnaissance shows up here, someone enumerating roles and policies before they touch anything. - Storage
DATA_READ, weighed, not reflexive. Object reads on a busy bucket are the classic volume trap. For the state bucket, which is small and sensitive and rarely read by humans, logging reads is worth it. For a bucket serving public assets, it is noise you pay for. So this one is scoped to the buckets that hold something worth auditing, not switched on globally.
The thing I do not do is enable DATA_READ for allServices. That is the switch that looks responsible and is actually reckless, because it converts every routine internal read into a billed log line, and the signal you wanted drowns in it.
The exemption trapdoor
There is a second half to the audit config that is easy to miss and important to get right: exempted members.
An audit config lets you enable a log type and then exempt specific principals from it. The legitimate use is your own high-volume service accounts whose reads are both expected and enormous, a data pipeline hammering a bucket a million times an hour, where logging every read tells you nothing and costs a fortune.
The trap is that the exemption list is also a beautiful place to hide. A principal exempted from Data Access logging can read the audited resource without leaving a Data Access record. So on a security-first build the rule is: exemptions are rare, each one is justified in a comment next to it, and the principals you would most want on the record, human operators, break-glass identities, anything that can read a secret, are never exempt. If a service account is noisy enough to need exempting, that itself is worth a second look, because a service that reads a secret ten thousand times an hour probably should be caching it.
The audit config is a plan diff like anything else, which means every change to who-is-exempted shows up in review and, because config changes are Admin Activity, in the immutable trail. The audit config edits itself into the audit trail. That is the property you want.
Why day 0 and not day 30
The cost of turning this on when the project is empty is close to zero. There are no reads yet, so there is no volume, so there is no bill and no noise. You get to make the scoping decision calmly, with the resources in front of you and their sensitivity clear, and write the reasoning down while it is fresh.
The cost of turning it on after an incident is that you have already lost the window you cared about. Audit logs are not retroactive. The read you want to investigate happened last Tuesday, and last Tuesday the logging was off, so the record does not exist and will never exist. You cannot backfill a trail. This is the entire argument for day 0 in one sentence: the trail only holds what it was recording at the time, so the only trail worth having is the one that was already running before you had a reason to look.
There is a smaller, quieter reason too. Turning on Data Access logging forces you to enumerate what is actually worth auditing, which forces you to enumerate what is actually sensitive, which is a security exercise disguised as a logging exercise. You cannot scope the audit config without answering "what here would hurt if it were read," and answering that on day 0, before the answer is urgent, is exactly the kind of unglamorous groundwork the whole series is about.
What this half does not solve
Turning the trail on is necessary and not sufficient. Where these records land is split the same way the trail itself is. Admin Activity flows into the _Required bucket, immutable and kept for 400 days with nothing to configure. The Data Access logs I just enabled flow into the _Default bucket, 30 days by default and then gone. So the reads I most wanted to capture have the shortest life, and even the 400-day half is neither long enough for a multi-year window nor something you can run a real query against. It is all recorded, and it is nearly impossible to ask a real question of. A trail you cannot query is a receipt you will never read.
That is the next problem, and it is the genuinely greenfield one, because in the real infra nothing routes this record anywhere durable yet. The next part builds that: a sink to durable storage and the first real questions answered against it.
Previous: encryption as a default, and what not to log. Next: making the record queryable.
The same decision on the other cloud: the account-wide audit trail on AWS.