We extended inDrive’s Terraform CI/CD from AWS to Google Cloud without creating a single service-account key, using Workload Identity Federation from runners that never left AWS. Here is what the documentation does not tell you — including why authentication was not the hardest part.

The confession: Google Cloud was outside our Terraform CI/CD

For years, our infrastructure CI was single-cloud — not by decision, but by construction.

The pipeline that plans and applies every Terraform change in our monorepo filtered changed files through an allowlist of directory prefixes. That allowlist simply never learned that gcp/ existed. The apply-on-merge workflow went further and explicitly excluded Google Cloud paths. There was no Google Cloud authentication in CI either: the generated provider block was bare, with no project and no credentials.

The result was predictable but easy to overlook: around 97 Terragrunt units across approximately 25 Google Cloud projects were being applied from engineers’ laptops, using whatever credentials happened to be available locally. There was no plan gate, no policy check, and no central audit trail.

Nobody chose this operating model. It accreted while most of our attention went to the much larger AWS estate.

Fixing it sounded simple: add Google Cloud authentication, then flip the path filter. Authentication was not what got us.

Three constraints for a keyless AWS-to-GCP pipeline

Before touching the pipeline, we deliberately constrained the solution:

1. Terraform state stays in Amazon S3. All our Google Cloud state already lived there. Moving it to Google Cloud Storage during the same project would have doubled the migration risk without creating user-visible value.

2. CI runners stay in AWS. We operate an autoscaled GitHub Actions runner fleet on Kubernetes in AWS. A second runner fleet in Google Cloud would become another platform to patch, scale, monitor, and secure indefinitely.

3. No service-account keys. Exported JSON keys are the default answer to “CI needs Google Cloud access,” but they introduce long-lived credentials that must be stored, rotated, and protected. Our organization policy already blocked key creation, and we were not going to request an exception.

Together, these constraints meant that an AWS-hosted runner had to call Google Cloud APIs without stored keys while continuing to read and write Terraform state in S3. The control plane crossed clouds in one direction; the state backend stayed in the other.

Choosing Google Cloud Workload Identity Federation

Google Cloud’s answer to keyless cross-cloud authentication is Workload Identity Federation (WIF). Google Cloud Security Token Service trusts an external AWS role identity and exchanges it for short-lived Google Cloud credentials. No persistent secret is stored anywhere in the chain.

WIF still leaves an important architecture choice:

• Option A — direct resource access. Grant Google Cloud IAM roles directly to the federated AWS principal. No service account is involved.
• Option B — WIF with service account impersonation. Allow the federated principal to do one thing: impersonate a purpose-scoped service account. The service account holds the actual roles on target projects.

Instead of debating the options in a design document, we built both end to end against real projects with production-shaped trust conditions.

Then we attacked our own configuration. An AWS role from the wrong account was rejected during token exchange. A role from the correct account but with the wrong role name was rejected by the trust condition. Each identity worked only through its intended path. Nothing failed open.

Why service account impersonation won

Option B won clearly for four reasons:

• Lower identity-coupling cost. Our AWS role ARN appears in two places instead of being woven through IAM policies in approximately 25 Google Cloud projects. When AWS-side identities change, the blast radius is two configuration lines.
• Broader API compatibility. A service account is an ordinary Google Cloud IAM member accepted by every API. Federated principals are not universally equivalent, and our environment is BigQuery-heavy, where direct-principal support was not yet proven for our use cases.
• Mature observability and operations. Service accounts come with familiar usage metrics, last-authenticated data, impersonation alerts, and a kill switch that every on-call engineer already understands.
• A purpose-scoped authorization boundary. The federated AWS principal receives only the ability to impersonate the CI service account; project permissions remain attached to a Google Cloud-native identity.

Measure revocation before production needs it

We did not assume the kill switch worked as expected. We measured it.

• After disabling the service account, requests for new tokens failed within approximately one second.
• Already-issued tokens were rejected within one to two minutes.
• Re-enabling the service account restored the flow within seconds.

The test also exposed a constraint that now shapes daily operations: our organization policy caps impersonated-token lifetime at one hour. Any single Terraform apply must fit inside that window. That is the kind of limitation you want to discover in a test harness, not when a production apply dies at minute 61.

How the keyless AWS-to-GCP authentication flow works

At runtime, the AWS-hosted runner uses its AWS identity to obtain short-lived Google Cloud credentials:

• The runner obtains temporary AWS credentials for the approved IAM role.
• The Google authentication library creates a signed AWS request that represents that role identity.
• Google Cloud Security Token Service validates the configured account, role, audience, and attribute conditions.
• The federated principal receives permission to impersonate the purpose-scoped Google Cloud service account.
• The Terraform Google provider uses the resulting short-lived access token to call Google Cloud APIs.
• Terraform continues to use Amazon S3 as its remote state backend through the runner’s AWS permissions.

Two implementation details the documentation undersells

First, the Google authentication library’s AWS flow does not understand Kubernetes-style web-identity credentials directly in our runner setup. We had to materialize standard environment credentials before starting Terraform. Without that step, the library silently spent several minutes probing a metadata endpoint that did not exist.

Second, gcloud is not required on the runners. The Terraform Google provider can perform the Workload Identity Federation exchange natively from a small external-account JSON configuration file. The file describes the federation flow but contains no private key and no reusable secret.

What the security review added

We took the service-account-impersonation prototype to our security team with working evidence rather than slides:

• an audit-log event showing the raw AWS identity and the CI session name;
• an impersonation event naming the Google Cloud service account;
• negative tests for the wrong AWS account and wrong IAM role;
• measured revocation and recovery timings.

That made the decision meeting short.

Security had one hard requirement that we had not planned: allowlist the egress IP addresses of the runner fleet’s AWS NAT gateways on the Google Cloud side. Federation already restricts who can authenticate; the IP allowlist also restricts where authentication can originate.

The control is inexpensive and provides useful defense in depth. It also makes the NAT gateway IP addresses part of the trust surface. Those addresses now have to be documented, monitored, and synchronized whenever the network changes. For us, that was a fair trade.

The hardest PoC debugging happened in AWS, not Google Cloud

The most difficult proof-of-concept problem was not on the Google Cloud side at all.

Our PoC role lived in a different AWS account from the S3 state bucket. Cross-account S3 access requires three permission layers to agree: the role’s identity policy, the bucket policy, and an exemption in a deny-list. Meanwhile, S3 can return 403 in situations where the underlying problem behaves like a missing object.

Production runners use the same AWS account as the state bucket, so this problem was limited to the PoC. It still consumed most of the debugging budget.

The hidden 80%: migrating 131 Terraform states

Our original estimate was 10–15 engineer-days for authentication, path filtering, and Google Cloud policy weights. Halfway through the project, we opened the S3 state bucket to generate the backend configuration and found the real scope.

Years of laptop-driven applies had scattered state objects across inconsistent legacy prefixes. Some keys did not match the current repository paths. Some were orphans from deleted experiments. Others were near-duplicates. The CI pipeline could not safely go live against that layout.

The middle of the project therefore became a full Terraform state migration:

• 131 state objects re-keyed into a clean and predictable structure;
• every Terragrunt unit re-pointed to its new backend key;
• the repository and state bucket verified unit by unit;
• zero Terraform state loss.

The pleasant counter-surprise was the dependency-ordering engine. Everyone assumed that the logic deciding which units applied before others was AWS-specific. It turned out to be cloud-agnostic and needed validation, not the rewrite we had budgeted.

Rolling out Google Cloud without breaking the AWS pipeline

The final risk was not Google Cloud. It was regressing the AWS Terraform pipeline that dozens of engineers rely on every day.

Three rollout habits kept the change safe:

1. Canary the policy model. Our policy gate scores each Terraform plan by risk. We added weights for 45 Google Cloud resource types, but first ran them against synthetic Terragrunt units so the first real Google Cloud pull requests were not blocked by poorly tuned thresholds.

2. Test every workflow leg. Plan, apply, and destroy flows — both comment-driven and merge-driven — were proven in a disposable harness project before we enabled the Google Cloud path filter.

3. Run an explicit AWS no-regression check. We compared the existing AWS behavior before and after the filter change. The result was zero regressions.

The unglamorous ending

Two practices attracted little attention but are the first things we would repeat.

First, we wrote the revocation runbook before go-live: exact commands, measured timings, and named owners. Access that cannot be revoked under pressure is not access; it is liability.

Second, the PoC received its own teardown ticket. We removed harness projects, trust configuration, temporary IAM roles, and bucket-policy entries, while preserving the findings in an architecture decision record.

One last gotcha: deleted Workload Identity Federation pools remain soft-deleted for approximately 30 days. Even teardown has a tail that must be tracked to completion.

Key takeaways for multi-cloud Terraform CI/CD

• Hidden scope lives in the boring parts. Authentication was the interesting problem; the Terraform state bucket was the actual project. Audit the current state before estimating.
• Bring security a working prototype, not a slide deck. Audit evidence and negative tests shorten the decision cycle — and security may still add one requirement you did not anticipate.
• Prefer short-lived federated credentials over service-account keys. Workload Identity Federation reduces secret-management risk and creates clearer audit trails.
• Test revocation, token lifetime, and recovery as product behavior. Do not leave them as assumptions.
• A PoC is not complete until it is deleted. Prototypes with real cloud access cannot retire in place.