← Back to posts

GCP ACE: CI/CD at Scale in GKE — Cloud Build, Cloud Deploy, Cloud Run, and Supply Chain Security

The containers article earlier in this GKE module covered building a single image with Cloud Build. At fleet scale, the pipeline gets more stages: progressive delivery across targets, reaching clusters with no public endpoint, and proving an image hasn’t been tampered with before it’s allowed anywhere near production.

Cloud Build and GKE — the recap, extended

The build step itself (gcloud builds submit, cloudbuild.yaml, pushing to Artifact Registry) was covered in depth in the containers article. What’s new at scale is where those builds run and what they can reach — covered below under private networking.

Cloud Deploy — progressive delivery to GKE

Cloud Deploy manages the rollout of an already-built image across one or more targets (a target is usually a specific GKE cluster), using two core objects:

apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: prod
description: prod GKE cluster
gke:
  cluster: projects/PROJECT_ID/locations/us-central1/clusters/prod-cluster
apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
  name: my-app-pipeline
description: main application pipeline
serialPipeline:
  stages:
  - targetId: prod
    strategy:
      canary:
        runtimeConfig:
          kubernetes:
            serviceNetworking:
              service: "my-service"
              deployment: "my-deployment"
        canaryDeployment:
          percentages: [50]
          verify: false

canaryDeployment.percentages: [50] means the canary phase sends the new version to 50% of traffic (via service-based networking, that’s 50% of the Pods) before you advance further.

How a rollout actually proceeds, phase by phase:

  1. First-ever release: there’s nothing running yet to split traffic against, so Cloud Deploy skips the canary phase entirely and goes straight to “Pending advance” at the stable phase. You explicitly click Advance to stable to bring it to 100%.
  2. A later release: now there is an existing version running, so the canary phase actually executes — the new version goes to 50%, and the rollout pauses there.
  3. Final advance: you (or an automated gate) trigger Advance Rollout, which brings the new version to 100% and completes it.

The exam-relevant point: a canary percentage only means something once there’s an existing stable version to split traffic against — the very first deployment of a pipeline is never actually a canary, regardless of what the pipeline config says.

Cloud Run and Knative Serving

Cloud Run is Google’s fully managed serverless container platform. It implements the API of Knative Serving, an open-source project — and Knative serving on GKE (formerly Cloud Run for Anthos) is what you get when you want Cloud Run-style deployment semantics (automatic revisions, traffic splitting, scale-to-zero) running on your own GKE cluster instead of on Cloud Run’s fully managed infrastructure.

The distinction that matters for the exam: Cloud Run (fully managed) and “Cloud Run for Anthos” / Knative Serving on GKE are the same programming model and the same serving.knative.dev API objects, but one runs on infrastructure Google manages entirely, and the other runs on your own cluster’s nodes, where you manage capacity, networking, and the surrounding cluster yourself. If a workload needs the simplicity of Cloud Run’s deployment model but also needs to live alongside other GKE workloads, share a cluster’s networking or GPUs, or meet a requirement that rules out fully managed Cloud Run, Knative Serving on GKE is the answer.

Cloud Deploy and these runtimes: Cloud Deploy has first-class target types for GKE and Cloud Run, and its built-in canary strategy works on both: on GKE through service-based networking or the Gateway API, on Cloud Run through its native revision traffic splitting.

CI/CD in a private network

A private GKE cluster (no public control-plane endpoint, from the security-conscious configurations referenced earlier in this series) isn’t reachable from Cloud Build’s standard, Google-managed workers — there’s no network path.

Cloud Build private pools solve this: dedicated build workers that live in a Google-owned project peered into your VPC via VPC Network Peering, so build steps execute with network access to your private resources — including, with enough networking set up, a private GKE cluster.

gcloud builds worker-pools create PRIVATE_POOL_NAME \
    --region=REGION \
    --peered-network=projects/PROJECT_ID/global/networks/PEERING_VPC_NAME

The one wrinkle worth knowing for the exam, even without memorizing the full setup: a private GKE cluster’s control plane itself lives in a separate, Google-managed network connected to your VPC via Private Service Connect — and VPC Network Peering doesn’t support transitive peering. So a private pool peered to your VPC still can’t automatically reach the control plane through a second hop. The documented fix is to connect the private pool’s network and the cluster’s network with your own VPN (Cloud VPN with BGP), so each side explicitly advertises routes to the other — closing the gap that peering alone can’t bridge. You don’t need the exact commands memorized; you need to recognize why a straightforward “just peer it” approach doesn’t fully work here. Google’s same guide also names the simpler alternative: give the cluster a DNS-based control plane endpoint, which a private pool can reach without any VPN.

Once connectivity exists, authorize the pool’s IP range on the cluster and grant the build service account cluster access:

gcloud container clusters update PRIVATE_CLUSTER_NAME \
    --enable-master-authorized-networks \
    --master-authorized-networks=PRIVATE_POOL_RANGE

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:BUILD_SERVICE_ACCOUNT \
    --role=roles/container.developer

Securing the software supply chain: Binary Authorization

Building and deploying faster is only safe if you can also guarantee what gets deployed is trustworthy. Binary Authorization enforces that an image can only be deployed to GKE if it carries a valid attestation — cryptographic proof that it passed a specific check (built by a trusted CI pipeline, passed a vulnerability scan, signed off by a security team) — from a trusted attestor.

gcloud container binauthz policy export > /tmp/policy.yaml
# /tmp/policy.yaml — edited
defaultAdmissionRule:
  evaluationMode: REQUIRE_ATTESTATION
  enforcementMode: ENFORCED_BLOCK_AND_AUDIT_LOG
  requireAttestationsBy:
  - projects/PROJECT_ID/attestors/ATTESTOR_NAME
globalPolicyEvaluationMode: ENABLE
name: projects/PROJECT_ID/policy
gcloud container binauthz policy import /tmp/policy.yaml

Tying this back to earlier articles: both Binary Authorization and Policy Controller act when a workload is admitted to the cluster, but they check different things. Binary Authorization checks the image’s provenance (does it carry the required attestations?); Policy Controller checks the manifest’s configuration (does it follow your rules?). Vulnerability scanning is a third, separate concern, handled by Artifact Analysis on images in Artifact Registry (covered in the security posture article that follows). All three can be true at once in a mature pipeline — Policy Controller blocking a misconfigured manifest, Binary Authorization blocking an unattested image, and Artifact Analysis reporting a CVE in an image that already made it through both.

Exam Takeaways

Sources

↗ LinkedIn