← Back to posts

GCP ACE: Managing GKE Configuration at Scale — Config Sync, Policy Controller, Config Connector, and Blueprints

With dozens of clusters in a fleet, manually kubectl apply-ing config to each one doesn’t scale, and it drifts the moment someone forgets a cluster. This module’s four tools solve that from different angles: pulling config from Git automatically, enforcing policy regardless of how config got there, managing Google Cloud infrastructure itself as Kubernetes objects, and packaging reusable configuration.

Config Sync — GitOps for your clusters

Config Sync keeps cluster configuration in sync with a Git repository: you push a change to the repo, and every cluster subscribed to it pulls and applies it automatically. No one runs kubectl apply by hand against individual clusters.

The two core objects:

ObjectScope
RootSyncSyncs cluster-wide configuration — can create and manage any namespace or cluster-scoped resource
RepoSyncSyncs configuration scoped to one namespace — can’t touch cluster-scoped resources or other namespaces

Installing Config Sync from the Google Cloud console or the gcloud CLI creates a RootSync named root-sync automatically. You can export it with kubectl get rootsync root-sync -n config-management-system -o yaml and edit the fields the console and gcloud don’t expose:

apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
  name: root-sync
  namespace: config-management-system
spec:
  git:
    auth: none
    branch: main
    dir: /quickstart/config-sync
    period: 15s
    repo: https://github.com/GoogleCloudPlatform/anthos-config-management-samples
    revision: HEAD
    secretRef: {}
  sourceFormat: unstructured

period: 15s is the polling interval — how often Config Sync checks the repo for changes. sourceFormat: unstructured means the repo doesn’t have to follow a specific directory hierarchy; Config Sync applies whatever valid Kubernetes manifests it finds under dir.

Secrets referenced by a sync object live next to it. If your Git server uses a private certificate authority, the Secret holding that CA certificate (referenced via spec.git.caCertSecretRef.name) goes in config-management-system for a RootSync, and in the same namespace as the RepoSync itself for a RepoSync.

The exam-relevant principle: RootSync = cluster-wide authority (platform team territory). RepoSync = namespace-scoped (safe to delegate to an individual application team, since it can’t reach outside its own namespace).

Policy Controller — enforcing rules regardless of how config arrived

Config Sync gets configuration onto clusters consistently. Policy Controller (built on open-source Gatekeeper) enforces rules about what that configuration is allowed to contain — whether it arrived via Config Sync, a CI/CD pipeline, or someone running kubectl apply directly.

Two objects work together:

Example: requiring every Namespace to carry a geo label, using the library’s K8sRequiredLabels template:

# ns-must-have-geo.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-geo
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels:
      - key: "geo"
kubectl apply -f ns-must-have-geo.yaml
kubectl describe constrainttemplate K8sRequiredLabels          # inspect a template's parameters
kubectl get constrainttemplates -l="configmanagement.gke.io/configmanagement=config-management"   # list the library

match decides what the constraint applies to; parameters supplies the template’s actual arguments. Once applied, any attempt to create a Namespace without a geo label is rejected at admission time — it never reaches etcd.

A fleet-level toggle worth knowing for exam purposes — enabling referential rules (constraints that compare one object against another, rather than validating an object in isolation) on a specific membership:

gcloud container fleet policycontroller update \
    --memberships=MEMBERSHIP_NAME \
    --referential-rules

--memberships accepts a comma-separated list, and --no-referential-rules turns it back off. Before enabling it, you create a Config object in the gatekeeper-system namespace that lists which kinds (for example, Namespaces or Ingresses) Policy Controller should sync and watch, since a referential rule can only compare against objects it knows about.

Config Connector — Google Cloud resources as Kubernetes objects

Config Sync and Policy Controller manage what’s inside your clusters. Config Connector flips that around: it lets you manage Google Cloud infrastructure itself — a Cloud Storage bucket, a Pub/Sub topic, an IAM binding — by applying Kubernetes manifests. Under the hood, a controller running in your cluster watches those objects and calls Google Cloud APIs to make reality match them, using an IAM service account bound via Workload Identity.

Installing it as a GKE add-on:

# On a new cluster
gcloud container clusters create CLUSTER_NAME \
    --release-channel CHANNEL \
    --addons ConfigConnector \
    --workload-pool=PROJECT_ID.svc.id.goog \
    --logging=SYSTEM \
    --monitoring=SYSTEM

# Or on an existing one
gcloud container clusters update CLUSTER_NAME --update-addons ConfigConnector=ENABLED

Binding the controller’s Kubernetes service account to a real IAM service account (the Workload Identity pattern from an earlier article in this series, applied here to the add-on itself):

gcloud iam service-accounts create SERVICE_ACCOUNT_NAME

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/editor"   # covers most resources, but not managing projects or org-level resources

gcloud iam service-accounts add-iam-policy-binding \
  SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager]" \
  --role="roles/iam.workloadIdentityUser"

The one object you apply directly configures the installation itself:

# configconnector.yaml
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnector
metadata:
  name: configconnector.core.cnrm.cloud.google.com   # this exact name is enforced — only one install per cluster
spec:
  mode: cluster
  googleServiceAccount: "SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com"
  stateIntoSpec: Absent
kubectl apply -f configconnector.yaml
kubectl create namespace NAMESPACE
kubectl annotate namespace NAMESPACE cnrm.cloud.google.com/project-id=PROJECT_ID
kubectl wait -n cnrm-system --for=condition=Ready pod --all

From here on, creating a Google Cloud resource is a kubectl apply — meaning it can go through the exact same Config Sync + Policy Controller pipeline as your application config, giving you GitOps and policy enforcement over your cloud infrastructure, not just your workloads.

Blueprints — packaged, reusable configuration

The last piece ties these together: a blueprint is a deployable, version-controlled package of configuration — Kubernetes manifests, Config Connector resources, Policy Controller constraints, IAM bindings — representing a reusable, opinionated setup (an application platform, a landing zone, a security baseline). Instead of writing a fleet’s worth of configuration from scratch, you take a blueprint, customize the parameters that genuinely differ for your organization, and deploy it through the same Config Sync pipeline as everything else.

Two packaging styles show up in Google’s material:

Either way, the point is the same: start from a complete, deployable setup instead of assembling isolated snippets yourself.

Exam Takeaways

Sources

↗ LinkedIn