GCP ACE: Managing GKE Configuration at Scale — Config Sync, Policy Controller, Config Connector, and Blueprints
With dozens of clusters in a fleet, manually kubectl apply-ing config to each one doesn’t scale, and it drifts the moment someone forgets a cluster. This module’s four tools solve that from different angles: pulling config from Git automatically, enforcing policy regardless of how config got there, managing Google Cloud infrastructure itself as Kubernetes objects, and packaging reusable configuration.
Config Sync — GitOps for your clusters
Config Sync keeps cluster configuration in sync with a Git repository: you push a change to the repo, and every cluster subscribed to it pulls and applies it automatically. No one runs kubectl apply by hand against individual clusters.
The two core objects:
| Object | Scope |
|---|---|
| RootSync | Syncs cluster-wide configuration — can create and manage any namespace or cluster-scoped resource |
| RepoSync | Syncs configuration scoped to one namespace — can’t touch cluster-scoped resources or other namespaces |
Installing Config Sync from the Google Cloud console or the gcloud CLI creates a RootSync named root-sync automatically. You can export it with kubectl get rootsync root-sync -n config-management-system -o yaml and edit the fields the console and gcloud don’t expose:
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
name: root-sync
namespace: config-management-system
spec:
git:
auth: none
branch: main
dir: /quickstart/config-sync
period: 15s
repo: https://github.com/GoogleCloudPlatform/anthos-config-management-samples
revision: HEAD
secretRef: {}
sourceFormat: unstructured
period: 15s is the polling interval — how often Config Sync checks the repo for changes. sourceFormat: unstructured means the repo doesn’t have to follow a specific directory hierarchy; Config Sync applies whatever valid Kubernetes manifests it finds under dir.
Secrets referenced by a sync object live next to it. If your Git server uses a private certificate authority, the Secret holding that CA certificate (referenced via spec.git.caCertSecretRef.name) goes in config-management-system for a RootSync, and in the same namespace as the RepoSync itself for a RepoSync.
The exam-relevant principle: RootSync = cluster-wide authority (platform team territory). RepoSync = namespace-scoped (safe to delegate to an individual application team, since it can’t reach outside its own namespace).
Policy Controller — enforcing rules regardless of how config arrived
Config Sync gets configuration onto clusters consistently. Policy Controller (built on open-source Gatekeeper) enforces rules about what that configuration is allowed to contain — whether it arrived via Config Sync, a CI/CD pipeline, or someone running kubectl apply directly.
Two objects work together:
- ConstraintTemplate — defines the schema and enforcement logic for a type of rule. Google ships a library of these, so you usually don’t write the underlying logic (Rego) yourself.
- Constraint — an instance of a ConstraintTemplate with actual parameters filled in, applied to specific resource types.
Example: requiring every Namespace to carry a geo label, using the library’s K8sRequiredLabels template:
# ns-must-have-geo.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
name: ns-must-have-geo
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Namespace"]
parameters:
labels:
- key: "geo"
kubectl apply -f ns-must-have-geo.yaml
kubectl describe constrainttemplate K8sRequiredLabels # inspect a template's parameters
kubectl get constrainttemplates -l="configmanagement.gke.io/configmanagement=config-management" # list the library
match decides what the constraint applies to; parameters supplies the template’s actual arguments. Once applied, any attempt to create a Namespace without a geo label is rejected at admission time — it never reaches etcd.
A fleet-level toggle worth knowing for exam purposes — enabling referential rules (constraints that compare one object against another, rather than validating an object in isolation) on a specific membership:
gcloud container fleet policycontroller update \
--memberships=MEMBERSHIP_NAME \
--referential-rules
--memberships accepts a comma-separated list, and --no-referential-rules turns it back off. Before enabling it, you create a Config object in the gatekeeper-system namespace that lists which kinds (for example, Namespaces or Ingresses) Policy Controller should sync and watch, since a referential rule can only compare against objects it knows about.
Config Connector — Google Cloud resources as Kubernetes objects
Config Sync and Policy Controller manage what’s inside your clusters. Config Connector flips that around: it lets you manage Google Cloud infrastructure itself — a Cloud Storage bucket, a Pub/Sub topic, an IAM binding — by applying Kubernetes manifests. Under the hood, a controller running in your cluster watches those objects and calls Google Cloud APIs to make reality match them, using an IAM service account bound via Workload Identity.
Installing it as a GKE add-on:
# On a new cluster
gcloud container clusters create CLUSTER_NAME \
--release-channel CHANNEL \
--addons ConfigConnector \
--workload-pool=PROJECT_ID.svc.id.goog \
--logging=SYSTEM \
--monitoring=SYSTEM
# Or on an existing one
gcloud container clusters update CLUSTER_NAME --update-addons ConfigConnector=ENABLED
Binding the controller’s Kubernetes service account to a real IAM service account (the Workload Identity pattern from an earlier article in this series, applied here to the add-on itself):
gcloud iam service-accounts create SERVICE_ACCOUNT_NAME
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/editor" # covers most resources, but not managing projects or org-level resources
gcloud iam service-accounts add-iam-policy-binding \
SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \
--member="serviceAccount:PROJECT_ID.svc.id.goog[cnrm-system/cnrm-controller-manager]" \
--role="roles/iam.workloadIdentityUser"
The one object you apply directly configures the installation itself:
# configconnector.yaml
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnector
metadata:
name: configconnector.core.cnrm.cloud.google.com # this exact name is enforced — only one install per cluster
spec:
mode: cluster
googleServiceAccount: "SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com"
stateIntoSpec: Absent
kubectl apply -f configconnector.yaml
kubectl create namespace NAMESPACE
kubectl annotate namespace NAMESPACE cnrm.cloud.google.com/project-id=PROJECT_ID
kubectl wait -n cnrm-system --for=condition=Ready pod --all
From here on, creating a Google Cloud resource is a kubectl apply — meaning it can go through the exact same Config Sync + Policy Controller pipeline as your application config, giving you GitOps and policy enforcement over your cloud infrastructure, not just your workloads.
Blueprints — packaged, reusable configuration
The last piece ties these together: a blueprint is a deployable, version-controlled package of configuration — Kubernetes manifests, Config Connector resources, Policy Controller constraints, IAM bindings — representing a reusable, opinionated setup (an application platform, a landing zone, a security baseline). Instead of writing a fleet’s worth of configuration from scratch, you take a blueprint, customize the parameters that genuinely differ for your organization, and deploy it through the same Config Sync pipeline as everything else.
Two packaging styles show up in Google’s material:
- KRM blueprints as kpt packages: Kubernetes and Config Connector manifests packaged with kpt, which supports versioning and setting parameters. These fit straight into the Config Sync flow above: update the package centrally, commit it, and the change rolls out fleet-wide.
- Reference-architecture blueprints: for example, Google’s enterprise application blueprint, a complete developer platform on GKE for multiple teams. It’s delivered as a GitHub repository of Terraform configurations and scripts plus a guide, not as a kpt package.
Either way, the point is the same: start from a complete, deployable setup instead of assembling isolated snippets yourself.
Exam Takeaways
- Config Sync: RootSync = cluster-wide, RepoSync = namespace-scoped. Polling interval and source format are both configurable fields on the sync object.
- Policy Controller enforces rules at admission time regardless of how config arrived — ConstraintTemplate defines the schema/logic, Constraint supplies the parameters and scope (
match). gcloud container fleet policycontroller update --referential-rulesis the fleet-level toggle for constraints that compare across objects, not just validate one in isolation.- Config Connector manages Google Cloud resources themselves (buckets, IAM, topics) as Kubernetes objects, authenticating via Workload Identity — install via
--addons ConfigConnectorat cluster creation or--update-addons ConfigConnector=ENABLEDafter. - The
ConfigConnectorobject’smetadata.nameis fixed/enforced — only one installation per cluster. - Blueprints package reusable, parameterized configuration. KRM blueprints are kpt packages that deploy through the same Config Sync pipeline as everything else; larger reference architectures like the enterprise application blueprint ship as Terraform.
- The unifying theme across all four: config lives in Git, clusters pull it (Config Sync), policy validates it regardless of source (Policy Controller), infrastructure itself becomes config (Config Connector), and reusable setups become packages (Blueprints).