Skip to main content

GCP GKE

Deploy the Watchlight AI Agent Runtime Governance control plane (Beacon) on Google Kubernetes Engine.

Prerequisites

RequirementDetails
GKE clusterStandard or Autopilot, Kubernetes 1.27+
gcloud CLIAuthenticated with appropriate project permissions
kubectlConfigured for your GKE cluster
Helmv3.12+
GCE Ingress ControllerEnabled by default on GKE clusters
GHCR accessImage pull secret for ghcr.io/watchlight-ai-beacon

Required IAM roles

The deploying principal needs:

  • roles/container.admin (GKE cluster management)
  • roles/cloudsql.admin (if provisioning Cloud SQL)
  • roles/iam.workloadIdentityUser (for Workload Identity binding)

Infrastructure Setup

1. Create the GKE cluster

Standard cluster:

gcloud container clusters create watchlight-beacon \
--region us-central1 \
--machine-type e2-standard-4 \
--num-nodes 2 \
--enable-autoscaling --min-nodes 2 --max-nodes 6 \
--workload-pool=PROJECT_ID.svc.id.goog \
--enable-ip-alias

Autopilot cluster (recommended for production -- GKE manages node scaling):

gcloud container clusters create-auto watchlight-beacon \
--region us-central1 \
--workload-policies=allow-net-admin

2. Provision Cloud SQL PostgreSQL

gcloud sql instances create beacon-db \
--database-version=POSTGRES_16 \
--tier=db-custom-2-7680 \
--region=us-central1 \
--availability-type=REGIONAL \
--storage-type=SSD \
--storage-size=50GB \
--storage-auto-increase \
--backup-start-time=03:00 \
--enable-point-in-time-recovery

gcloud sql databases create beacon --instance=beacon-db

gcloud sql users create beacon \
--instance=beacon-db \
--password="$(openssl rand -base64 24)"

3. Set up the Cloud SQL Auth Proxy

The Cloud SQL Auth Proxy provides secure, IAM-authenticated connections from GKE to Cloud SQL without exposing the database to the network.

# Get the connection name
gcloud sql instances describe beacon-db --format='value(connectionName)'
# Output: PROJECT_ID:us-central1:beacon-db

The proxy runs as a sidecar. The connection string uses a Unix socket:

postgres://beacon:PASSWORD@/beacon?host=/cloudsql/PROJECT_ID:us-central1:beacon-db

4. Configure Workload Identity

Workload Identity lets Kubernetes service accounts act as Google IAM service accounts, eliminating the need for exported keys.

# Create a Google service account
gcloud iam service-accounts create beacon-sa \
--display-name="Watchlight Beacon"

# Grant Cloud SQL client access
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:beacon-sa@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/cloudsql.client"

# Bind the Kubernetes service account to the Google service account
gcloud iam service-accounts add-iam-policy-binding \
beacon-sa@PROJECT_ID.iam.gserviceaccount.com \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:PROJECT_ID.svc.id.goog[watchlight/beacon-registry]"

5. Create the GHCR image pull secret

kubectl create namespace watchlight

kubectl create secret docker-registry ghcr-credentials \
-n watchlight \
--docker-server=ghcr.io \
--docker-username=YOUR_GITHUB_USERNAME \
--docker-password=YOUR_GITHUB_PAT

6. Create the database connection secret

kubectl create secret generic beacon-db-credentials \
-n watchlight \
--from-literal=DATABASE_URL="postgres://beacon:PASSWORD@/beacon?host=/cloudsql/PROJECT_ID:us-central1:beacon-db"

7. Create a Google-managed certificate

# Create a ManagedCertificate resource
cat <<EOF | kubectl apply -n watchlight -f -
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: beacon-cert
spec:
domains:
- registry.example.com
EOF

Ensure your domain's DNS A record points to the reserved static IP (see below).

gcloud compute addresses create beacon-ip --global

gcloud compute addresses describe beacon-ip --global \
--format='value(address)'

Helm Installation

Review the values file

The GCP-specific values file is at deploy/helm/watchlight-beacon/values-gcp.yaml. Key settings:

global:
imageRegistry: ghcr.io/watchlight-ai-beacon
imagePullSecrets:
- name: ghcr-credentials
storageClass: standard-rwo

ingress:
className: gce
annotations:
kubernetes.io/ingress.class: gce
# For Google-managed certificates:
# networking.gke.io/managed-certificates: beacon-cert

Before installing, update the values file with:

  1. Uncomment the networking.gke.io/managed-certificates annotation and set it to beacon-cert
  2. Your domain name under wl-registry.ingress.hosts[].host and wl-registry-frontend.ingress.hosts[].host
  3. Your Cloud SQL connection string under global.externalDatabase.url (and set postgresql.enabled: false)

Note that GCE Ingress uses ImplementationSpecific path type with glob-style paths (/api/*, /*).

Install

helm install beacon ./deploy/helm/watchlight-beacon \
-n watchlight \
-f ./deploy/helm/watchlight-beacon/values-gcp.yaml

To use Cloud SQL instead of the built-in PostgreSQL:

helm install beacon ./deploy/helm/watchlight-beacon \
-n watchlight \
-f ./deploy/helm/watchlight-beacon/values-gcp.yaml \
--set postgresql.enabled=false \
--set global.externalDatabase.url="postgres://beacon:PASSWORD@/beacon?host=/cloudsql/PROJECT_ID:us-central1:beacon-db"

Validation

Check pod status

kubectl get pods -n watchlight

All pods should reach Running status within a few minutes.

Verify the ingress

kubectl get ingress -n watchlight

GCE ingress provisioning can take 5-10 minutes. Wait until the ADDRESS column shows an IP.

Check the managed certificate status:

kubectl describe managedcertificate beacon-cert -n watchlight

The status should transition to Active once DNS propagation completes.

Health checks

# Registry API
curl -s https://registry.example.com/api/v1/health | jq .

# Registry frontend
curl -s -o /dev/null -w "%{http_code}" https://registry.example.com/

Smoke test

# List registered agents
curl -s https://registry.example.com/api/v1/agents | jq .

# Check wl-discover logs
kubectl logs -n watchlight -l app.kubernetes.io/name=wl-discover --tail=20

Production Hardening

High availability

The GCP values file enables HPA and PodDisruptionBudgets:

  • wl-registry: 2-10 replicas, minimum 1 available during disruptions
  • wl-registry-frontend: 2 replicas

For Standard clusters, ensure your node pool spans at least 2 zones. Autopilot clusters handle this automatically.

Monitoring

GKE integrates with Google Cloud Monitoring. Enable GKE metrics collection:

gcloud container clusters update watchlight-beacon \
--region us-central1 \
--monitoring=SYSTEM,WORKLOAD

For Prometheus-based monitoring, install the Google-managed Prometheus collector:

gcloud container clusters update watchlight-beacon \
--region us-central1 \
--enable-managed-prometheus

Beacon services expose /metrics endpoints that the managed collector scrapes automatically via the ServiceMonitor resources in the Helm chart.

Backup

  • Cloud SQL: Automated backups are enabled by default with the configuration above. Enable point-in-time recovery for sub-day RPO.
  • Configuration: Store your values file in version control.

Scaling considerations

ComponentScaling strategy
wl-registryHPA on CPU (included in values). Autopilot auto-provisions resources.
wl-registry-frontendHPA or fixed 2+ replicas.
wl-discoverDaemonSet -- scales automatically with cluster size.
Cloud SQLVertical scaling via tier change. Read replicas for read-heavy workloads.

Network security

  • Use VPC-native (alias IP) clusters for pod-level network policies.
  • Cloud SQL is not exposed publicly by default when using the Auth Proxy.
  • Apply NetworkPolicy resources to restrict inter-pod traffic to only the required paths (e.g., wl-discover to wl-registry, wl-registry to PostgreSQL).
  • Consider using GKE private clusters if Beacon should not have public node IPs.

Workload Identity best practices

  • Never export service account keys. Use Workload Identity exclusively.
  • Grant the minimum required IAM roles per service account.
  • Annotate each Kubernetes service account with its bound Google service account:
apiVersion: v1
kind: ServiceAccount
metadata:
name: beacon-registry
namespace: watchlight
annotations:
iam.gke.io/gcp-service-account: beacon-sa@PROJECT_ID.iam.gserviceaccount.com