Upgrades and Rollback

Pre-Upgrade Checklist

Before upgrading, complete these steps:

1. Review release notes for the target version at docs.watchlight.ai.
2. Back up the database. See Backup Recommendations below.
3. Check current release status:

   helm list -n watchlight
   helm history beacon -n watchlight

4. Diff the changes to preview what will change:

   helm diff upgrade beacon ./deploy/helm/watchlight-beacon \
     --namespace watchlight \
     -f your-values.yaml

   (Requires the helm-diff plugin.)
5. Verify cluster resources are healthy:

   kubectl get pods -n watchlight

Upgrade

Standard Upgrade

helm upgrade beacon ./deploy/helm/watchlight-beacon \
  --namespace watchlight \
  -f your-values.yaml

Upgrade to a New Image Version

helm upgrade beacon ./deploy/helm/watchlight-beacon \
  --namespace watchlight \
  -f your-values.yaml \
  --set global.imageTag="0.4.0-preview"

Enable Additional Tiers

To add Tier 2 (authorization) to an existing Tier 1 deployment:

helm upgrade beacon ./deploy/helm/watchlight-beacon \
  --namespace watchlight \
  -f your-values.yaml \
  --set wl-apdp.enabled=true \
  --set wl-apdp-frontend.enabled=true

Database Migrations

The wl-registry service runs database migrations automatically on startup. No manual migration step is required during upgrades.

Key behaviors:

• Migrations are idempotent and tracked by checksum.
• New migrations ship with each release and apply on pod startup.
• If a migration fails, the pod will crash-loop. Check logs to diagnose.
• Never modify applied migration files — checksums are verified on startup and changing even a comment breaks it.

To verify migration status after an upgrade:

kubectl logs -n watchlight -l app.kubernetes.io/name=wl-registry --tail=50

Look for applied N migrations in the startup output.

Version Compatibility

Chart Version	App Version	Notes
0.3.0	0.3.0-preview	Current release. Tier 1-3 support.

When upgrading across minor versions (for example, 0.3.x to 0.4.x), always review the release notes for breaking changes in values or API schemas.

Rollback

Quick Rollback

Roll back to the previous release:

helm rollback beacon -n watchlight

Rollback to a Specific Revision

List revision history first:

helm history beacon -n watchlight

Then roll back to a specific revision number:

helm rollback beacon 3 -n watchlight

Important Rollback Considerations

• Database migrations are forward-only. Rolling back the Helm chart does not revert database schema changes. Older application versions must be compatible with the current schema. Watchlight maintains backward compatibility within the same minor version.
• PVCs are retained across rollbacks. Data is not lost unless you explicitly delete volumes.
• ConfigMaps and Secrets are versioned by Helm and will revert to the target revision.

Backup Recommendations

Built-in PostgreSQL

If using the chart's built-in PostgreSQL, create a backup before upgrading:

# Create a backup
kubectl exec -n watchlight sts/beacon-postgresql -- \
  pg_dump -U beacon -d beacon > beacon-backup-$(date +%Y%m%d).sql

Managed Database (BYO)

If using RDS, Cloud SQL, or Azure Database for PostgreSQL, use your cloud provider's snapshot or backup features before upgrading.

Full State Export

Export the current Helm release values as a restore reference:

helm get values beacon -n watchlight -o yaml > beacon-values-backup.yaml

Blue-Green Upgrade Strategy

For zero-downtime upgrades in production, use a blue-green approach:

1. Deploy the new version to a separate namespace:

   kubectl create namespace watchlight-green
   
   helm install beacon-green ./deploy/helm/watchlight-beacon \
     --namespace watchlight-green \
     -f your-values.yaml \
     --set global.imageTag="0.4.0-preview" \
     --set postgresql.enabled=false \
     --set global.externalDatabase.url="$DATABASE_URL"

2. Validate the new deployment:

   kubectl get pods -n watchlight-green
   kubectl port-forward -n watchlight-green svc/beacon-green-wl-registry 8081:8080
   curl http://localhost:8081/health

3. Switch traffic by updating your ingress or DNS to point to the new namespace.

4. Tear down the old deployment after confirming the new release is stable:

   helm uninstall beacon --namespace watchlight
   kubectl delete namespace watchlight

Both blue and green instances must point to the same database. Since migrations are forward-only and idempotent, the new version applies any pending migrations on startup while the old version continues to operate with the updated schema.