Standard Upgrade Guide
📘 Overview
This guide explains how to safely upgrade CSGHUB in a production environment. The upgrade process ensures data security, service continuity, and minimal downtime.
💡 Scope: Applicable for CSGHUB clusters deployed via Helm Chart.
⚙️ Prerequisites
✅ Environment Requirements
| Item | Requirement |
|---|---|
| Kubernetes Cluster | v1.28+, cluster in healthy state (kubectl get nodes all Ready) |
| Helm | v3.12.0+, with administrator privileges |
| CSGHUB Deployment | Running instance successfully deployed via Helm |
⚠️ Note: Always backup data and save configuration before upgrading to prevent data loss.
💾 Pre-Upgrade Preparation
1️⃣ Backup Current Configuration
Export current Helm values:
helm get values csghub -n csghub -o yaml > csghub-values-backup.yaml
2️⃣ Backup Critical Data
At minimum, backup the following:
| Component | Backup Method |
|---|---|
| Database (PostgreSQL) | Follow database backup and restore guide |
| Helm Values | Already exported as csghub-values-backup.yaml |
💡 Recommended: Use snapshot-based backups (e.g., Ceph Snapshot / RDS Snapshot) for consistency.
🧩 Upgrade Steps
1️⃣ Update Helm Repository
helm repo update
Check available versions:
helm search repo csghub/csghub --versions
Example output:
NAME CHART VERSION APP VERSION DESCRIPTION
csghub/csghub 1.15.0 v1.15.0 CSGHub charts with CE and EE.
......
2️⃣ Execute Upgrade Command
helm upgrade --install csghub csghub/csghub \
--namespace csghub \
-f csghub-values-backup.yaml
To upgrade to a specific version:
helm upgrade csghub csghub/csghub \
--namespace csghub \
-f csghub-values-backup.yaml \
--version 1.15.0
⚠️ If you modified the domain or enabled HTTPS, ensure the following parameters remain consistent:
--set global.ingress.domain="<your-domain>" \
--set global.ingress.tls.enabled=true \
--set global.ingress.tls.secretName="csghub-tls-certs"
3️⃣ Verify Upgrade Status
helm status csghub -n csghub
Check if all Pods are running:
kubectl get pods -n csghub
Normally, all components should be in Running or Completed state.
🔄 Rollback (If Needed)
To roll back if issues occur:
Check revision history:
helm history csghub -n csghub
Rollback to a previous revision:
helm rollback csghub <revision> -n csghub
⚠️ Rollback does not automatically restore external database or storage data. Use backups if needed.
🔍 Post-Upgrade Validation
Verify services:
| Service | Validation Command / Action |
|---|---|
| CSGHUB Portal | Access http://csghub.example.com |
| Casdoor Console | Access http://casdoor.example.com |
| Temporal Console | Access http://csghub.example.com/-/temporal |
| MinIO Console | Access http://minio.example.com:30080/console |
| Registry | Execute docker login csghub.example.com to verify availability |
To retrieve credentials:
kubectl get secret -n csghub csghub-casdoor-init \
-o jsonpath='{.data.INIT_ADMIN_USER}' | base64 -d && echo -n " / " && \
kubectl get secret -n csghub csghub-casdoor-init \
-o jsonpath='{.data.INIT_ADMIN_PASSWORD}' | base64 -d
⚡ Common Issues
-
Pod stuck in CrashLoopBackOff
Troubleshoot steps:
- Run kubectl describe pod
-n csghub or kubectl logs -f pod -n csghub to check the error. - Image pull failures due to network issues.
- Dependencies not started—check dependent resources.
- Database connection issues.
- Run kubectl describe pod
-
Database not found errors
Occurs if a feature was disabled during initial deployment but enabled during upgrade. Create missing database manually:
kubectl exec -it csghub-postgresql-0 -n csghub -- su - postgres -lc 'psql -U csghub'
> CREATE DATABASE <DB_NAME> ENCODING UTF8 OWNER csghub;
kubectl rollout restart deploy/sts <resource> -n csghub
-
Access returns 404 after upgrade
Ensure all resources are ready, especially csghub-server / csghub-portal.
-
Page fails to fetch tags
Log out and log back in to CSGHub.
-
StatefulSet patch errors during upgrade
Error message:
Error: UPGRADE FAILED: cannot patch "my-app" with kind StatefulSet: StatefulSet.apps "my-app" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenThis issue is usually caused by adjustments to resource files after version optimization and refactoring, which prevents StatefulSets from updating correctly. Follow these steps to fix it:
-
Delete all StatefulSets resources
kubectl delete sts --all -n csghub -
Retrying upgrade:
helm upgrade --install csghub csghub/csghub \
--namespace csghub \
-f csghub-values-backup.yaml
-
🔒 Safe Upgrade Recommendations
- ✅ Test version compatibility in a staging environment before production.
- ✅ Retain at least 2 historical versions for quick rollback.
📚 References
- Official Documentation: https://github.com/OpenCSGs/csghub-charts
- Helm Documentation: https://helm.sh/docs/
- Kubernetes Admin Guide: https://kubernetes.io/docs/tasks/administer-cluster/