Skip to main content

Troubleshooting

1. http: server gave HTTP response to HTTPS client​

CSGHub is installed by default using an insecure registry (i.e., <domain or IPv4>:5000 mentioned above). You need to ensure that Kubernetes can pull images from this registry. Therefore, you need to configure the following on each Kubernetes node:

  1. Before configuration, please confirm whether the configuration file /etc/containerd/config.toml exists. If it does not exist, you can use the following command to create it.
mkdir -p /etc/containerd/ && containerd config default >/etc/containerd/config.toml

  1. Configure config_path

    • Containerd 1.x

      version = 2

      [plugins."io.containerd.grpc.v1.cri".registry]
           config_path = "/etc/containerd/certs.d"

    • Containerd 2.x

       version = 3

       [plugins."io.containerd.cri.v1.images".registry]
            config_path = "/etc/containerd/certs.d"

  2. Configure hosts.toml

    # Create Registry certs.d directory
    mkdir /etc/containerd/certs.d/<domain or IPv4>:5000

    # Defined untrusted docker configuraiton
    cat /etc/containerd/certs.d/<domain or IPv4>:5000/hosts.toml
    server = "http://<domain or IPv4>:5000"

    [host."http://<domain or IPv4>:5000"]
       capabilities = ["pull", "resolve", "push"]
       skip_verify = true
       plain-http = true
    EOF

  3. Restart containerd service

    systemctl restart containerd

2. dial tcp: lookup casdoor.example.com on 10.43.0.10:53: no such host​

This problem occurs because the cluster cannot resolve the domain name. If it is a public domain name, please configure domain name resolution. If it is a custom domain name, please configure CoreDNS and Hosts resolution. CoreDNS resolution configuration is as follows:

# Add custom domain name resolution
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  example.server: |
    example.com {
      hosts {
        172.25.11.131 csghub.example.com csghub
        172.25.11.131 casdoor.example.com casdoor
        172.25.11.131 registry.example.com registry
        172.25.11.131 minio.example.com minio
        172.25.11.131 temporal.example.com temporal
        172.25.11.131 starship.example.com starship
        172.25.11.131 starship-api.example.com starship-api
      }
    }
EOF

# Update coredns pods
$ kubectl -n kube-system rollout restart deploy coredns

3. ssh: connect to host csghub.example.com port 22: Connection refused​

This problem is often caused by the failure of gitlab-shell job execution. If this problem occurs, please follow the following methods to troubleshoot:

  1. View
$ kubectl get cm csghub-ingress-nginx-tcp -n csghub -o yaml
apiVersion: v1
data:
  "22": default/csghub-gitlab-shell:22
......

Confirm whether the service name corresponding to port 22 is correct.

  1. If it is incorrect, modify it manually
$ kubectl -n csghub edit configmap/csghub-ingress-nginx-tcp
apiVersion: v1
data:
  "22": csghub/csghub-gitlab-shell:22

# Update ingress-nginx-controller
$ kubectl rollout restart deploy csghub-ingress-nginx-controller -n csghub

4. How to disable Multi-Sync and cleanup relative data​

CSGHub will start with multi-source synchronization enabled by default, but in some cases you may need to disable multi-source synchronization and only host local models. Therefore, you need to disable multi-source synchronization and clear the multi-source synchronization list data that has been synchronized to the local computer. The specific operation is as follows:

  • Docker

    • Set parametes:

      services:
        csghub:
          image: opencsg-registry.cn-beijing.cr.aliyuncs.com/opencsghq/omnibus-csghub:<version>
          environment:
            CSGHUB_OMNIBUS_CONFIG: |
              csghub:
                external_url: "http://csghub.example.com"
              ......
              server:
                multi_sync:
                  enabled: false   # Disable Multi-Sync
              ......

    • Apply changes

      docker compose up -d

    • Cleanup Multi-Sync data

      docker exec -it omnibus-csghub-1 csghub-psql "SELECT cleanup_multi_sync_data();"

  • Helm Chart

    • After v1.10.0

      helm upgrade --install csghub csghub/csghub \
        ......
        --set csghub.server.multiSync.enabled=false \
        ......

    • v1.10.0 and before

      helm upgrade --install csghub csghub/csghub \
        ......
        --set csghub.server.environments.STARHUB_SERVER_MULTI_SYNC_ENABLED=false \
        ......

    • Cleanup Multi-Sync data

      kubectl exec -it pod/csghub-postgresql-0 -n csghub -- su - postgres -lc "psql -d csghub_server -c 'SELECT cleanup_multi_sync_data();'"