Skip to main content

Uploading Models

To upload models, you will need to create an account at OpenCSG Community. Models are Git-based repositories, which give you versioning, branches, discoverability and sharing features. You can upload anything you want to the model repository, including model files, configuration files, and any other files.

Currently, we support four ways to upload files: via Git, Web side, command line, and SDK. The following details how to use these four methods.

Upload Files to a Repository Using Git​

  1. First clone the repository to your local machine. Copy the file to the corresponding repository.

  1. Assuming that your files are located in the /work/my_model_dir local directory, you can upload the local files to the platform with the following command:

    cd test_model_1
    cp -rf /work/my_model_dir/* .
    git add .
    git commit -m "commit message"
    git push

[Note]​

Files with the following suffixes are automatically uploaded with git-lfs:
.7z,.arrow,.bin,.bz2,.ckpt,.ftz,.gz,.h5,.joblib,.mlmodel,.model,.msgpack,.npy,.npz,.onnx,.ot,.parquet,.pb,.pickle,.pkl,.pt,.pth,.rar,.safetensors,.tar,.tflite,.tgz,.wasm,.xz,.zip,*.zst

If there are other types of large files, run the following command to make them upload as lfs:

git lfs track <your_file_name>

Upload Files to a Repository Using Web Interface​

To add files to your repository with the web interface, start by selecting the Files tab, and then clicking Add file. You will be given the option to create a new file or upload a file.

Add new file

Creating a New File​

Click Create new file, add the contents and click Create File to save your file.

Add new file

Uploading a File​

Click Upload file, you can choose a local file to upload.

Upload new file

Uploading Data via Command Line​

You can conveniently upload data using the command line tool csghub-cli. The installation method is as follows:

pip install csghub-sdk

Here is an example of uploading a local folder to the root path of a repo:

export CSG_TOKEN=your_access_token

# Upload the local folder '/Users/hhwang/temp/jsonl' to the root path of the repo 'wanghh2000/m01' with the default branch
csghub-cli upload wanghh2000/m01 /Users/hhwang/temp/jsonl

Uploading Data Using the SDK​

The CSGHub SDK provides a Python library that allows you to upload data through the SDK in your code.

Here is an example code to upload a repository:

from pycsghub.repository import Repository

token = "your access token"
r = Repository(
    repo_id="wanghh2003/ds15",
    upload_path="/Users/hhwang/temp/bbb/jsonl",
    user_name="wanghh2003",
    token=token,
    repo_type="model",
)
r.upload()

The SDK also supports uploading single or multiple files. For detailed examples, please refer to the SDK Documentation.