Skip to main content

Uploading Models

To upload models, you will need to create an account at CSGHub. Models are Git-based repositories, which give you versioning, branches, discoverability and sharing features. You can upload anything you want to the model repository, including model files, configuration files, and any other files.

Currently, we support four ways to upload files: via Git, Web side, command line, and SDK. The following details how to use these four methods.

Upload Files to a Repository Using Git​

  1. First clone the repository to your local machine. Copy the file to the corresponding repository.

  2. Assuming that your files are located in the /work/my_model_dir local directory, you can upload the local files to the platform with the following command:

    cd model123
    cp -rf /work/my_model_dir/* .
    git add .
    git commit -m "commit message"
    git push

[Note]​

Files with the following suffixes are automatically uploaded with git-lfs:
.7z,.arrow,.bin,.bz2,.ckpt,.ftz,.gz,.h5,.joblib,.mlmodel,.model,.msgpack,.npy,.npz,.onnx,.ot,.parquet,.pb,.pickle,.pkl,.pt,.pth,.rar,.safetensors,.tar,.tflite,.tgz,.wasm,.xz,.zip,*.zst

If there are other types of large files, run the following command to make them upload as lfs:

git lfs track <your_file_name>

Upload Files to a Repository Using Web Interface​

To add files to your repository with the web interface, start by selecting the Files tab, and then clicking Add file. You will be given the option to create a new file or upload a file.

Add new file

Creating a New File​

Click Create new file, add the contents and click Create File to save your file.

Add new file

Uploading a File​

Click Upload file, you can choose a local file to upload.

Upload new file

Uploading Data via Command Line​

You can conveniently upload data using the command line tool csghub-cli. The installation method is as follows:

pip install csghub-sdk

Here is an example of uploading a local folder to the root path of a repo:

export CSG_TOKEN=your_access_token

# Upload the local folder '/Users/hhwang/temp/jsonl' to the root path of the repo 'wanghh2000/m01' with the default branch
csghub-cli upload wanghh2000/m01 /Users/hhwang/temp/jsonl

Uploading Data Using the SDK​

The CSGHub SDK provides a Python library that allows you to upload data through the SDK in your code.

Here is an example code to upload a repository:

from pycsghub.repository import Repository

token = "your access token"
r = Repository(
    repo_id="wanghh2003/ds15",
    upload_path="/Users/hhwang/temp/bbb/jsonl",
    user_name="wanghh2003",
    token=token,
    repo_type="model",
)
r.upload()

The SDK also supports uploading single or multiple files. For detailed examples, please refer to the SDK Documentation.

Viewing the Model Repository History​

Each time you perform the add-commit-push, the model repository tracks every change you make to the files. You can browse the model files and commits, and view the differences (also known as diff) introduced by each commit. To view the history, click on commit history. Commit History

You can also click on an individual commit to see what changes were introduced in that specific commit: Introduced Changes