Skip to main content

Make Your First Model Request

This guide uses an enabled text model to demonstrate non-streaming and streaming requests through the OpenAI-compatible interface. To deploy or connect a model, see Connect and Call Models.

Prerequisites​

InformationWhere to get it
API base addressThe model service or endpoint's API instructions; this example uses an address ending in /v1
Model nameObtain an identifier from the available model list, or use the identifier in the service's example
API keyCreate a valid key in personal or organization API Keys settings, with access to the service

For paid services, ensure that the caller has available spending allowance and the required account balance. See Identity and Access.

1. Configure the request​

The Python examples use the following environment variables. Replace the address and model name. Set CSGHUB_API_KEY in the application's runtime environment or secret manager; do not commit real credentials to source control.

export CSGHUB_API_BASE="https://your-gateway-host/v1"
export CSGHUB_MODEL="your-model-name"

2. Make a request​

Run this in an application environment with the OpenAI Python SDK installed:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["CSGHUB_API_KEY"],
    base_url=os.environ["CSGHUB_API_BASE"],
)

response = client.chat.completions.create(
    model=os.environ["CSGHUB_MODEL"],
    messages=[{"role": "user", "content": "Describe your capabilities in one sentence."}],
)
print(response.choices[0].message.content)
print(response.usage)

On success, inspect the reply and any usage information returned by the service. Usage fields depend on the model and upstream. See Metering and Costs to interpret charges.

3. Stream the response​

Reuse client from the previous example and set stream=True:

stream = client.chat.completions.create(
    model=os.environ["CSGHUB_MODEL"],
    messages=[{"role": "user", "content": "Briefly introduce model services."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="", flush=True)
print()

This example reads text fields and skips chunks without text. Handle tool calls and other events according to the interface’s documentation.

If the request fails​

Check the API address, model name, and credential type first. Then check whether the key has expired, reached its allowance, or been revoked. Do not substitute a Git Access Token for a gateway API key. See Troubleshooting.

Next steps​