Skip to main content

OpenCSG AI Gateway: auto Model Guide

💡 Note: In the following examples, $AIGW represents the entry point URL of your AI Gateway instance. Please replace it with your actual gateway domain or endpoint (e.g., the SaaS public URL or self-hosted service endpoint).

1. Role of Request Headers​

HeaderDescriptionFormat Requirement
CSG-Router-Session-IdRouter Session ID, used to maintain session context consistency for model rankingMust be ses_<32-hex-characters>
CSG-Router-Turn-IdTurn ID within the same conversation turn, reusing saved ranking results to keep the model fixedMust be turn_<32-hex-characters>, and must be accompanied by Session ID
X-Session-ID① Upstream affinity: pins requests to the same upstream via consistent hashing to reuse KV/prefix cache; ② If the value matches the ses_ format, also acts as an alias for Router SessionAny string for ①; only ses_ format takes effect for ②

Key Rules:

  • These IDs are generated by the Router; clients must not fabricate them. The gateway returns CSG-Router-Session-Id and CSG-Router-Turn-Id in the response headers. The client should save them and send them back in subsequent requests.
  • Invalid formats are silently dropped without raising errors; Router will treat the request as a new session and generate fresh IDs. If only Turn-Id is provided without Session-Id, Turn-Id is also discarded.
  • The gateway strips CSG-Router-* headers before forwarding requests to the upstream model; X-Session-ID is preserved because upstream affinity requires it.

Turn 1, User Prompt: Do not include any Router headers.

curl -i $AIGW/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Help me write quicksort"}],"tools":[...]}'
# Response headers:
# CSG-Router-Session-Id: ses_8f3a... (32-hex)
# CSG-Router-Turn-Id:    turn_1c9e... (32-hex)

Tool call loop within the same turn (tool_call → tool result → next request): Include both Session and Turn.

curl $AIGW/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -H "CSG-Router-Session-Id: ses_8f3a..." \
  -H "CSG-Router-Turn-Id: turn_1c9e..." \
  -d '{"model":"auto","messages":[ user, assistant(tool_calls), tool(result) ],"tools":[...]}'

The Router returns the saved ranking result for this turn, ensuring the model is not switched mid-loop.

Turn 2, User sends a new message: Include Session only, omit Turn.

curl -i $AIGW/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -H "CSG-Router-Session-Id: ses_8f3a..." \
  -d '{"model":"auto","messages":[ ...history..., {"role":"user","content":"Now change it to non-recursive"}]}'
# Router re-evaluates ranking with conversation context and returns a new Turn-Id

Python (OpenAI SDK) Example:

from openai import OpenAI

client = OpenAI(base_url=f"{AIGW}/v1", api_key=KEY)
sid = tid = None

def call(messages, new_turn):
    global sid, tid
    headers = {}
    if sid:
        headers["CSG-Router-Session-Id"] = sid
        if not new_turn and tid:
            headers["CSG-Router-Turn-Id"] = tid
    raw = client.chat.completions.with_raw_response.create(
        model="auto", messages=messages, extra_headers=headers)
    sid = raw.headers.get("CSG-Router-Session-Id") or sid
    tid = raw.headers.get("CSG-Router-Turn-Id") or tid
    return raw.parse()

# User sends new message -> call(msgs, new_turn=True)
# Tool execution result returned -> call(msgs, new_turn=False)

3. Header Presence Comparison​

ScenarioBehaviorConsequence
Omit all headersEach request is treated as a new session with independent rankingFine for single-turn Q&A. In Agent scenarios, the model may change across iterations within the same tool loop, causing inconsistent tool_call formats and reasoning styles, plus higher latency from repeated ranking
Session only (New turn)Re-evaluates ranking within the same session and generates a new TurnRecommended for every new user question
Session + Turn (Within same turn)Reuses the turn's saved ranking result; model remains unchangedRecommended for tool loops and retries
Turn onlyGateway drops Turn; equivalent to omitting headersNo error, but silently loses session stickiness
Fabricated / Invalid ID formatSilently dropped; treated as a new sessionSame as above; hard to detect during debugging
X-Session-ID: <arbitrary>Not treated as Router Session, only used for upstream affinity; disables capacity-aware upstream re-selection and pins to the hash-selected upstreamNo stickiness across models, but upstream affinity is maintained, improving cache hit rates
X-Session-ID: ses_xxxActs as both Router Session and upstream affinity keyConvenient for legacy clients already sending X-Session-ID; overridden by CSG-Router-Session-Id if both are present
Responses API with previous_response_idSession already bound to an upstream; reuses model owning that upstream without invoking RouterNo ranking, no fallback candidates, returns no Router headers; fails immediately if the bound upstream is unavailable

Summary​

Set model to auto. Extract Session and Turn from response headers: include both within the same turn, and include only Session when starting a new turn. Never fabricate IDs on the client side.

Connect and Call Models · Routing and Reliability · Performance and Cost Optimization