Skip to main content

CSGHub Release Notes v2026.07

1 Prefill-Decode (PD) Disaggregation Architecture (EE)​

The inference engine officially introduces the Prefill-Decode (PD) disaggregated deployment and resource scheduling architecture, optimizing computing resource utilization while significantly boosting inference throughput and reducing time-to-first-token (TTFT) latency for LLMs under high-concurrency workloads.

  • Prefill-Decode (PD) Deployment and Scheduling: Supports decoupling Prefill (first token generation) and Decode (subsequent token generation) onto separate compute nodes, allowing fine-grained optimization and resource isolation tailored to stage-specific workloads.

2 Inference Engine Upgrades & Heterogeneous Compute Support​

Comprehensively updated mainstream open-source inference engines while expanding hardware compatibility for heterogeneous compute.

  • Mainstream Inference Engine Upgrades: Fully upgraded vLLM and SGLang to their latest stable releases, alongside adding compatibility for GLM-5.1 inference services to ensure smooth deployment of state-of-the-art open models.
  • Heterogeneous Compute Support (EE): Expanded platform support for specialized compute hardware, adding llama.cpp model deployment capabilities optimized for the AMD (ROCm) architecture.

3 AI Gateway Multi-Modal Inference Enhancements (EE)​

Delivered full-stack upgrades for multi-modal tasks and API execution across AI Gateway, enhancing interactive online experiences.

  • Image-to-Image Inference Component: Added new Image-to-Image model inference components and interactive playground UI, empowering image-guided multi-modal generation and processing workflows.
  • Streaming Audio Inference Services: Upgraded Audio API endpoints to support streaming transmission (Stream); integrated upstream audio duration validation for self-hosted ASR models to reduce latency and improve accuracy.
  • Feature Extraction & API Usability: Adapted Feature Extraction inference components for AI Gateway with interactive API call samples; streamlined model ID paths for Serverless and inference endpoints.

4 ClawEval Evaluation Framework Integration​

Elevated model evaluation capabilities by providing developers and enterprises with out-of-the-box automated LLM evaluation.

  • ClawEval Evaluation Framework: Integrated the open-source ClawEval benchmark framework, offering standardized multi-dimensional LLM capability evaluations and analytical reports to help users comprehensively assess model performance.

5 Business Analytics & Superset BI Dashboard Integration (EE)​

Provides platform administrators with intuitive, multi-dimensional business analytics and compute resource insights.

  • Superset BI Integration: Successfully integrated the open-source Superset BI engine to enable deep data mining across platform operational stats, user activity, and compute usage.
  • Executive Analytics Dashboard: Introduced executive analytics dashboards presenting compute consumption trends, resource utilization, and cost breakdowns to support data-driven decision-making.

6 Agent Infrastructure & Sandbox Hardening (EE)​

Reinforced security isolation and runtime stability across Agent infrastructure.

  • Agent Sandbox Stability: Implemented deep runtime isolation and stability optimizations for Agent sandboxes, safeguarding system security and reliability during execution of long-running, complex tasks.

7 LLM Log Data Workflow & Repository Sync Refactoring​

Completed the end-to-end data governance pipeline for LLM logs and refactored underlying repository and mirror synchronization architecture.

  • LLM Log-to-Dataset Workflow: Supported automated syncing of online LLM logs to Datasets, seamlessly linking data processing and annotation stages; configured dynamic JetStream retention policies to control storage growth.
  • Repository and Mirror Sync Refactoring: Overhauled sync mechanisms with authentication (Username + Token), retriable/non-retriable failure handling, and real-time frontend progress indicators, resolving timeout bottlenecks for large assets and Skill repositories.

Historical Versions​

Click Release List to view all historical versions.