---
title: "How to Deploy GLM 5.2 with SGLang on Yotta GPU Pods"
slug: how-to-deploy-glm-5-2-with-sglang-on-yotta-gpu-pods
description: "A step-by-step guide to self-hosting GLM 5.2 on Yotta GPU Pods with SGLang: hardware, the FP8 checkpoint, the serve command, and how to verify the OpenAI-compatible endpoint."
author: "Yotta Labs"
date: 2026-07-07
categories: ["Products"]
canonical: https://www.yottalabs.ai/post/how-to-deploy-glm-5-2-with-sglang-on-yotta-gpu-pods
---

# How to Deploy GLM 5.2 with SGLang on Yotta GPU Pods

![](https://cdn.sanity.io/images/wy75wyma/production/b0d0596269db514a49cc6b6dbc2fd29adf2b2b83-1200x627.png)

We recently covered [deploying GLM 5.2 with vLLM](https://www.yottalabs.ai/post/how-to-deploy-glm-5-2-with-vllm-on-yotta-gpu-pods) on Yotta GPU Pods. This is the same deployment with SGLang. Same model, same hardware, same Pod setup. Different engine, and a few differences worth knowing about before you pick.

Quick recap on the model: GLM 5.2 is a roughly 753B mixture-of-experts model with about 40B active parameters, open weights under MIT, and a 1M-token context. At FP8 it needs around 744 GB of VRAM, which in practice means an 8x H200 node.

If you would rather not run hardware at all, GLM 5.2 is also on the [Yotta AI Gateway](https://yottalabs.ai/ai-gateway) as an OpenAI-compatible API. This guide is for the self-host path.

## What you need

- A GPU Pod sized for the model. For FP8, an 8x H200 node (141 GB each, ~1128 GB total) gives you comfortable headroom over the ~744 GB the FP8 weights need.
- The FP8 checkpoint: zai-org/GLM-5.2-FP8 on Hugging Face. Same checkpoint the vLLM guide uses.
- SGLang. The `lmsysorg/sglang:latest` Docker image has everything, including the DeepEP and speculative decoding dependencies the recipes below use.

One difference from the vLLM path: SGLang's [official GLM-5.2 recipe](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.2) does not list an INT4 build. On Hopper, FP8 on 8x H200 is the single-node setup. There is an NVFP4 checkpoint at roughly half the size, but it is Blackwell-only (B200 and up), so it does not buy you a smaller H200 or H100 footprint.

Versions and flags for GLM 5.2 move fast. Confirm the current ones against the official recipe and the model card before you deploy. The steps below are the shape, not a frozen spec.

## Step 1: Launch the GPU Pod

In the [Yotta console](https://console.yottalabs.ai/), launch a GPU Pod with an 8x H200 configuration. Attach a volume large enough to hold the checkpoint so you are not re-downloading ~750B of weights every time the Pod restarts, and expose a port for the inference server (30000 is the SGLang default). If you prefer to bake SGLang and its dependencies into an image rather than installing at runtime, Yotta supports custom images, which cuts cold-start time on later launches.

For a walkthrough of Pod setup itself, the current Pods docs are the source of truth.

## Step 2: Serve GLM 5.2 with SGLang

The simplest path is the SGLang Docker image:

```bash
docker run --gpus all \
  --shm-size 32g \
  -p 30000:30000 \
  --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang:latest \
  sglang serve \
    --model-path zai-org/GLM-5.2-FP8 \
    --tp 8 \
    --speculative-algorithm EAGLE \
    --speculative-num-steps 5 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 6 \
    --mem-fraction-static 0.8 \
    --tool-call-parser glm47 \
    --reasoning-parser glm45 \
    --host 0.0.0.0 \
    --port 30000
```

What the important flags do:

- `--tp 8` shards the ~753B weights across all eight GPUs, same as tensor parallel in vLLM.
- The `--speculative-*` flags turn on EAGLE speculative decoding, which drafts multiple tokens per step and is one of the things SGLang ships in its default GLM 5.2 recipe rather than as an add-on. This config is tuned for low-latency chat.
- `--tool-call-parser glm47` and `--reasoning-parser glm45` wire up tool calling and thinking mode. GLM 5.2 emits a newer tool-call format that specifically needs the glm47 parser, so don't swap these.
- `--mem-fraction-static 0.8` is the recipe's starting point on H200. Tune up from there.

Two things you do not need to set. SGLang auto-selects the KV cache dtype for this model (BF16 on Hopper, FP8 on Blackwell) and the attention backends, so there is no `--kv-cache-dtype` flag in the recipe. And one caveat: with speculative decoding on, SGLang caps `--max-running-requests` at 48 unless you set it yourself.

The first run downloads the checkpoint, which is large, so give it time and make sure it lands on your mounted volume.

## Step 3: Pick your recipe for the workload

SGLang publishes three verified configurations for GLM 5.2 on 8x H200, and the differences are real.

The command above is the low-latency recipe, built for chat-style traffic and small concurrency. For typical multi-user serving, the balanced recipe adds data parallel attention (`--dp 8 --enable-dp-attention`), the DeepEP MoE backend, `--chunked-prefill-size 32768`, and `--max-running-requests 256`, with lighter speculative settings. For pure batch throughput it drops speculative decoding entirely.

Per SGLang's own testing, raising chunked prefill to 32768 on the balanced recipe gave +34 to 78% output throughput at 8K-token inputs. That is a vendor-published number, so validate it on your own workload, but the direction matches what we see: at long context, the default prefill chunk is the bottleneck.

On the 1M context window, the same advice from the vLLM guide applies. The KV cache grows with context length, and memory reserved for context you never use is memory you are not spending on batching. If your prompts run 32K, cap the context near there instead of serving the full million. We broke down the underlying memory math in [why GPU utilization is low in LLM inference](https://www.yottalabs.ai/post/why-gpu-utilization-is-low-in-llm-inference-and-how-to-fix-it).

## Step 4: Verify the endpoint

Once the server is up, it speaks the OpenAI API on port 30000:

```bash
curl http://<your-pod-address>:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.2-FP8",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}]
  }'
```

A clean JSON response means GLM 5.2 is serving. Point any OpenAI-compatible client at the Pod by swapping the base URL and model name. SGLang also exposes an Anthropic-compatible `/v1/messages` endpoint on the same server, which is handy if some of your tooling speaks that API instead.

## vLLM or SGLang?

Both serve the GLM 5 family well. We compared them in [vLLM vs SGLang](https://www.yottalabs.ai/post/vllm-vs-sglang-which-inference-engine-should-you-use-in-2026) if you want the full breakdown.

The short version for this model: SGLang's GLM 5.2 recipe ships speculative decoding, DP attention, and auto-selected attention backends as the verified default, which means less flag archaeology to get to a tuned config. vLLM's recipe gives you explicit control over things like KV cache dtype and integrates with a broader plugin ecosystem. If your workload is chat-heavy or agent-heavy with lots of shared prefixes, SGLang's defaults, plus its HiCache prefix caching for long-context work, are a strong starting point. Benchmark both on your traffic before committing either way.

## When to skip all this

Same math as the vLLM guide. Self-hosting pays off at high, steady utilization, because a dedicated node costs the same idle or loaded. If your traffic is bursty or you are still evaluating GLM 5.2, call it through the [Yotta AI Gateway](https://yottalabs.ai/ai-gateway) and skip the ops entirely. Check current GPU rates on [pricing](https://yottalabs.ai/pricing) to run that math.

## FAQ

**What GPUs do I need to run GLM 5.2 with SGLang?** At FP8, about 744 GB of VRAM, which is an 8x H200 node. SGLang's recipe does not list an INT4 build; the smaller NVFP4 checkpoint requires Blackwell GPUs.

**Is it the same checkpoint as the vLLM deployment?** Yes. zai-org/GLM-5.2-FP8 serves on both engines. Switching engines does not mean re-downloading weights.

**Does SGLang serve an OpenAI-compatible API?** Yes, on port 30000 by default. It also exposes an Anthropic-compatible /v1/messages endpoint on the same server.

**Do I need to set the KV cache dtype?** No. SGLang auto-selects it for GLM 5.2: BF16 on Hopper, FP8 on Blackwell.

**Should I use vLLM or SGLang for GLM 5.2?** Both work well. SGLang's default recipe includes speculative decoding and tuned multi-user configs out of the box. Benchmark on your own workload, and see [vLLM vs SGLang](https://www.yottalabs.ai/post/vllm-vs-sglang-which-inference-engine-should-you-use-in-2026) for the detailed comparison.

**Can I run this without managing a Pod?** Yes. GLM 5.2 is on the [Yotta AI Gateway](https://yottalabs.ai/ai-gateway) as an OpenAI-compatible API. Self-host when volume and control justify the ops.

## Bottom line

Deploying GLM 5.2 with SGLang comes down to the same three things as the vLLM path: an 8x H200 node, the GLM-5.2-FP8 checkpoint, and a serve command matched to your workload. The SGLang-specific parts are the recipe strategy you pick, the glm47 tool-call parser, and the fact that speculative decoding and KV cache handling come pre-configured. Confirm versions against the official SGLang recipe, verify with a single curl, and you have a frontier open-weight model running on hardware you control.

Spin up a Pod in the [Yotta console](https://console.yottalabs.ai/), or if you would rather just call an API, reach GLM 5.2 through the [Yotta AI Gateway](https://yottalabs.ai/ai-gateway).