How to Run Qwen 3.8 27B Locally: Ollama, GGUF, and Single-GPU Setup (2026)

The download is the easy part. Picking the quant that fits your VRAM is the decision that matters.

Weights status: live. Alibaba released Qwen3.8-27B on Hugging Face in the August 13-14 rollout under an Apache 2.0 license, at the official repo Qwen/Qwen3.8-27B. And the spec sheet has a surprise: it's not text-only. The model ships with a vision encoder, 28B parameters, and a 262k native context window extensible to 1M tokens.

This guide is the fastest path from zero to tokens: the official download, which quantization fits your hardware, the Ollama and llama.cpp routes for local use, and the vLLM route when you want a real endpoint. If you're still deciding between the 27B and the API flagship, start with our full lineup comparison and come back.

TL;DR

• Weights: live at Qwen/Qwen3.8-27B under Apache 2.0. 28B parameters, BF16 safetensors
• Confirmed shape: 28B dense with hybrid attention, a vision encoder (yes, it's multimodal), and 262k native context, extensible to 1M
• Easiest local route: Ollama with a 4-bit quant, roughly 17GB of VRAM or unified memory
• Quality route: FP8 on a 48GB card, BF16 on an 80GB-class card
• Production route: vLLM or SGLang with an OpenAI-compatible endpoint
• No local hardware? Single GPUs by the hour on Yotta GPU Pods, from a 4090 to an H200

Where to Download Qwen 3.8 27B

The official repository is Qwen/Qwen3.8-27B on Hugging Face, released under Apache 2.0, one of the most permissive licenses available: commercial use, modification, and redistribution are all allowed.

Check the publisher name is Qwen before downloading. Placeholder and fork repos squatted on this model's name for weeks before release, and some are still around.

The quant ecosystem showed up within hours: the model card already lists quantized builds across llama.cpp, LM Studio, Jan, and Ollama. For most single-GPU setups, a community GGUF quant is the practical download rather than the BF16 originals.

Which Quant Will Fit Your GPU

The spec sheet is out: 28B parameters, dense with hybrid attention. The memory math, weights only, before KV cache:

KV cache comes on top and scales with context and concurrency, and it matters more here than on most models this size: 262k of native context is an invitation to long sessions that quietly eat several extra GB. Leave headroom past the weights number. The mechanics are in our breakdown of what actually limits inference speed.

If your card is borderline, drop the quant, not the context. A 4-bit build with room for KV cache beats an FP8 build that OOMs mid-conversation.

Route 1: Ollama (Fastest Local Start)

The official Ollama library entry is live: an 18GB quantized build with vision support, plus an MLX variant for Apple Silicon. The whole setup is:

ollama pull qwen3.8:27b
ollama run qwen3.8:27b

(The default tag resolves to the 27B; Apple Silicon users can pull qwen3.8:27b-mlx.) Ollama serves an OpenAI-compatible endpoint on localhost:11434 out of the box, so existing client code points at it with a one-line base URL change.

We also published a dedicated Ollama guide with the tags table, the context-window fix, and the vision setup.

Route 2: llama.cpp / GGUF (Most Control on Consumer Hardware)

Grab the GGUF that fits from the community quant repos, then:

llama-server -m qwen3.8-27b-Q4_K_M.gguf -c 8192 --port 8080

Pick your quant by the VRAM table above; Q4_K_M has been the community sweet spot for the 3.6 generation. This is also the route for Macs: a 4-bit 27B-class GGUF runs on Apple Silicon with roughly 24GB or more of unified memory.

Route 3: vLLM or SGLang (Real Serving)

For anything beyond personal use, run a proper engine:

vllm serve Qwen/Qwen3.8-27B

That's the confirmed official repo name. Qwen releases have shipped with day-one support in vLLM and SGLang historically; check each engine's release notes for 3.8-specific flags, especially around the vision encoder. Either engine gives you an OpenAI-compatible endpoint, continuous batching, and sane multi-user throughput. If you're choosing between them, our vLLM vs SGLang comparison covers the decision, and the production Docker guide covers deployment mechanics.

When Local Isn't Enough

A 27B on your desk is great for evaluation and personal agents. The moment it needs to serve a team or an app, the constraint stops being "does it fit" and becomes throughput, concurrency, and uptime, and that's a rented-GPU problem, not a bigger-desktop problem. The full production guide covers that move end to end.

Single GPUs in every class in the table above are available by the hour on Yotta GPU Pods: a 4090 to sanity-check the quantized build, an L40S or RTX Pro 6000 for FP8 serving, an H100 or H200 when you want BF16 quality with KV headroom. And if the goal is making the model yours rather than just running it, fine-tuning fits on the same rented cards: our Unsloth fine-tuning guide covers the setup. The full sizing logic is in our Qwen 3.8 27B hardware guide, and the same playbook we used for running Qwen 3.7 in production applies here.

Frequently Asked Questions

Is Qwen 3.8 27B on Hugging Face?

Yes, as of the August 13-14 rollout: Qwen/Qwen3.8-27B, under Apache 2.0. Download from the official Qwen org, not the name-alike community repos that predate the release.

What GPU do I need to run Qwen 3.8 27B locally?

A 24GB card (RTX 4090 class) runs 4-bit quants at roughly 14 to 17GB. FP8 wants a 48GB card. BF16 wants 80GB-class hardware at ~56GB of weights. Add headroom for KV cache in every case.

Can I run Qwen 3.8 27B on a Mac?

Yes: a 4-bit GGUF fits Apple Silicon machines with roughly 24GB or more of unified memory, through llama.cpp, LM Studio, or Ollama.

Will Qwen 3.8 27B work with Ollama?

Yes. The official library entry is live: an 18GB download with vision support. Our Ollama guide covers the commands, the context-window setting, and the MLX variant.

What license is Qwen 3.8 27B?

Apache 2.0, which permits commercial use, modification, and redistribution. That's the permissive outcome the ecosystem was hoping for.

Is Qwen 3.8 27B better than Qwen3.6-27B?

The model card publishes benchmark results (including 61.7 on SWE-bench Pro), which is more than the 3.8 flagship has shown so far. Independent replication is still pending, so run your own eval before switching production workloads.

Should I run the 27B locally or use Qwen 3.8-Max's API?

Different jobs. The 27B gives you weights, privacy, and fixed costs on your own hardware; the Max gives frontier capability at $2 in / $6 out per million tokens with zero infrastructure. The full decision is in our lineup comparison.

Bottom Line

Qwen3.8-27B landed better than the rumor mill guessed: Apache 2.0, a vision encoder nobody promised, 262k of native context, and a quant ecosystem that showed up within hours. The waiting phase is over; the running phase starts now.

The path of least resistance: Ollama or a GGUF with a 4-bit quant if you have 24GB, FP8 on a 48GB card if you want quality without the flagship bill, vLLM the moment anyone else depends on your endpoint. And if the model you want to run outgrows the GPU on your desk, that's a rental, not a purchase: single GPUs by the hour on Yotta GPU Pods, sized to whatever precision you land on.