pplx-computer-qwen-3-8-27b-dflash2-20260824

模型发布 Hugging Face 模型 · perplexity-ai 原文语言:中文

官方机构模型仓库创建于 2026-08-25。仓库创建时间不等于正式发布日期。任务类型:未提供

全文

pplx-computer-qwen-3-8-27b-dflash2-20260824

The qwen3.8_27b RFT checkpoint (Qwen3.8-27B, arch class Qwen3_5ForConditionalGeneration, RFT/SDPO tools post-training) quantized to NVFP4 (ModelOpt mixed-precision mirroring RadixArk/Qwen3.8-27B-NVFP4's recipe, FP8 KV cache) with one deliberate change: an unquantized bf16 lm_head (DFlash2 refuses a quantized one — every public NVFP4 quant packs it to FP4), plus baked-in dspark2 MTP draft tensors (mtp.* in model-mtp.safetensors, referenced by model.safetensors.index.json).

How to run (DGX Spark / GB10)

Everything needed is in serving/. Stock vLLM can not serve this with DFlash2 — it needs the pure-Python vLLM PR #52816 overlay plus a one-line lm_head fix (see notes below), packaged here as two Dockerfiles.

Serve flags

Notes / gotchas

The DFlash2 draft is vendored at draft/ (a mirror of incoai/Qwen3.8-27B-DFlash2, 2026-08-19) and referenced by local path in the compose — the repo is fully self-contained, no external downloads at serve time. The draft is separate from the target weights: the in-checkpoint mtp.* tensors are the dspark2 MTP module — MTP k=7 reaches similar acceptance but decodes ~2× slower (sequential draft passes), so DFlash2 is the recommended path. bf16 lm_head is load-bearing. DFlash2 requires an unquantized target lm_head; it is excluded via exclude_modules in both hf_quant_config.json and config.json (vLLM's modelopt loader keys on the exclude wildcards, not on absence from quantized_layers). Why Dockerfile.lmheadfix: vLLM gives an excluded lm_head an UnquantizedLinearMethod, which PR #52816's isinstance check wrongly rejects (it only accepts UnquantizedEmbeddingMethod). The second image layer relaxes that check. --max-num-batched-tokens 8192 is required: with spec decode enabled, vLLM otherwise clamps scheduling to 2048 and prefill collapses.

正文以来源语言显示。

参考来源

← 返回每日简报