PixVid 将潜空间视频扩散提升到像素空间

研究论文 Hugging Face Papers 原文语言:中文

论文提出 PixVid,把预训练潜空间视频扩散模型提升到像素空间,直接逐帧生成视频。它保留扩散 Transformer,用卷积头替换 VAE 解码器,目标是在高分辨率、可控和交互式生成上突破潜空间压缩的限制;结果来自作者报告。

全文

Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation -- doubling resolution quadruples token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Finetuned from Wan2.1-14B, PixVid is competitive with the latent baseline within sim 1.3M training clips. Working framewise in pixels, it generates at 2times higher resolution with the same per-frame token count, keeps depth-conditioned video editing aligned under fast motion, and responds to each action at the next frame. Project page: https://chenshuo20.github.io/PixVid/

View arXiv page View PDF Project page Add to collection

正文以来源语言显示。

参考来源

← 返回每日简报