WaveView:噪声中的新视角生成

媒体报道 Hugging Face Papers 原文语言:中文

WaveView 在预训练扩散生成器的初始噪声相位中注入经过扭曲的源图像,利用双树复小波包变换实现局部各向异性相位注入,从而在遮挡和视线线性区域保持更少约束,在 DL3DV 基准上所有重叠区间均优于对手,最难区间提升 0.84 dB PSNR 并加速 2.1 倍。

全文

Generative novel view synthesis (NVS) employs a diffusion model to render a static or dynamic scene from a target camera view. Explicit warp methods complete a reprojection of the source with video diffusion models, and implicit methods encode the camera parameters or 3D cues. Both add inference overhead, and the generated layout often drifts from the ground truth. We present WaveView, which injects the warped source into the phase of the initial noise of a pretrained generator. The Dual-Tree Complex Wavelet Packet Transform (DT-CWPT) allows local and anisotropic phase injection. The generator is thus constrained less in disoccluded regions and along the epipolar line. WaveView keeps the architecture and adds negligible overhead. On the DL3DV benchmark, it outperforms all baselines in every overlap bin. In the hardest bin, it gains 0.84 dB PSNR on the covisible region over MetaView, which shares its generator, while running 2.1 times faster. For dynamic NVS, fine-tuning AnyView with the same noise raises its zero-shot PSNR on AnyViewBench from 11.58 to 12.35 dB.

View arXiv page View PDF Add to collection

正文以来源语言显示。

参考来源

← 返回每日AI报告