TwinD: Generalizing Deepfake Detection from Blending to Entire Face Synthesis

Media reports Hugging Face Papers Source language: Chinese

论文摘要(作者报告,未经本站复测):Facial deepfake detectors are usually trained on blending fakes, where a generated face is pasted back into a real frame. Recent generators synthesize the entire face and leave no traces of blending.

Full text

TwinD: Generalizing Deepfake Detection from Blending to Entire Face Synthesis

Andrii Yermakov Affiliation: Czech Technical University in Prague Email: yermaand@fel.cvut.cz Jan Cech Affiliation: Czech Technical University in Prague Email: cechj@fel.cvut.cz Mario Fritz Affiliation: CISPA Helmholtz Center for Information Security Email: matas@fel.cvut.cz Jiri Matas Affiliation: Czech Technical University in Prague Email: fritz@cispa.de

Abstract

Facial deepfake detectors are usually trained on blending fakes, where a generated face is pasted back into a real frame. Recent generators synthesize the entire face and leave no traces of blending. On a benchmark of 19 datasets released from 2019 to 2026, the best prior facial deepfake detector reaches a mean AUROC of 96.2% on ten blending datasets, but none exceeds 78.1% on five datasets of entire face synthesis (EFS). AI-generated content (AIGC) detectors do not close this gap. We propose TwinD, a detector trained on FaceForensics++ fakes and two kinds of twins: fakes made from a real frame. A reconstruction twin is a real frame reconstructed by an autoencoder, with the real pixels put back in a random region. It targets the neural traces of a decoder. A blob twin blends an altered copy of a real frame with the frame itself, targeting blending artifacts. It needs no generator and no face-shaped mask. Together, these twins provide two complementary forensic cues that generalize to unseen manipulations. TwinD reaches a mean AUROC of 96.3%, 8.9 points above the prior state-of-the-art, with 98.1% on the blending datasets and 95.6% on the EFS datasets. Trained on twins alone, without a deepfake dataset, single models still reach 94.3% on average over three seeds. For reproducibility, the code, trained model, and training data are at: https://github.com/yermandy/TwinD

1 Introduction

Figure 1: Overview of TwinD. (a) A twin is a fake made from a real frame. A reconstruction twin is an autoencoder reconstruction with the real pixels put back inside a random grid or noise mask. A blob twin blends an altered copy of the frame with the frame itself through a blob mask. Right halves of the twins show the difference to the real frame. The traces of the autoencoder are amplified 10×10\times for visualization. (b) Each fake is paired with the real frame it was made from. Both get the
Figure 1: Overview of TwinD. (a) A twin is a fake made from a real frame. A reconstruction twin is an autoencoder reconstruction with the real pixels put back inside a random grid or noise mask. A blob twin blends an altered copy of the frame with the frame itself through a blob mask. Right halves of the twins show the difference to the real frame. The traces of the autoencoder are amplified 10×10\times for visualization. (b) Each fake is paired with the real frame it was made from. Both get the

Figure 1: Overview of TwinD. (a) A twin is a fake made from a real frame. A reconstruction twin is an autoencoder reconstruction with the real pixels put back inside a random grid or noise mask. A blob twin blends an altered copy of the frame with the frame itself through a blob mask. Right halves of the twins show the difference to the real frame. The traces of the autoencoder are amplified 10×10\times for visualization. (b) Each fake is paired with the real frame it was made from. Both get the same augmentation, so they differ only by the manipulation. TwinD is a frozen PEcoreL with LoRA, trained on batches of such pairs. (c) Mean video-level AUROC (%) on the EFS and Blending datasets of Tab. 1. Gray bars show the best prior detector of each kind.

Most facial deepfake datasets contain blending fakes, where a swapped or reenacted face is blended back into the real frame [36, 8, 28, 59, 21, 18, 15, 51, 20]. Recent generators synthesize the entire video frame without blending, and datasets of such entire face synthesis (EFS) fakes have appeared only recently [50, 26, 30, 43, 40]. A facial deepfake detector has to catch both kinds. The fundamental challenge is to generalize to unseen generators.

Facial deepfake detectors [38, 54, 6, 60, 47, 42, 57] are usually trained on the FaceForensics++ dataset (FF++) [36], which contains only blending fakes. They can learn the traces of blending and generalize well to other blending datasets [25, 38]. EFS leaves no such traces. In Tab. 1, the best of them reaches a mean AUROC of 96.2% on ten blending datasets, but none exceed 78.1% on five EFS datasets.

Detectors of AI-generated content (AIGC) come from a separate line of research. They target fully synthetic images and promise to generalize to many unseen generators [31, 52, 55, 5, 10]. Some of them are trained on real images and their generative reconstructions [4, 5, 10, 35], which differ from the real images only by the neural traces of the generator. So, they could be expected to handle EFS deepfakes. They do not. The best AIGC detector reaches a mean AUROC of 73.8% on the EFS datasets, below the best facial deepfake detector. On the blending datasets, none exceeds 71.4%.

No evaluated detector works well on both blending and EFS fakes. We close this gap with a better training set. We introduce two kinds of twins, “pseudo-fakes” that are made from a real frame. Reconstruction twins improve generalization on EFS fakes and blob twins on blending fakes, see Fig. 1.

A reconstruction twin is a real face frame reconstructed by an autoencoder, as in AIGC detectors, but with the real pixels put back inside a random mask. So, the detector learns decoder traces even when they cover only part of the frame.

A blob twin blends an altered copy of a real frame with the frame itself through a smooth round mask, so it needs no generator. It simplifies the popular self-blended images (SBI) [38]: the mask is round instead of aligned with the facial boundaries, so it needs no landmarks.

We propose TwinD, a detector trained on FF++ fakes and two kinds of twins. Each fake is paired with the real frame from which it was made, and both receive the same training augmentation, so the only difference between them is the manipulation. The contributions of this paper are:

1.

The proposed TwinD is evaluated against 15 recent detectors on 19 facial deepfake datasets released from 2019 to 2026. To our knowledge, it is the largest evaluation of released detectors on the same face crops that covers both older blending fakes and recent EFS fakes. TwinD outperforms the previous state-of-the-art by a large margin – a mean AUROC of 96.3%, 8.9 points above the best prior detector, with 98.1% on the blending datasets and 95.6% on the EFS datasets. It reaches at least 90.4% on each of the 19 datasets.

2.

We show that the two kinds of twins are complementary: blob twins cover blending fakes, and reconstruction twins cover EFS fakes. Trained on twins alone, without seeing any deepfake dataset, single models still reach AUROC of 94.3% averaged over three seeds, which is above all 15 prior detectors.

2 Related work

Pseudo-fakes

Pseudo-fake methods build training fakes from real images by using only non-generative manipulations. Face X-ray [25] pastes the face of another real person, matched by facial landmarks, into a frame. Self-blended images (SBI) [38] replace a person’s face with an altered copy of the same face, blend it through a face-shaped mask built from facial landmarks, and randomize the mask and the geometry of the copy. The proposed blob twins (Sec. 3.1.4) are simpler than both. Their mask is a round blob instead of a face shape, so it needs no landmarks, and a small shift is the only change in geometry. They outperform SBI on the blending datasets (Tab. 4).

Training on reconstructions

Current state-of-the-art AIGC detectors are trained on real images and their reconstructions, so the two classes differ only by the traces of the generator. DRCT [4] and B-Free [10] reconstruct real images using a diffusion model. AlignedForensics [35] and DDA [5] use only the autoencoder of a latent diffusion model. AlignedForensics also puts each reconstruction in the same batch as its real image and gives both the same augmentation, which TwinD does as well (Sec. 3.1.5). Two methods also mix real pixels back into the reconstruction. DDA averages the reconstruction and the real image with one random weight, so no region remains purely real. B-Free puts the real pixels back outside one object, marked by its segmentation mask or bounding box, and stores the result in its training set. All of them are trained on general images, and the methods we evaluate, AlignedForensics, DDA, and B-Free, do not transfer to facial deepfakes (Tab. 1). Reconstruction twins (Sec. 3.1.3) build on these methods. They are reconstructions of real face frames, with the real pixels put back inside a random mask. Unlike in B-Free, the mask needs no segmentation labels and is redrawn at every use.

Parameter-efficient fine-tuning

Recent deepfake detectors adapt a large pre-trained backbone by fine-tuning a small part of it. GenD [57] adapts the image encoders of PEcoreL [3], CLIP [34] and DINOv3 [41] by training only their LayerNorm parameters (LN-tuning [58]). Effort [54] adapts CLIP by training the last singular components of each weight, like MiLoRA [48]. DDA [5] trains LoRA [16] on a DINOv2 [32] backbone. TwinD trains LoRA on PEcoreL (Sec. 3.2).

Benchmarks

Facial deepfake detectors are commonly trained on FF++ [36] and tested on a few other datasets. DeepfakeBench [53] unified the preprocessing and evaluation protocol. Its test datasets, like those used by most detectors, contain mostly blending fakes. Recent EFS fakes [50, 26, 30, 43, 40] are not yet part of this protocol. GenD [57] uses 14 datasets from 2019 to 2025. The benchmark of this paper (Sec. 4.1) includes 19 datasets released between 2019 and 2026 and groups them by how their fakes are made. For a fair comparison, it crops the faces in the same way for all detectors and keeps the resizing and normalization each detector was trained with.

Table 1: Video-level AUROC (%) of SOTA methods on 19 datasets sorted by their macro-average (Avg). Above the dataset names are their release years. †A temporal model, which takes several frames as input in a single forward pass.

Blending Blending + EFS EFS

2019 2020 2020 2021 2021 2022 2024 2024 2024 2025 2025 2025 2025 2026 2026 2026 2026 2026 2026

DFD FSh CDFv2 FFIW FAVC DFDM PGF IDF DSv1 HiDF CDFv3 DSv2 RF HF THB OF DFAV MV DF26 Avg

AIGC detectors

Effort-SDv1.4 [54] 51.5 58.9 48.8 56.5 55.4 55.3 38.4 63.7 31.6 55.0 63.5 51.5 79.4 75.8 79.6 77.3 48.9 38.6 62.3 57.5

DGS-Net [52] 58.4 71.9 55.4 73.6 45.1 72.8 49.4 38.1 41.5 47.8 71.9 41.7 56.8 66.2 66.0 49.0 68.5 69.8 75.0 58.9

LTD-UFD [55] 83.3 77.2 57.2 72.6 61.9 52.1 60.4 50.9 47.2 64.6 66.2 38.7 64.1 53.2 75.9 30.8 75.1 55.9 87.6 61.8

AlignedFor. [35] 55.0 62.9 61.0 78.3 93.2 65.5 56.8 93.7 43.0 47.8 28.6 74.8 68.8 90.3 93.5 56.9 83.6 40.0 67.8 66.4

DDA [5] 64.4 53.2 55.5 57.5 73.8 51.6 60.2 70.2 71.2 66.8 60.4 84.2 75.4 97.0 85.3 36.5 93.5 71.2 75.1 68.6

B-Free [10] 69.4 69.6 57.1 58.1 79.9 74.6 78.5 75.8 82.0 69.4 59.5 82.7 77.2 98.5 86.2 48.6 91.6 74.8 68.0 73.8

Facial deepfake detectors

SBI [38] 90.2 77.2 87.5 90.9 98.7 99.8 75.0 98.4 63.1 82.1 59.9 59.2 62.1 61.4 71.2 75.9 57.8 43.3 60.1 74.4

FSFM [46] 90.8 72.2 90.5 78.4 90.9 96.8 85.8 82.5 83.5 91.9 79.6 69.0 66.2 72.1 47.4 49.5 74.2 62.7 52.6 75.6

ForAda [6] 98.1 81.3 95.5 92.8 93.1 97.3 87.3 90.7 83.0 91.6 75.6 73.3 69.5 57.5 60.2 65.1 73.1 65.7 56.5 79.3

Effort [54] 96.6 91.3 93.4 95.1 92.4 98.1 86.8 96.0 83.2 90.9 78.7 65.3 64.8 62.6 73.1 75.4 72.5 57.1 44.0 79.9

DFD-FCG† [14] 94.7 91.8 94.0 94.0 90.6 99.6 78.5 92.8 82.7 94.6 93.1 78.5 64.5 65.6 75.3 76.0 65.2 65.4 53.0 81.6

VLAForge [60] 98.1 92.9 95.0 96.8 94.5 99.5 89.7 97.6 87.0 93.9 72.4 70.0 69.0 57.9 77.2 71.7 74.0 62.8 54.3 81.8

FS-VFM [47] 97.8 80.7 94.4 92.0 97.4 98.4 90.8 94.7 92.3 97.3 85.0 79.5 74.6 77.7 61.9 67.3 81.7 60.1 57.7 83.2

DFD-HR [42] 98.7 92.9 94.2 98.0 98.7 99.7 94.6 98.7 90.7 96.3 81.6 83.2 67.5 59.6 81.2 74.8 86.3 80.8 63.4 86.4

GenD-PE [57] 98.6 88.3 95.9 95.4 97.5 98.8 94.3 98.0 89.8 93.7 89.9 78.5 76.6 75.3 83.7 72.3 85.1 79.8 69.7 87.4

TwinD 98.9 96.8 97.1 98.9 99.9 99.9 97.3 99.5 93.5 98.7 90.4 92.1 92.3 96.1 98.5 94.9 93.7 95.0 95.8 96.3

3 Method

TwinD is a frame-level model that outputs the probability that a face crop is fake. It learns from real faces and three kinds of fakes: fakes from the FaceForensics++ dataset, reconstruction twins, and blob twins (Sec. 3.1). A twin is a fake that we make from a real frame. Each fake, twin or not, is paired with the real frame it was made from, and both receive the same augmentation. The only difference between them is the manipulation. The backbone is a pre-trained PEcoreL adapted with LoRA (Sec. 3.2). The final model is the weighted average of three runs with different seeds (Sec. 3.3). At test time, the video score is the mean of its frame scores (Sec. 3.4). The flowchart is shown in Fig. 1.

3.1 Training data

3.1.1 Real faces

Real frames come from two sources: the FaceForensics++ (FF++) training videos [36], 8 evenly spaced frames per video (5,739 frames), and 1,913 random frames from Casual Conversations v2 (CCv2) [33]. This gives a 3 : 1 ratio of FF++ to CCv2 frames. From FF++, we drop face crops with a side of at most 100 px, often false detections.

3.1.2 FF++ fakes

We use all four FF++ generators: Deepfakes, Face2Face, FaceSwap, and NeuralTextures. Each fake is cropped with the face box of its real frame, so the two crops match pixel by pixel. We drop fakes whose ground-truth mask covers less than 10% of the crop, as these are false face detections.

3.1.3 Reconstruction twins

Most recent generators operate in a latent space. Their last step decodes latents into pixels with the decoder of an autoencoder. The decoder leaves traces in every image it outputs, regardless of what the previous generation steps were. A reconstruction twin thus teaches TwinD to detect these traces. We will show that training on the traces of a single decoder is enough to detect fakes from other generators, see Tab. 3.

We pass every real frame II through the CogVideoX video autoencoder [56]. Its reconstruction I′I^{\prime} is aligned with II and differs from it only by the decoder’s traces. These traces cover the whole frame. To make the detector find traces that cover only part of a frame, we put the real pixels back inside a region defined by a binary mask MrM_{\mathrm{r}}.

I^=Mr⊙I+(1−Mr)⊙I′,\hat{I}=M_{\mathrm{r}}\odot I+(1-M_{\mathrm{r}})\odot I^{\prime}, (1)

and label I^\hat{I} as fake, and ⊙\odot denotes the pixel-wise product. This partial reconstruction is the reconstruction twin of II. The real region covers a random share of 0 to 90% of the frame. At 0%, the twin is the full reconstruction. Half of the time, MrM_{\mathrm{r}} is a grid mask: a random set of cells of a 32×3232\times 32 grid. Otherwise, MrM_{\mathrm{r}} is a noise mask: a set of irregular blobs. To make it, we draw white noise over the frame and blur it with a Gaussian filter whose width is 3 to 9% of the image side. The real region is where the blurred noise exceeds the threshold that gives exactly the sampled share of the frame. The mask edges are hard. Figure 1 shows both masks. Details are in the supplementary Sec. S2.1. The masking is meant to reduce overfitting to a specific decoder.

3.1.4 Blob twins

Blending fakes paste a generated/manipulated face into a real frame. This leaves compositing artifacts: the pasted region and the rest of the frame have different image statistics. A blob twin teaches TwinD to detect them.

A blob twin blends an image FF into an image BB inside a round blob defined by a soft mask MbM_{\mathrm{b}}. One image, chosen with equal chance, is a real frame II. The other is an altered copy of II: its contrast and hue are changed, and it is downsampled 2×2\times (nearest-neighbor) and upsampled back (bilinear). The mask blends them into the twin I~\tilde{I}:

I~=s​Mb⊙F+(1−s​Mb)⊙B,\tilde{I}=sM_{\mathrm{b}}\odot F+(1-sM_{\mathrm{b}})\odot B, (2)

where the blend strength ss is 0.3 or 1 with equal probability. The twin has the label fake. The mask is Mb=2−(d/ρ)8M_{\mathrm{b}}=2^{-(d/\rho)^{8}}, where dd is the distance of a pixel from the blob’s center and ρ\rho is the blob’s radius. It is close to 1 inside the blob, 0.5 on its edge, and close to 0 outside. The blob is centered at 50% of the crop’s width and 60% of its height and covers 8 to 50% of the crop at random. Before blending, a random shift moves FF and MbM_{\mathrm{b}} together by up to 1.8% of the crop along each axis, details are in the supplementary Sec. S2.2.

3.1.5 Training pairs

We train on pairs of a fake and the real frame from which it was made, which mitigates shortcut learning [57]. FF++ has four fakes for every real frame, one per manipulation. To have as many twins, we set the number of twin pairs in an epoch to four times the number of FF++ real frames. In each twin pair, the fake is a twin of a real frame of Sec. 3.1.1: a reconstruction twin or a blob twin, with equal probability. FF++ fake pairs, re-sampled at random every epoch, add half as many pairs. So each kind of fake makes up about a third of an epoch, which has 34,434 pairs in batches of 64.

Each time a pair is used, both images receive the same new random augmentation, as in [35]: a horizontal flip, an affine transform, Gaussian blur, brightness and contrast jitter, sharpening, and JPEG compression; details are in the supplementary Sec. S2.3. The augmentation is applied to the face crop before resizing and normalization (Sec. 3.4).

3.2 Model architecture

TwinD uses the vision part of the Perception Encoder [3] (PEcoreL), Meta’s pre-trained foundation model, with a 224×224224\times 224 input. We add a binary linear classifier to the image embedding from PEcoreL that predicts the real or fake class. We adapt the backbone with LoRA [16], which keeps each pre-trained weight W0W_{0} frozen and trains a low-rank update A​BAB of rank rr:

W=W0+αr​A​B.W=W_{0}+\frac{\alpha}{r}AB. (3)

Training starts from A​B=0AB=0, so at the start of training, the model is equal to the pre-trained one. We use r=32r=32 and α=32\alpha=32, and apply LoRA to the query (WqW_{q}), key (WkW_{k}), value (WvW_{v}), and output (WoW_{o}) projection matrices of the self-attention in all 24 blocks. This trains 6.29 M parameters, 1.95% of the entire model.

3.3 Training

TwinD is trained using a standard cross-entropy loss for 10 epochs, at a constant learning rate of 3⋅10−43\cdot 10^{-4} after one epoch of linear warm-up from 10−510^{-5}. Training uses the Adam [22] optimizer and bf16 mixed precision. Instead of weight decay, we use L2-SP [27] in the formulation of [45], which after each Adam step pulls the parameters back toward their initial values (Sec. S2.5 in the supplementary).

A validation set tells little about which checkpoint is the best when the test datasets come from a different distribution [1]. So TwinD uses no validation set. The number of epochs was fixed in advance and never tuned. The learning rate is a common default for Adam and was not tuned either. With no way to tell which checkpoint is the best, TwinD does not select one. It uses stochastic weight averaging (SWA) [17], which averages all checkpoints of the last 5 epochs into a single model. SWA was shown [17] to improve generalization compared to individual checkpoints, needs no held-out data, and has a negligible computational cost. Every experiment in this paper uses the same schedule and the same number of samples per epoch.

We repeat training three times with different random seeds and average the final weights into a single model: a uniform model soup [49]. Averaging weights works only if any weighted average of two runs also has a low loss, a property called linear mode connectivity [9]. We expect it to hold because the three runs start from the same pre-trained model, and L2-SP keeps them close to its weights. The soup improves accuracy and out-of-distribution robustness [49].

Before averaging, we merge each LoRA update into its frozen weight, Eq. 3, because averaging AA and BB separately differs from averaging the product A​BAB. So TwinD has the same inference cost as PEcoreL with a linear classifier.

One training run takes about an hour on one A100 GPU.

Table 2: Summary of test datasets. The table reports the number of real and fake test videos. For image datasets (†) and image subsets (‡) each image counts as a video with a single frame. “Gen.” is the number of generators.

Dataset Venue Gen. Real Fake

Blending DFD [8] Google’19 5 363 3,068

FSh [24] CVPR’20 1 140 140

CDFv2 [28] CVPR’20 1 178 340

FFIW [59] CVPR’21 3 1,357 1,357

FAVC [21] NeurIPS’21 4 500 20,566

DFDM [18] ICIP’22 4 415 1,720

PGF [15] ICPR’24 10 708 11,685

IDF [51] MM’24 9 18,834 2,987

DSv1 [2] CVPR’26 5 1,381 1,395

HiDF [20] KDD’25 2 4,361 4,361

B+EFS CDFv3 [29] arXiv’25 22 178 5,239

DSv2 [2] CVPR’26 6 1,766 1,289

RF† [37] arXiv’25 9 7,411 4,536

HF‡ [44] ICLR’26 20 15,907 11,078

EFS THB [50] WACV’26 5 334 844

OF [26] CVPR’26 4 3,146 3,400

DFAV [30] ICASSP’26 3 8,223 21,597

MV [43] ICML’26 3 705 2,114

DF26 [40] arXiv’26 7 273 2,423

Total (19) – 66,180 100,139

Table 3: Results per generator. Video-level AUROC (%) on DF26, OF and MV for each of their generators. The row of TwinD is the AUROC. Every other row is the difference to TwinD in percentage points, green if higher and red if lower. The prior detectors are the four best AIGC detectors and the four best facial deepfake detectors of Tab. 1. The other seven are in Tab. S11 in the supplementary. The rows under TwinD make the reconstruction twins with another autoencoder instead of CogVideoX. HV is HunyuanVideo, T2V is text-to-video and I2V is image-to-video. Mean is the average over the 17 generators. The best value in each column is in bold.

DF26 [40] OF [26] MV [43]

closed-source open-source

Grok T2V

Kling T2V

Veo3 T2V

Wan 2.6 T2V

HV 1.5 I2V

HV 1.5 T2V

Wan 2.2 I2V

Wan 2.2 T2V

LTX 2.3 I2V

LTX 2.3 T2V

ACTalker

Ditto

EDTalk

FLOAT

FantasyTalk.

OmniAvatar

Phantom 14B

Mean

AIGC detectors

LTD-UFD [55] −-2.7 −-40.7 −-0.5 ++2.8 −-9.3 −-2.8 −-16.4 ++0.7 −-13.2 −-4.7 −-34.5 −-64.3 −-79.6 −-60.4 −-64.5 −-47.1 −-5.6 −-26.0

AlignedFor. [35] −-5.4 ++3.5 −-15.0 −-4.0 −-40.5 −-51.0 −-48.6 −-54.4 −-21.4 −-21.9 −-2.1 −-33.4 −-30.9 −-64.1 −-86.5 −-64.4 −-13.9 −-32.6

DDA [5] −-8.4 ++1.6 −-9.8 ++7.6 −-14.8 −-28.3 −-25.9 −-33.3 −-24.0 −-34.7 −-0.2 −-70.1 −-66.4 −-62.0 −-54.1 −-11.1 −-6.1 −-25.9

B-Free [10] −-53.0 −-23.2 −-41.1 ++6.5 −-20.1 −-18.3 −-22.2 −-33.4 −-21.6 −-41.6 −-0.1 −-48.5 −-62.8 −-46.0 −-27.7 −-9.4 −-23.4 −-28.6

Facial deepfake detectors

VLAForge [60] −-64.0 −-36.1 −-64.0 −-30.2 −-46.1 −-20.2 −-33.8 −-46.9 −-33.7 −-49.3 −-5.4 −-34.8 −-15.0 −-26.9 −-48.5 −-18.7 −-29.2 −-35.5

FS-VFM [47] −-43.7 −-35.2 −-40.3 −-16.5 −-48.0 −-19.2 −-32.5 −-51.5 −-27.0 −-47.4 −-4.7 −-36.8 −-16.8 −-38.4 −-44.4 −-17.5 −-42.7 −-33.1

DFD-HR [42] −-53.5 −-23.1 −-45.2 −-27.0 −-33.8 −-15.2 −-22.2 −-45.4 −-13.8 −-40.0 −-2.0 −-31.1 −-6.1 −-30.3 −-22.5 −-4.7 −-15.3 −-25.4

GenD-PE [57] −-30.9 −-24.2 −-18.6 −-3.1 −-30.3 −-17.0 −-29.2 −-35.0 −-16.2 −-25.1 −-2.8 −-37.0 −-7.2 −-31.4 −-27.3 −-6.9 −-11.4 −-20.8

TwinD 97.4 95.2 95.1 91.4 95.1 99.7 95.3 93.7 97.6 96.5 99.9 88.8 98.9 95.0 92.9 92.1 99.9 95.6

w/ KVAE 2D −-1.3 −-7.2 −-1.0 −-2.6 −-4.2 −-1.4 −-7.4 −-18.2 −-2.9 −-2.0 −-0.1 −-10.5 −-4.2 −-7.3 −-19.9 −-15.5 ++0.1 −-6.2

w/ LTX-Video ++0.7 −-6.5 ++3.8 ++3.2 ++1.0 −-0.2 −-8.0 −-10.7 −-0.4 ++1.5 −-0.2 −-10.5 −-4.0 −-12.4 −-10.4 −-10.7 ++0.1 −-3.7

w/ HV 1.0 ++1.1 −-6.3 ++1.2 ++1.1 ++0.5 −-0.4 −-1.4 −-1.3 −-1.0 ++1.3 −-1.3 −-18.9 −-7.6 −-15.4 −-4.9 −-4.7 ++0.1 −-3.4

w/ LTX-2 ++1.8 −-8.7 ++4.1 ++7.6 ++3.2 ++0.2 −-0.3 ++1.6 ++0.9 ++2.7 0.0 −-25.9 −-8.7 −-27.5 −-1.3 −-5.8 ++0.1 −-3.3

3.4 Inference

From each test video we take up to 32 evenly spaced frames. In each frame, we detect the face, align it with a similarity transform on its five keypoints, enlarge the face box 1.3×1.3\times, and crop it (Sec. 4.2). The crop is then resized (bicubic) to 224×224224\times 224 and normalized to [−1,1][-1,1] range (mean and standard deviation 0.5 per channel), the standard preprocessing of PEcoreL [3]. TwinD gives the probability that a frame is fake, and the video score is the mean over its frames. With batches of 128 frames, the torch-compiled model processes about 1,000 frames per second on one A100 GPU.

Table 4: Ablations. The first row is the video-level AUROC (%) of the single models averaged into TwinD, mean of 3 seeds. Each other row changes one thing in training and shows the change in percentage points. Gray: mean±\pmstd intervals overlap the first row.

Blending B+EFS EFS All

TwinD single models (%) 97.9 92.6 95.3 96.1

Training fakes

w/o FF++ fakes −-1.8 −-3.0 −-0.7 −-1.8

w/o blob twins −-5.8 −-0.4 −-0.9 −-3.4

w/o reconstruction twins ++0.4 −-10.3 −-18.1 −-6.7

blob twins only −-0.7 −-15.2 −-21.9 −-9.3

FF++ fakes only −-1.3 −-15.7 −-24.4 −-10.4

reconstruction twins only −-33.0 −-9.4 −-3.9 −-20.4

Design of twins

w/o partial reconstructions 0.0 −-3.5 −-2.7 −-1.5

SBI instead of blob twins −-0.9 −-0.1 −-0.4 −-0.6

Training sources

w/o CCv2 reals and twins −-1.8 −-2.4 −-1.2 −-1.8

Training

w/o L2-SP −-0.2 −-1.3 −-1.0 −-0.7

LoRA rank

r=2r=2 −-0.3 −-2.3 −-0.8 −-0.9

r=4r=4 0.0 −-1.4 −-0.1 −-0.4

r=8r=8 ++0.2 −-1.1 −-0.1 −-0.2

r=16r=16 ++0.1 −-0.4 −-0.1 −-0.1

r=64r=64 −-0.1 −-0.9 −-0.3 −-0.3

4 Experiments

4.1 Benchmark

We evaluate all methods on 19 datasets released between 2019 and 2026, using test splits where available (otherwise, the full dataset). We divide them into three groups by how their generators create fakes: Blending, EFS, and B+EFS. We use these groups throughout for analysis.

Blending

Every generator blends the generated content back into the original frame. These 10 datasets are: Google’s DFD dataset [8], FaceShifter (FSh) [24], Celeb-DF-v2 (CDFv2) [28], Face Forensics in the Wild (FFIW) [59], FakeAVCeleb (FAVC) [21], DeepFakes from Different Models (DFDM) [18], PolyGlotFake (PGF) [15], IDForge (IDF) [51], DeepSpeak v1.1 (DSv1) [2], and Human-Indistinguishable Deepfake Dataset (HiDF) [20].

EFS

Every generator synthesizes the entire video frame with a face, which is called entire face synthesis (EFS). No step of the deepfake production pipeline blends generated content back into the original frame. These 5 datasets are: TalkingHeadBench (THB) [50], OmniFake (OF) [26], DigiFakeAV (DFAV) [30], MintVid (MV) [43], and DF26 [40].

B+EFS

These 4 datasets contain both blending and EFS generators: Celeb-DF++ (CDFv3) [29], DeepSpeak v2.0 (DSv2) [2], RedFace (RF) [37], and HydraFake (HF) [44].

Cleaning

We clean the test datasets in two ways. First, in Blending datasets, where each fake can be matched with its source video, we remove failed fakes that are pixel-identical to their source. Second, we remove videos where the face in the original footage is smaller than 100 px on any side. Such detections are often not faces at all, and when they are, the fakes are too small and blurred to deceive anyone.

No overlap with training data

The training and test data do not overlap. FF++ [36] is the common training set in facial deepfake detection. The videos of CCv2 [33] are unique recordings, and no test dataset uses them. To confirm this, we ranked CCv2 faces by their similarity to the faces of every test dataset using the face recognition model from the InsightFace buffalo_l pack [7] and inspected the 100 most similar pairs per dataset. None showed the same person. Importantly, no test dataset contains fakes generated by CogVideoX [56], whose autoencoder makes the reconstruction twins (Sec. 3.1.3), and no open-source generator in the test datasets is built on it. Table 2 lists the number of generators and real and fake videos in each dataset.

Evaluation metric

The primary metric is the video-level area under the ROC curve (AUROC), where each score represents one video, as described in Sec. 3.4. The equal error rate (EER) and the mean average precision over the two classes (mAP) are reported in the supplementary Sec. S3.

4.2 Preprocessing

All methods, including TwinD, get the same face crops. We follow DeepfakeBench [53] but use a state-of-the-art face detector, SCRFD-10GF [11] from the InsightFace buffalo_l model pack [7]. From each test video we take up to 32 evenly spaced frames. In each frame, we detect the face and align it with a similarity transform on its five keypoints, which keeps the face close to its original size. We then enlarge the face box 1.3×1.3\times and crop it. Each method resizes and normalizes the crop with its own preprocessing, the one it was trained with (for TwinD, see Sec. 3.4).

4.3 Comparison with the state-of-the-art

Table 1 compares TwinD with six AIGC detectors and nine facial deepfake detectors. We run the released model checkpoints of every method on the same face crops (Sec. 4.2).

Overall

TwinD reaches a mean AUROC of 96.3%, 8.9 percentage points above GenD-PE [57], the best prior detector. TwinD is the best method on 17 of the 19 datasets. The two exceptions are CDFv3, where the temporal model DFD-FCG [14] is 2.7 points higher, and HF, where the AIGC detector B-Free [10] is 2.4 points higher. TwinD is at least 90.4% on every dataset, signifying no major failures.

Blending

Prior facial deepfake detectors are strong on Blending, where DFD-HR [42] reaches a mean of 96.2%. TwinD reaches 98.1% and is the best on each dataset.

EFS

Prior detectors fall behind on the five EFS datasets. The best prior facial deepfake detector, GenD-PE, reaches a mean of 78.1%, and the best AIGC detector, B-Free, 73.8%. TwinD reaches 95.6%. On the four B+EFS datasets, TwinD reaches 92.7% against 80.1% for GenD-PE.

AIGC detectors

AIGC detectors generalize poorly on facial datasets. Over all 19 datasets, B-Free, the best one, also reaches 73.8%, with 98.5% on HF but 48.6% on OF.

Training set of TwinD

TwinD trains on 38,211 frames. Of these, 7,652 are real: 5,739 from FF++, 8 per video, and 1,913 from CCv2. The rest are fake: 7,652 CogVideoX reconstructions, one per real frame, and 22,907 FF++ fake frames from four FF++ manipulations. Blob twins and the masks of reconstruction twins are generated during training.

Training sets of prior detectors

The gain of TwinD in Tab. 1 cannot be simply explained by the number of training samples. Prior facial deepfake detectors sample FF++ at least as densely. SBI [38], Effort [54] and DFD-HR also sample 8 frames per FF++ video. ForAda [6] and GenD-PE sample 32, and GenD-PE, the best prior detector, trains on about 115,000 frames. FSFM [46] samples 128 frames per real video and pre-trains on about 3.3 M face images. The AIGC detectors that report their training set, AlignedForensics [35], DDA [5] and B-Free, train on 51,517 to 179,257 real images and 118,000 to 309,102 fake images.

4.4 Generalization across generators

The EFS fakes of DF26 [40], OF [26] and MV [43] are labeled with the generator that made them. Table 3 gives the AUROC of TwinD for each generator and, for every other detector, the difference to TwinD. DF26 has seven generators. Four are closed-source, and the three open-source ones each have a text-to-video and an image-to-video mode. With the four generators of OF and the three of MV, the table has 17 generators. The table shows the four best AIGC and the four best facial deepfake detectors of Tab. 1. The other seven detectors are in Tab. S11 in the supplementary. The same comparison for the generators of THB, DFAV and HF is in the supplementary (Tabs. S12 and S13), where some prior detectors are above TwinD on single generators.

Unseen generators

TwinD saw none of these generators in training (Sec. 4.1). Its AUROC is 95.6% on average and at least 88.8% in every column. On the four closed-source generators, it is between 91.4% and 97.4%.

Prior detectors

GenD-PE [57], the best prior detector on average, is 20.8 points below TwinD. Prior detectors are also uneven across generators. B-Free [10] is 0.1 points below TwinD on ACTalker but 62.8 points below on EDTalk. AlignedForensics [35] is 3.5 points above TwinD on Kling but 86.5 points below on FantasyTalking. In the table, a prior detector is above TwinD in six cells only. All six are AIGC detectors on Kling, Wan 2.6 or Wan 2.2. No prior facial deepfake detector is above TwinD in any of the 17 columns of Tab. 3. None of the seven detectors of Tab. 1 that the table leaves out is above TwinD on even a single generator (Tab. S11 in the supplementary).

Other autoencoders

The last four rows of Tab. 3 make the reconstruction twins with another autoencoder instead of CogVideoX (Sec. 3.1.3): KVAE 2D [39], LTX-Video [12], HunyuanVideo 1.0 [23] or LTX-2 [13]. Each of them is trained exactly as TwinD and is also a soup of three seeds (Sec. 3.3). Only the autoencoder differs. None of the open-source generators in Tab. 3 decodes with CogVideoX or with one of the four other autoencoders. The autoencoders of the four closed-source generators are not public. On average, all four other autoencoders are below CogVideoX, by 3.3 to 6.2 points. This is still far above GenD-PE (−20.8-20.8). So the gain of TwinD does not depend on the choice of CogVideoX. No autoencoder is the best on every generator. LTX-2 is above CogVideoX in 8 of the 10 columns of DF26, by up to 7.6 points, but below it on OF, by up to 27.5 points. All four are below CogVideoX on Kling, Ditto and FLOAT.

Figure 2: Robustness to image degradations. Mean video-level AUROC (%) over the 19 test datasets as the strength of each degradation increases. “Clean” is the undegraded benchmark of Tab. 1.

4.5 Ablations

We conducted an ablation study, see Tab. 4. Every row is the mean AUROC of three models trained with different seeds, which allows us to distinguish the effect of a change from seed variance. Therefore, the first row is the mean AUROC of three models that make the soup of TwinD (Sec. 3.3), not the AUROC of the soup itself from Tab. 1. Each subsequent row changes one factor in training and reports the change compared to the first row. In every row, an epoch has 34,434 pairs. When a kind of fake is removed, the epoch is filled with pairs of the other kinds. The supplementary material gives further details (Sec. S4) and the AUROC on each dataset (Tabs. S4, S6 and S6).

FF++ fakes only

This row is the usual training protocol for the facial deepfake detectors in Tab. 1: training on FF++ reals and FF++ fakes, with no CCv2 reals and no twins. Blending drops slightly (−1.3-1.3), but EFS drops by 24.4 points. This indicates that FF++ fakes mostly teach the compositing artifacts, while EFS generators leave no such traces. On All, this detector reaches 85.7%, which is 1.7 points below GenD-PE [57], the best prior detector. GenD-PE samples four times more frames per FF++ video (32 against 8). So, on the standard training data, the model and the training recipe of TwinD do not surpass prior detectors. The gain of TwinD comes from the training data: mainly the twins and partially the CCv2 reals.

Twins alone surpass prior detectors

Without FF++ fakes, all fakes are twins made from the FF++ and CCv2 reals. No fake in the training set changes the identity or the expression of a face. This detector reaches 94.3% on All, 6.9 points above GenD-PE, the best prior detector. Compared to the full training set, it loses 1.8 points, mostly on Blending (−1.8-1.8) and B+EFS (−3.0-3.0), while the change on EFS (−0.7-0.7) is within the seed variance. So, state-of-the-art generalization needs no face swap or reenactment in the training set. Here, twins alone surpass all prior detectors. Making the twins from more diverse reals helps further. Trained on FF++ reals and their twins only, without FF++ fakes and CCv2, the detector reaches 91.3% (Tab. S10 in the supplementary). This is 3.0 points lower but still above prior work.

Each kind of twin covers one kind of fake

Without reconstruction twins, EFS drops by 18.1 points, while Blending is within the seed variance. Therefore, EFS accuracy comes from reconstruction twins. Without blob twins, Blending drops by 5.8 points, while EFS is within the seed variance.

No single kind of fake is enough

With reconstruction twins only, Blending drops by 33.0 points, because the decoder’s traces do not teach blending artifacts. Blob twins only, made without any generator, do at least as well as FF++ fakes only (−9.3-9.3 vs −10.4-10.4 on All).

Design of twins

Putting back the real pixels into the reconstruction twins matters for EFS. Without partial reconstructions, EFS drops by 2.7 points and B+EFS by 3.5, while Blending does not change. Replacing blob twins with the more complex self-blended images (SBI) [38] lowers Blending by 0.9 points (Sec. S5 in the supplementary).

CCv2 reals

FF++ reals are mostly news-style YouTube videos with similar framing and compression. The CCv2 reals add many identities, cameras, and lighting conditions. Removing them and their twins from the full training set lowers All by 1.8 points. With CCv2, the detector improves mostly on the real videos of the test datasets and not on their fakes (Sec. S7 in the supplementary). In a training set without FF++ fakes, the CCv2 reals help only together with their twins (Sec. S7.10 in the supplementary).

L2-SP

Removing L2-SP lowers All by 0.7 points, see supplementary Sec. S2.5.

LoRA rank

We started with r=8r=8, the rank used by DDA [5], and then trained TwinD with ranks from 2 to 64. With a fixed α\alpha, the factor α/r\alpha/r in Eq. 3 gets smaller as the rank grows. So we follow rank-stabilized LoRA [19] and set α=32​r\alpha=\sqrt{32r}, which gives α=32\alpha=32 at r=32r=32. The sweep then changes the dimensionality of the update, not the scale. The rank matters little. The mean AUROC on All grows from 95.2% at r=2r=2 to 96.1% at r=32r=32 and drops to 95.8% at r=64r=64. Ranks 8 and 16 are within the seed variance of rank 32. Every rank surpasses all prior detectors. Even r=2r=2 is 7.8 points above GenD-PE, the best of them (Tab. 1).

4.6 Robustness to image degradations

Videos shared online are often recompressed, resized, or of low quality. We test how TwinD and five facial deepfake detectors from Tab. 1, GenD-PE [57], FS-VFM [47], VLAForge [60], Effort [54], and ForAda [6], hold up under four degradations. We leave out DFD-HR because its released code is 15 times slower than TwinD, which makes it too expensive to run on every degradation. We apply each degradation to the face crops of Sec. 4.2, before the preprocessing pipeline of each method, one degradation and one strength at a time. JPEG compression with PIL uses quality 80, 60, 40, 20, and 10. Gaussian blur uses σ\sigma of 0.5, 1.0, 1.5, 2.0, and 3.0 px. The kernel size is 2​⌈3​σ⌉+12\lceil 3\sigma\rceil+1. Gaussian noise has zero mean and variance of 0.001, 0.002, 0.005, 0.01, and 0.05 on pixel values in [0,1][0,1]. Its standard deviation is the square root of the variance, rounded to three decimals. It is drawn independently for each pixel and color channel, and the result is clipped to [0,1][0,1]. Downscaling resizes the square face crop to 112×112112\times 112 and 64×6464\times 64 px with nearest-neighbor interpolation.

Figure 2 shows the results. Averaged over the 19 datasets, TwinD has the highest AUROC under every degradation and strength. Under Gaussian noise and downscaling, the margin over GenD-PE narrows, and at the strongest noise the two methods are on par. Like GenD-PE and Effort, TwinD trains with JPEG compression and Gaussian blur (Sec. S2.3 in the supplementary). It trains without noise or downscaling.

5 Limitations and future work

Noise and downscaling

Both lower the AUROC of TwinD more than for prior detectors (Fig. 2), most on EFS (Tab. S15 in the supplementary). Unlike JPEG and blur, neither is used in training, and both likely mask high-frequency decoder traces. Training for robustness to them is future work.

One autoencoder

Reconstruction twins come from a single autoencoder. No autoencoder that we tested is the best on every generator (Tab. 3). Making twins with several autoencoders at once is future work.

6 Conclusions

The cross-dataset generalization of TwinD comes from two complementary training signals, the twins: fakes made from a real frame and paired with it. Reconstruction twins target the traces of a decoder, and blob twins the traces of blending. Twins alone surpass all 15 prior detectors. The proposed TwinD fine-tunes the PEcoreL foundation model with techniques to prevent overfitting: LoRA, L2-SP, and weight averaging. TwinD outperforms the prior art by a large margin. Against 15 recent detectors on 19 datasets from 2019 to 2026, TwinD reaches a mean AUROC of 96.3%, 8.9 points above the best of them, and at least 90.

Sources

← Back to Daily AI Report