Every method here is trained on a single generator (Open-Sora, 2,000 clips per class) and then tested on 19 unseen open-source and 11 unseen commercial generators. Published comparisons often put a small detector next to a much larger one, which makes it hard to tell a better method from a bigger backbone. So we retrained all nine baselines ourselves, on the same data, splits and frame extraction, with size-matched encoders of about 22M parameters.

SEAM leads on close-OOD and holds its own on far-OOD. Its close-OOD AUC is 0.923, against 0.885 for the next-best method. On commercial generators the methods are much closer, and differences of about 0.015 AUC are within the seed-to-seed noise we measured, so we read that column as a tie.

AUC against state-of-the-art detectors. Held-out Open-Sora (ID), 19 unseen open-source generators (OOD-C), 11 unseen commercial generators (OOD-F), in-the-wild clips from 2025-era commercial generators (MNW) and talking-head generation (MintVid). The SEAM row takes its better configuration per column.

Compression is SEAM's weakness. After H.264 re-encoding, SEAM's far-OOD AUC drops from -0.120 at CRF 23 to -0.271 at CRF 47. It holds up better than Effort at light compression but falls behind it from CRF 35, and ForgeLens degrades less throughout. The frame-to-frame changes that its temporal streams read are what compression corrupts.

Compression resilience. Change in OOD-F AUC after H.264 re-encoding at increasing CRF; closer to zero is better. The two strongest far-OOD baselines are shown.

Social media videos are wildly different from current deepfake detection benchmarks

Generally, dataset benchmarks for deepfake detection are too clean to represent content often found on social media. Social media videos tend to be cropped, captioned, filtered, recompressed, etc. SLOP collects AI-generated and real videos from Instagram, TikTok and X so we can test on that kind of footage.

SLOP has 6,000 posts in a 50:50 split of real and AI-generated content, balanced across Instagram, TikTok and X. Splits are account-disjoint (70% train, 10% validation, 20% test), so no creator contributes to more than one split and a detector cannot exploit creator-specific cues. It mixes platform-representative random panels, dedicated AI-persona and AI-influencer accounts, and matched human creator accounts for the real class.

No existing dataset combines in-the-wild collection, platform-native processing, full-frame AI generation and AI-persona accounts with a balanced, split-ready release.

To better understand our dataset as well as social media content in general, we created a taxonomy that sorts clips along three axes: style, format, and content. See examples of this below.

Trained on the SLOP train split and tested on its creator-disjoint test split, SEAM reaches 0.961 AP and 0.957 AUROC, ahead of Effort and ForgeLens.

In-domain performance on SLOP. Trained on the train split, evaluated on the creator-disjoint test split. AP: average precision. The two strongest far-OOD baselines are shown.

Re-fitting the densities and the small head recovers SLOP. SEAM trained on AIGVDBench scores only 0.569 on SLOP, and re-fitting the densities and head on SLOP's train split lifts that to 0.957 without touching the encoder. Across all five test sets SEAM has the best grid mean (0.769, against 0.742 for Effort and 0.717 for ForgeLens), and the best single transfer cell is 0.904, a MintVid-trained model evaluated on AIGVDBench. All three methods score low on VRT (0.49 to 0.63), whose real class is ASMR only.

Cross-training generalization. Each method is trained on one dataset (rows) and evaluated on five test sets. MNW and VRT are evaluation-only. SM is SLOP. The two strongest far-OOD baselines are shown.

Misclassifications made by SEAM

SEAM tends to miss more AI-generated content than infer real content to be AI. On SLOP, 72% of the errors are AI clips predicted as real, and the errors are spread roughly evenly across platforms (39% X, 30% Instagram, 31% TikTok). On AIGVDBench the errors concentrate in the unseen-generator conditions, with LTX and EasyAnimate the largest sources of missed fakes.

We then went through the SLOP clips that a SEAM trained on SLOP gets wrong, using the operating threshold selected for that model. Below are the errors whose videos we reviewed: 13 of 29 real clips flagged as AI and 20 of 73 AI clips passed as real, ordered from most to least confident. Each card shows the score and the split recorded for the clip.

Real flagged as AI. These include many clips from a single polished pet-influencer account, plus heavily edited footage: glitch and particle effects, template overlays.

AI passed as real. Many are photoreal vertical talking-head or street-interview clips, several featuring the same recurring AI persona. The gallery shows at most three clips per account; that persona account alone makes up 28 of the 73 misses.

These errors may be caused by the model keying on how polished a video looks rather than on whether a camera captured it.

Explore the embedding space yourself

Each point is one clip, embedded by the frozen OpenVision ViT-S/16 encoder and projected to two dimensions with PCA. Hover a point for details, click a legend entry to hide a group, and switch the coloring above each plot.

Unseen generators hide among real clips. Press play on the AIGVDBench plot. The training generator sits far from real footage, but once unseen generators arrive they land on top of it: in this view, none of the training generator's clips fall inside the cloud of real footage, against most of the unseen generators'. On SLOP, AI and real clips overlap heavily, so no single linear direction separates them.

Citation

Citation coming soon.

Acknowledgments