Logo KuaiMod: VLM-based SVP Moderator

VLM as Policy: Common-Law Content Moderation Framework for Short Video Platform

Xingyu Lu1,2, Tianke Zhang1, Chang Meng1, Xiaobei Wang1, Jinpeng Wang2, Yi-Fan Zhang1,3, Shisong Tang1, Changyi Liu1, Haojie Ding1, Kaiyu Jiang1, Kaiyu Tang1, Bin Wen1, Fan Yang1, Tingting Gao1, Di Zhang1, Kun Gai1
1Kuaishou Technology, 2THU SIGS, 3UCAS,

Introduction

   Exponentially growing short video platforms (SVPs) face significant challenges in moderating content detrimental to users' mental health, particularly for minors. The dissemination of such content on SVPs can lead to catastrophic societal consequences. Although substantial efforts have been dedicated to moderating such content, existing methods suffer from critical limitations:
    (1) Manual review is prone to human bias and incurs high operational costs.
    (2) Automated methods, though efficient, lack nuanced content understanding, resulting in lower accuracy.
    (3) Industrial moderation regulations struggle to adapt to rapidly evolving trends due to long update cycles.
   We present KuaiMod, the first SVP content moderation benchmark with authentic user/reviewer feedback to fill the absence of benchmark in this field. We evaluate various methods on the benchmark to verify the existence of the aforementioned limitations.
   We further propose our common-law content moderation framework to address these challenges. KuaiMod moderation framework consists of three components: training data construction, offline adaptation, and online deployment & refinement. Leveraging large vision language model (VLM) and Chain-of-Thought (CoT) reasoning, KuaiMod adequately models video toxicity based on sparse user feedback and fosters dynamic moderation policy with rapid update speed and high accuracy.
  Offline experiments and large-scale online A/B test demonstrates the superiority of KuaiMod: KuaiMod achieves the best moderation performance on our benchmark. The deployment of KuaiMod reduces the user reporting rate by 20% and its application in video recommendation increases both Daily Active User (DAU) and APP Usage Time (AUT) on several Kuaishou scenarios.
data-composition

Visualization about three moderation paradigms: Manual, traditional AI-assisted and our KuaiMod paradigm. With common-law principle and VLM, KuaiMod overcomes drawbacks of traditional paradigms.

KuaiMod Benchmark

Benchmark Statistics

data-composition

Data Distrubtion: KuaiMod data distribution on the initial taxonomy: The initial version of the taxonomy divides violative videos into 15 categories.

Data Examples

Examples of KuaiMod: We select one example for each of the 16 subcategories in KuaiMod, and all the samples in KuaiMod are real videos from the Kuaishou platform.

Offline Evaluation

Method Main Violative Category Recall Negative Content Positive Content Overall Accuracy
Law&Safety Content Commercial Intellectual Precision Recall Precision Recall
Binary Classification
Perspectivetoxicity 0.6036 0.4598 0.3455 0.2941 0.6591 0.7059 0.5538 0.5000 0.6190
Perspectivesevere 0.1716 0.1092 0.0909 0.1765 0.5922 0.9170 0.5429 0.1351 0.5870
GPT-4o mini 0.7235 0.6833 0.5091 0.5294 0.7679 0.7958 0.7057 0.6706 0.7430
GPT-4o 0.8176 0.8101 0.8182 0.4706 0.8346 0.7336 0.6870 0.8009 0.7620
RoBERTa 0.8530 0.7667 0.7455 0.5882 0.6448 0.7915 0.6817 0.8174 0.7400
CN-CLIP 0.7396 0.6034 0.7636 0.5882 0.7836 0.6777 0.7858 0.8633 0.7850
YOLO 0.7929 0.5920 0.7818 0.4706 0.8189 0.6967 0.8003 0.8875 0.8070
Intern-VL-7B-SFT 0.8471 0.8056 0.8000 0.6471 0.8152 0.7765 0.8600 0.8287 0.8230
Multi-Class Classification
GPT-4o mini 0.8706 0.7889 0.8727 0.4118 0.6161 0.8175 0.8250 0.6280 0.7130
GPT-4o 0.8647 0.8278 0.8364 0.5294 0.6381 0.8318 0.8422 0.6557 0.7300
Intern-VL-7B-SFT 0.6706 0.5944 0.7091 0.2353 0.7976 0.6256 0.7638 0.8841 0.7750
KuaiMod-7B 0.8765 0.8333 0.8545 0.7059 0.9701 0.8460 0.8972 0.9810 0.9240

Online Experiment Results

Comprehensive Ecosystem Governance
Scenario
Report Rate (%) ↓ DAU (%) ↑ Total AUT (%) ↑
NEBULA -24.34 +0.016 -0.002
Featured -18.98 -0.002 +0.012
Personalized Recommendation Enhancement
Scenario
DAU (%) ↑ Avg AUT (%) ↑ Total AUT (%) ↑
Main Site +0.028 +0.033 +0.063

Case Study

Case Study: We select 6 cases from online scenarios to demonstrate Kuaimod's review capabilities in handling complex situations.