VMDT: Decoding the Trustworthiness of Video Foundation Models

We construct a comprehensive safety evaluation dataset comprising 780 prompts for T2V models and 990 prompts for V2T models, spanning 13 and 27 risk categories, respectively. Our risk taxonomy is grounded in established industry policies and benchmarks, while also addressing the unique characteristics of the video modality such as temporal and physical harm risks that cannot be detected in static frames. We design novel scenarios to test the performance of models under diverse conditions, including transformed instructions, synthetic content, and real-world content. Our evaluation and analysis reveal several critical findings: (1) Open-source T2V models universally lack safety refusal mechanisms, while even closed-source models struggle with video-specific risks like temporal and physical harm. (2) T2V models generate less harmful content in transformed scenarios, likely reflecting capability limitations rather than improved safety. (3) A substantial safety gap exists between open and closed-source V2T models, with all open-source variants demonstrating significantly higher harmful content generation rates. (4) Closed-source V2T models like Claude and GPT exhibit better safety overall but remain particularly vulnerable to fraud and deception risks, highlighting critical alignment gaps across all VFMs.

We construct a diverse dataset comprising 1,650 prompts for T2V models and 1,218 prompts for V2T models with the aim of measuring hallucination under different scenarios. Our hallucination dataset incorporates various scenarios including naturally difficult prompts, distraction, misleading, counterfactual reasoning, temporal activities, and OCR. We evaluate these scenarios across a set of tasks focusing on objects, attributes, actions, counting, and spatial understanding, as well as a scene understanding task for V2T. Our analysis reveals the following: 1) For T2V, all evaluated open-source models hallucinate significantly more than closed-source models, Luma and Pika, in nearly all scenarios. 2) Object recognition is the easiest task for T2V models, while OCR presents one of the most challenging scenarios. This aligns with the hallucination results observed in text-to-image (T2I) models, suggesting that T2V and T2I models share common challenges. 3) Within the same model class, an increase in V2T model size is associated with a decrease in hallucination. 4) For V2T models, we find the best-performing model on average is InternVL2.5-78B, an open-source model, which is the opposite of what is seen in T2V models.

We construct an extensive dataset comprising 1,086 prompts for T2V models and 5,008 prompts for V2T models. This fairness dataset aims to assess model fairness across various contexts, including social stereotypes (e.g., occupation) and decision-making scenarios (e.g., hiring). We also examine "overkill fairness," where models sacrifice factual/historical accuracy in pursuit of diversity (e.g., generating videos of Black female Founding Fathers). Our evaluation reveals several significant findings: 1) T2V models exhibit substantial overrepresentation towards males, White individuals, and younger people, while demonstrating some degree of overkill fairness. 2) This overrepresentation surpasses that of T2I models, yet shows lower levels of overkill fairness, suggesting a trade-off between these two dimensions. 3) V2T model fairness demonstrates a significant negative correlation with model size, with larger models exhibiting increased unfairness. 4) All V2T models show significant overkill fairness, generating historically inaccurate outputs to promote diversity.

We construct a balanced dataset of 1,000 text prompts for T2V models and 200 video samples for V2T models to evaluate the privacy memorization and extractive capabilities, respectively. Our T2V dataset comprises text prompts sampled from a pretraining corpus (i.e., caption-video pairs) used for most contemporary T2V models. Our V2T dataset comprises driving scene videos along with their location information (e.g., zip code) to evaluate inference capabilities to predict sensitive location data. Our evaluation results reveal the following: 1) T2V models generally exhibit weak data memorization. 2) However, we observe that the T2V VideoCrafter2 model sometimes includes watermarks from copyrighted training data in its generated videos, indicating some level of data memorization does occur. 3) Larger V2T models tend to demonstrate stronger location inference, suggesting that privacy risks increase as model size increases.