Skip to main content

VideoChat3: An open model that outperformed GPT-5 in video analysis

Ilustrační obrázek
Open-source software has just stepped into the premier league of multimodal AI. The newly introduced VideoChat3 model proves that thanks to intelligent data compression, a relatively small model with 4 billion parameters can achieve better results in video analysis than giant proprietary systems from OpenAI or Google. The key to success is not brute force, but the efficient way in which the model "sees" time and motion.

In the field of artificial intelligence, we often encounter the notion that bigger means better. In the case of video analysis, however, a fundamental paradigm shift is underway. A research team from Nanjing University, Shanghai AI Laboratory, and other prestigious institutions has introduced the VideoChat3 model, which outperformed the top-tier GPT-5 and Gemini 2.5 Flash models in key tests. What makes this so interesting? VideoChat3 is not a closed commercial product, but a fully open-source model whose weights and training data are publicly available.

Victory in "temporal grounding"

To understand the significance of this achievement, we need to explain the concept of temporal grounding. Imagine you have an hour-long recording from a security camera and you ask the question: "At which second did the man in the red jacket put the bag on the table?" A model with good temporal grounding can precisely identify the start and end of the given segment. This is one of the hardest tasks for video AI.

According to results published in the VideoChat3 research paper, the model achieves a score of 56.1 mIoU on the Charades-STA benchmark. For comparison, GPT-5 achieved only 40.5 and Gemini 2.5 Flash 48.6. A similar gap is seen in other tests such as ActivityNet Captions or QVHighlights. This difference between the open model and closed systems is considered a major milestone in the community.

Why is VideoChat3 more efficient?

Standard video models suffer from enormous computational demands. Each video frame is typically processed as a separate image, which creates millions of redundant (duplicate) data points. VideoChat3 solves this problem using the I3D-ViT (Inflated 3D Vision Transformer) architecture.

Instead of examining each frame in isolation, it groups them into blocks and analyzes their relationships in time and space simultaneously. As a result, VideoChat3 generates about 50% fewer visual tokens than, for example, the Qwen3-VL model under the same input conditions. For developers, this means one thing: faster and cheaper inference. While latency increases dramatically for competing models with long videos (1024 frames), VideoChat3 maintains high speed thanks to intelligent information compression.

Practical impact: What does this mean for Czech companies and developers?

This breakthrough has several direct implications that should interest not only scientists but also entrepreneurs in the Czech Republic and the EU:

  • Reduced operating costs: Thanks to lower token requirements, companies can implement advanced video analysis on less powerful hardware. This opens the door for local deployment in small and medium-sized businesses that don't have budgets for massive OpenAI cloud services.
  • Privacy and GDPR: Since VideoChat3 is open-source, it can be run on your own servers (on-premise). For Czech companies working with sensitive data (e.g. hospitals, security agencies), this is crucial — data doesn't have to leave the country or be processed in the cloud of American giants, which makes compliance with EU regulations easier.
  • Specialization: Developers can use the model as a foundation for specific tools — from automated analysis of sports recordings to intelligent monitoring in industrial halls.

Availability and price: The model is free to download via the Hugging Face platform. The only cost is the operating hardware (GPU). Given the 4B parameter range (4 billion parameters), it can run on relatively affordable hardware, which is very attractive for the Czech market with its technical infrastructure.

Performance comparison in key metrics

Benchmark VideoChat3 (4B) GPT-5 Gemini 2.5 Flash
Charades-STA (mIoU) 56.1 40.5 48.6
ActivityNet Captions 54.6 42.9 52.5
QVHighlights 67.0 52.1 64.3

Conclusion: A new era of local video AI

VideoChat3 is not just another model in the lineup. It is proof that efficient architecture can defeat brute-force parameter scaling. For the European ecosystem, which is striving for technological sovereignty, this innovation is extremely important. It offers a path to advanced multimodal AI that is transparent, verifiable, and above all — controllable.

Can I use VideoChat3 for video analysis in Czech?

The model itself focuses on understanding visual content and temporal relationships. Whether the output (video description) will be in Czech depends on the language model you connect to VideoChat3. If you pair it with a multilingual LLM (such as Llama 3 or GPT-4o), it will be able to generate responses in Czech without issues.

Is a supercomputer needed to run this model?

Thanks to the 4 billion parameter size and high token efficiency, a GPU cluster is not needed. The model should be manageable on a high-quality workstation with a single powerful graphics card (e.g. NVIDIA RTX series), making it an accessible tool for developers and smaller companies alike.

What are the main security risks when deploying it?

Since it is an open-source model, the biggest risk is improper implementation within the data security framework. However, the very ability to run locally (on-premise) significantly reduces risks associated with data leakage to the cloud, which is an unavoidable problem with proprietary models like GPT-5.

X

Don't miss out!

Subscribe for the latest news and updates.