Skip to content

Commit 04142d0

Browse files
authored
add intern video2 (#2958)
1 parent 0d268c7 commit 04142d0

File tree

7 files changed

+1596
-3
lines changed

7 files changed

+1596
-3
lines changed

.ci/ignore_treon_docker.txt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,4 +86,5 @@ notebooks/omniparser/omniparser.ipynb
8686
notebooks/olmocr-pdf-vlm/olmocr-pdf-vlm.ipynb
8787
notebooks/minicpm-o-omnimodal-chatbot/minicpm-o-omnimodal-chatbot.ipynb
8888
notebooks/kokoro/kokoro.ipynb
89-
notebooks/qwen2.5-omni-chatbot/qwen2.5-omni-chatbot.ipynb
89+
notebooks/qwen2.5-omni-chatbot/qwen2.5-omni-chatbot.ipynb
90+
notebooks/intern-video2-classiciation/intern-video2-classification.ipynb

.ci/skipped_notebooks.yml

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -530,9 +530,15 @@
530530
- macos-13
531531
- ubuntu-22.04
532532
- windows-2019
533-
- notebook: "notebooks/deepseek-vl2/deepseek-vl2.ipynb"
533+
- notebook: notebooks/deepseek-vl2/deepseek-vl2.ipynb
534534
skips:
535535
- os:
536536
- macos-13
537537
- ubuntu-22.04
538-
- windows-2019
538+
- windows-2019
539+
- notebook: notebooks/intern-video2-classiciation/intern-video2-classification.ipynb
540+
skips:
541+
- os:
542+
- macos-13
543+
- ubuntu-22.04
544+
- windows-2019

.ci/spellcheck/.pyspelling.wordlist.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ BLACKBOX
8585
boolean
8686
CatVTON
8787
CentOS
88+
centric
8889
CFG
8990
charlist
9091
charlists
@@ -403,6 +404,7 @@ intel
403404
interactable
404405
InternLM
405406
internlm
407+
InternVideo
406408
Interpolative
407409
interpretable
408410
invertible
@@ -1074,6 +1076,7 @@ vec
10741076
VegaRT
10751077
verovio
10761078
videpth
1079+
ViFM
10771080
VIO
10781081
virtualenv
10791082
VisCPM
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Video Classification with InternVideo2 and OpenVINO
2+
3+
InternVideo2 is family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue.
4+
You can find more information about model in [model card](https://huggingface.co/OpenGVLab/InternVideo2-Stage2_6B), [paper](https://arxiv.org/pdf/2403.15377) and original [repository](https://github.com/OpenGVLab/InternVideo/tree/main/InternVideo2/multi_modality).
5+
6+
In this tutorial we consider how to convert, optimize and run InternVideo2 Stage2 model for video classification using OpenVINO.
7+
8+
## Notebook contents
9+
The tutorial consists from following steps:
10+
11+
- Install requirements
12+
- Convert and Optimize model
13+
- Run OpenVINO model inference
14+
- Launch Interactive demo
15+
16+
In this demonstration, you'll create text-to-video retrieval pipeline which is responsible to find the most suitable text caption for video content.
17+
18+
The image bellow illustrates example of model inference result.
19+
![example.png](https://github.com/user-attachments/assets/6720efe0-ab24-4d73-a22f-a8a0499558d8)
20+
21+
## Installation instructions
22+
This is a self-contained example that relies solely on its own code.</br>
23+
We recommend running the notebook in a virtual environment. You only need a Jupyter server to start.
24+
For details, please refer to [Installation Guide](../../README.md).
25+
26+
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5b5a4db0-7875-4bfb-bdbd-01698b5b1a77&file=notebooks/intern-video2-classiciation/README.md" />
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
import gradio as gr
2+
3+
4+
def make_demo(classify):
5+
demo = gr.Interface(
6+
classify,
7+
[
8+
gr.Video(label="Video"),
9+
gr.Textbox(label="Labels", info="Comma-separated list of class labels"),
10+
],
11+
gr.Label(label="Result"),
12+
examples=[["coco.mp4", "airplane, dog, car"]],
13+
allow_flagging="never",
14+
)
15+
16+
return demo

notebooks/intern-video2-classiciation/intern-video2-classification.ipynb

Lines changed: 739 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)