Guide

Checkpoints

• Models checkpoints are stored in /mnt/12T/02_duong/shared/checkpoints

Setup Docker

• To build Docker image, run the following command:

sh cli/cli_docker/build_image.sh

• To run the API and web, run the following command:

sh cli/cli_docker/up_docker_compose.sh

Retrieve dynamic Gradio domain

• To retrieve the dynamic Gradio domain, run the following command:

sh cli/cli_network/get_gradio_live.sh

• The dynamic Gradio domain will be shown in the terminal. For example https://3efc2c84793c3f6a59.gradio.live

Demo

Image examples

• Images and lables are stored in src/assets/images/examples

Comparison

Model Parameters

Model	Model ID	Parameters (M)
LLaVA Med	microsoft/llava-med-v1.5-mistral-7b	7.56
Ours	-	7.67

Time Inference

Model	Model ID	Time inferece (s)
LLaVA Med	microsoft/llava-med-v1.5-mistral-7b	0.97
Ours	-	0.5

Specfications

Infrastructure

• GPU: NVIDIA RTX A6000/ 48 GB

Time Inference

Pre process	Stage 1	Stage 2	Time inferece (s)
x	-	-	0.15
-	x	-	0.5
-	-	x	?
x	x	-	0.65
-	x	x	?
x	x	x	?
Note:

• This report is calculated by the average of 10 times inference.