mllama-3.2 Throws Gibberish Result

[ekurniaw]

Describe the bug
I use the following notebook.
https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/mllama-3.2/mllama-3.2.ipynb

I tested OpenVINO 2025.2.0.dev20250421 nightly on ARL-S iGPU, A770 dGPU, and B580 dGPU, all are throwing gibberish results. iGPU throws "!!!!!!!!!!!!!!!!!!..." and dGPUs throw "eyseeyseeyseeyse...".

OpenVINO 2024.6 and 2025.0 on ARL-S iGPU and A770 dGPU throws the expected result.

B580 needs to use OpenVINO 2025.2 nightly as it needs Multiply_10 operation to be implemented.

Expected behavior
Normal readable answer.

Installation instructions (Please mark the checkbox)

• I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks.

Environment information
$ python check_install.py
/home/eka/Workspace/openvino_2025.1_env/lib/python3.12/site-packages/openvino/runtime/init.py:10: DeprecationWarning: The openvino.runtime module is deprecated and will be removed in the 2026.0 release. Please replace openvino.runtime with openvino.
warnings.warn(
System information:
Python executable: /home/eka/Workspace/openvino_2025.1_env/bin/python
Pip version: 25.0.1
OpenVINO source: /home/eka/Workspace/openvino_2025.1_env/lib/python3.12/site-packages/openvino
OpenVINO IE version: 2025.2.0-18823-4ebe493140b
OpenVINO environment activated: NOT OK
Jupyter kernel installed for openvino_env: NOT OK
Python version: 3.12 NOT OK
OpenVINO pip package installed: OK
OpenVINO import succeeds: OK
OpenVINO not installed globally: NOT OK
No broken requirements: OK

It appears that you are not running Python in an openvino_env
environment. It is possible use the notebooks in a different
environment, but if you run into trouble, please follow the instructions
in the README to install and activate the openvino_env environment.

The Python version in openvino_env does not match the openvino_env Jupyter kernel.
This may not be an issue. If you experience issues, please follow the instructions
in the README to reinstall the kernel.
It appears that you installed OpenVINO globally (for example with
the OpenVINO installer, or a package manager).
This may cause conflicts with the OpenVINO environment installed by
pip install. If you encounter issues, please make sure to start the
notebooks from a terminal where you did not run setupvars.sh/setupvars.bat,
and where you did not add OpenVINO paths to your PATH or LD_LIBRARY_PATH.

LD_LIBRARY_PATH:
''

You may have added the command to source setuptools.sh to your
.bashrc, or added the OpenVINO paths to LD_LIBRARY_PATH there.
You can delete the lines from .bashrc and open a new terminal window
or temporarily reset your LD_LIBRARY_PATH by executing
export LD_LIBRARY_PATH= in your current terminal.

The README.md file is located in the openvino_notebooks directory
and at https://github.com/openvinotoolkit/openvino_notebooks

[Aznie-Intel]

Hi @ekurniaw, Thank you for reporting this issue. It looks like this might be a regression in the 2025.2 nightly build, possibly related to output postprocessing or tokenizer decoding. From your setup details, I noticed you're using Python 3.12, which could be a factor. Python 3.12 isn’t fully validated yet across all OpenVINO notebook components and may cause some compatibility issues.

Also, it seems the openvino_env isn't fully activated, and there's a mismatch between the virtual environment and the Jupyter kernel. There may also be a global OpenVINO installation interfering with the pip-installed one, which can sometimes cause conflicts.

As a workaround, I recommend switching to Python 3.10 or 3.11 instead of 3.12, those versions are better supported with the notebooks.

Make sure the correct virtual environment is activated before launching Jupyter:

Let us know if switching Python versions helps with the gibberish output, and feel free to share any additional findings.

[ekaakurniawan]

@Aznie-Intel , I tried Python 3.11 with the same result.

$ python check_install.py
System information:
Python executable: /home/eka/Workspace/openvino_env/bin/python
Pip version: 25.1
OpenVINO source: /home/eka/Workspace/openvino_env/lib/python3.11/site-packages/openvino
OpenVINO IE version: 2025.2.0-18866-252f94ff02c
OpenVINO environment activated: OK
Jupyter kernel installed for openvino_env: NOT OK
Python version: 3.11 OK
OpenVINO pip package installed: OK
OpenVINO import succeeds: OK
OpenVINO not installed globally: OK
No broken requirements: OK

The Python version in openvino_env does not match the openvino_env Jupyter kernel.
This may not be an issue. If you experience issues, please follow the instructions
in the README to reinstall the kernel.

The README.md file is located in the openvino_notebooks directory 
and at https://github.com/openvinotoolkit/openvino_notebooks

I cannot get Jupyter kernel installed for openvino_env. I have only python3.

$ jupyter kernelspec list
Available kernels:
  python3    /home/eka/Workspace/openvino_env/share/jupyter/kernels/python3

[Aznie-Intel]

Hi @ekaakurniawan Could you please try installing the Jupyter kernel inside your openvino_env environment by running pip install ipykernel followed by python -m ipykernel install --user --name openvino_env --display-name "Python (openvino_env)"? This should properly register the environment as a selectable kernel in Jupyter."

[ekurniaw]

@Aznie-Intel , is this a known issue? Should we update the OpenVINO version up to 2025.0 for now? I can pick up the task. Thanks.

[Aznie-Intel]

@ekaakurniawan Yes, this is a known issue with llama model running with GPU plugin. The root cause appears to be related to output postprocessing or tokenizer decoding in the GPU backend introduced in the 2025.2 nightly builds. If your model requires operations like Multiply_10, which are only supported in 2025.2+, you might need to switch to CPU inference until a fix is released temporarily.​

For now, since OpenVINO 2024.6 and 2025.0 seem to work fine, it would be ideal to use these versions, especially if the nightly build is not critical for your use case.