Skip to content

Optimized compression for FP8 modes #3748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
AlexanderDokuchaev merged 10 commits into from
Dec 19, 2025
Merged

Optimized compression for FP8 modes #3748

AlexanderDokuchaev merged 10 commits into from
Dec 19, 2025

Conversation

@nikita-savelyevv
Copy link
Collaborator

@nikita-savelyevv nikita-savelyevv commented Nov 20, 2025 •
edited
Loading

Changes

Added optimized OpenVINO weights compression for fp8e4m3 data type.

optimum-cli export openvino time:

Model Memory Before (MiB) Memory After (MiB) Time Before (sec) Time After (sec)
Llama-3.2-1B 2328.03 2394.14 (+2.84%) 63.52 14.03 (-77.92%)
Phi-4-mini 5608.48 5197.70 (-7.33%) 187.34 28.05 (-85.03%)
Llama-3.1-8B 9918.52 8443.87 (-14.86%) 399.14 48.48 (-87.86%)

Reason for changes

UX improvement.

Tests

Extended existing tests.

https://github.com/openvinotoolkit/nncf/actions/runs/19767009608

@github-actions github-actions bot added the NNCF OpenVINO Pull requests that updates NNCF OpenVINO label Nov 20, 2025
@nikita-savelyevv nikita-savelyevv force-pushed the ns/f8e4m3-optimized-compression branch from bd421c7 to d5cc3f8 Compare November 28, 2025 11:50
@nikita-savelyevv nikita-savelyevv marked this pull request as ready for review November 28, 2025 16:04
@nikita-savelyevv nikita-savelyevv requested a review from a team as a code owner November 28, 2025 16:04
@ljaljushkin ljaljushkin requested a review from Copilot December 15, 2025 15:46
Copilot AI reviewed Dec 15, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI reviewed Dec 16, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Nikita Savelyev and others added 3 commits December 16, 2025 13:20
Update src/nncf/tensor/functions/numeric.py
759c9fc 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Extend edge_cases test
3962183
Merge branch 'ns/f8e4m3-optimized-compression' of github.com:nikita-s…
21d7ce0 
…avelyevv/nncf into ns/f8e4m3-optimized-compression
AlexanderDokuchaev
AlexanderDokuchaev approved these changes Dec 18, 2025
View reviewed changes
@AlexanderDokuchaev AlexanderDokuchaev merged commit 1451632 into openvinotoolkit:develop Dec 19, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@ljaljushkin ljaljushkin ljaljushkin approved these changes

@AlexanderDokuchaev AlexanderDokuchaev AlexanderDokuchaev approved these changes

@andreyanufr andreyanufr andreyanufr approved these changes

Assignees

No one assigned

Labels

NNCF OpenVINO Pull requests that updates NNCF OpenVINO

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nikita-savelyevv @ljaljushkin @AlexanderDokuchaev @andreyanufr