Properly handle incomplete calibration for static quantization#2460
Open
wpietka wants to merge 6 commits into
Open
Properly handle incomplete calibration for static quantization#2460wpietka wants to merge 6 commits into
wpietka wants to merge 6 commits into
Conversation
wpietka
force-pushed
the
wpietkax/add-gemma-quantization-test
branch
2 times, most recently
from
3385be8 to
b07f4ff
Compare
wpietka
force-pushed
the
wpietkax/fix-incomplete-static-quantization
branch
2 times, most recently
from
36d5100 to
942cdf3
Compare
anko-intel
requested changes
May 5, 2026
Contributor
anko-intel
left a comment
anko-intel
left a comment
There was a problem hiding this comment.
I would like to avoid adding so many load_own_variables_preprocess
Lets discuss it offline, Probably we can postpone post_quantization_cleanup actions into load_own_variables
wpietka
force-pushed
the
wpietkax/fix-incomplete-static-quantization
branch
2 times, most recently
from
e1b1040 to
709bc31
Compare
Signed-off-by: Wojciech Piętka <wojciechx.pietka@intel.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
anko-intel
approved these changes
May 8, 2026
Contributor
anko-intel
left a comment
anko-intel
left a comment
There was a problem hiding this comment.
It looks very well, thanks
Signed-off-by: Wojciech Piętka <wojciechx.pietka@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Wojciech Piętka <wojciechx.pietka@intel.com>
Signed-off-by: Wojciech Piętka <wojciechx.pietka@intel.com>
Signed-off-by: Wojciech Piętka <wojciechx.pietka@intel.com>
anko-intel
approved these changes
May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of Change
Bug fix
Description
Currently when calibration function for static quantization is provided with incomplete input sample - not activating all model layers during calibration - certain layers are effectively broken, due to scales being set to 0.
This change detects if quantization failed for given layer and recovers original call() methods and weights. Hence the layer will behave exactly as if it was not quantized at all
Expected Behavior & Potential Risk
Output provided by quantized model will be correctly even if quantization fails for certain layers
How has this PR been tested?
pytest test/jax/test_gemma3_model.py::test_static_quantization_with_incomplete_calibration
Dependency Change?
No dependencies changed