Unexpected Class Swap in AblationCAM with Hugging Face ResNet-50

[UnsupportedOperation]

Hi,

I am experiencing an issue when applying AblationCAM to the Hugging Face ResNet-50. I followed this guide to apply Grad-CAM methods to Hugging Face models, and everything works as expected.

However, when I use AblationCAM with this specific model, I notice an unusual behavior. For example, when setting the target category to "bull mastiff", I obtain the following result:

But if I set the target category to "tabby, tabby cat", I get this:

It looks like the two classes are swapped. Using other CAM methods on the same model or using AblationCAM on other models produces the expected results.

Additionally, when I apply AblationCAM to the torchvision ResNet-50 (as shown in the cam.py file), I get this correct result (for "bull mastiff"):

Question:

What could be causing this issue? Could it be related to the Hugging Face ResNet-50 implementation?

Details:

• The target layer used for the Hugging Face model is:

  model.resnet.encoder.stages[-1].layers[-1]

• Other CAM methods work as expected.
• AblationCAM on other Hugging Face models (like DeIT or Swin Transfomer) works well.
• The issue does not occur when using the torchvision ResNet-50.

Any insights would be greatly appreciated!

Thanks in advance.