Replace unittest skipTest from transformers with pytest.skip

[albertvillanova]

Replace unittest skipTest from transformers (implicitly used by requirement markers) with pytest.skip markers:

• require_liger_kernel
• require_flash_attn
• require_torch_accelerator
• require_torch_multi_accelerator
• require_wandb

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[kashif]

Suggested change

	# Function ported from transformers.testing_utils
	def require_flash_attn():
	flash_attn_available = is_flash_attn_2_available()
	kernels_available = is_kernels_available()
	try:
	from kernels import get_kernel
	get_kernel("kernels-community/flash-attn")
	except Exception:
	kernels_available = False
	return pytest.mark.skipif(not (kernels_available or flash_attn_available), reason="test requires Flash Attention")
	# Function ported from transformers.testing_utils
	def require_flash_attn(func=None):
	flash_attn_available = is_flash_attn_2_available()
	kernels_available = is_kernels_available()
	try:
	from kernels import get_kernel
	get_kernel("kernels-community/flash-attn")
	except Exception:
	kernels_available = False
	marker = pytest.mark.skipif(not (kernels_available or flash_attn_available), reason="test requires Flash Attention")
	if func is not None:
	return marker(func)
	return marker

[kashif]

other wise we have to change our decorators to @require_flash_attn()

[albertvillanova]

Thanks a lot for your review, @kashif! 🤗

Finally, I aligned the implementation with the rest of the "require" pytest-markers, by calling an auxiliary function is_x_available.

[qgallouedec]

I don't think this one it 100% correct, because if kernels is available, but not flash-attn, and you try to use atto_implementation="flash_attention2", it will most likely fail.

[albertvillanova]

I just ported it from transformers. 😅
https://github.com/huggingface/transformers/blob/12a50f294d50e3d0e124511f2b6f43625f73ffce/src/transformers/testing_utils.py#L575-L591

def require_flash_attn(test_case):
    flash_attn_available = is_flash_attn_2_available()
    kernels_available = is_kernels_available()
    try:
        from kernels import get_kernel

        get_kernel("kernels-community/flash-attn")
    except Exception as _:
        kernels_available = False

    return unittest.skipUnless(kernels_available | flash_attn_available, "test requires Flash Attention")(test_case)

Do you think I made an error in porting it? @qgallouedec

[albertvillanova]

The logic on transformers side was recently changed by this PR:

• [kernels] If flash attention2 is not installed / fails to import (cc on our cluster) default to kernels transformers#40178

Modified require_flash_attn in testing_utils.py to allow tests to run if either FlashAttention2 or the community kernel is available, broadening test coverage and reliability.

[qgallouedec]

OK I see.
A lot of our tests rely on flash-attn lib; it's probably a good time to drop flash-attn and rely only on kernels:

• replace all attn_implementation="flash_attention_2" -> attn_implementation="kernels-community/flash-attn"
• replace require_flash_attn by require_kernels

@albertvillanova we can do this in a future PR

[albertvillanova]

I agree! I created an issue for that:

• Replace flash-attn with kernels in CI #4319