Skip to content

🐍 Drop Python 3.9 #4183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

🐍 Drop Python 3.9 #4183

wants to merge 21 commits into from
+977 −997

Conversation

qgallouedec
Copy link
Member

@qgallouedec qgallouedec commented Sep 30, 2025 •
edited
Loading

What does this PR do?

Python 3.14 release: 2025-10-07, see https://devguide.python.org/versions/

TRL usage:

Screenshot 2025-09-30 at 9 15 31 AM

Main changes:

  • Drop support for Python 3.9; Ruff target python version 3.9 -> 3.10
  • Union[X, Y] becomes X | Y (syntax Python 3.10+)
  • Optional[X] becomes X | None`
  • Pass strict: bool when using zip. In most cases, it's True, expect in apply_chat_template

@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

albertvillanova
albertvillanova approved these changes Oct 2, 2025
View reviewed changes
Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

albertvillanova
albertvillanova requested changes Oct 3, 2025
View reviewed changes
qgallouedec
qgallouedec commented Oct 5, 2025
View reviewed changes
trl/data_utils.py
# Handle DeepSeek-R1 <tool_call> token, see the above comment for details
prompt = "".join(x for x, _ in takewhile(lambda x: x[0] == x[1], zip(prompt, prompt_rejected)))
prompt = "".join(
x for x, _ in takewhile(lambda x: x[0] == x[1], zip(prompt, prompt_rejected, strict=False))
Copy link
Member Author

@qgallouedec qgallouedec Oct 5, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only exceptions for strict=False I think

@qgallouedec qgallouedec changed the title 🐍 Python is dead, long live Python 🐍 Drop Python 3.9 Oct 5, 2025
qgallouedec and others added 9 commits October 6, 2025 19:45
@qgallouedec
Squashed commit of the following:
b07df79 
commit ae6837f
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 18:40:18 2025 +0200

    Removed tokenizer/processor creation from example scripts (#4211)

commit 56a8f11
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 17:45:44 2025 +0200

    Replace setup with pyproject and fix packaging unintended modules (#4194)

commit 5291015
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 16:04:06 2025 +0200

    Remove `Optional` from `processing_class` in `PPOTrainer` (#4212)

commit 0588b1f
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 15:57:17 2025 +0200

    Updated vLLM integration guide (#4162)

    Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>

commit 45ee98b
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 11:14:54 2025 +0200

    Replace unittest with pytest (#4188)

commit 3800a6e
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 11:13:21 2025 +0200

    Hotfix: Exclude transformers 4.57.0 for Python 3.9 (#4209)

    Co-authored-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>

commit 7ad9ce8
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 11:04:20 2025 +0200

    Remove tokenizer creation from `sft` example script (#4197)

commit 0c2dc14
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 08:31:58 2025 +0200

    Remove custome_container for building the docs (#4198)

commit ced8b33
Author: burtenshaw <ben.burtenshaw@gmail.com>
Date:   Mon Oct 6 08:23:11 2025 +0200

    [DOCS/FIX] lora without regrets - fix lr (#4207)
@qgallouedec
Squashed commit of the following:
68959ad 
commit 65eb45c
Author: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Date:   Mon Oct 6 13:07:18 2025 -0600

    Apply style and revert change in `sft_video_llm` example (#4214)

commit ae6837f
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 18:40:18 2025 +0200

    Removed tokenizer/processor creation from example scripts (#4211)

commit 56a8f11
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 17:45:44 2025 +0200

    Replace setup with pyproject and fix packaging unintended modules (#4194)

commit 5291015
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 16:04:06 2025 +0200

    Remove `Optional` from `processing_class` in `PPOTrainer` (#4212)

commit 0588b1f
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 15:57:17 2025 +0200

    Updated vLLM integration guide (#4162)

    Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>

commit 45ee98b
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 11:14:54 2025 +0200

    Replace unittest with pytest (#4188)

commit 3800a6e
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 11:13:21 2025 +0200

    Hotfix: Exclude transformers 4.57.0 for Python 3.9 (#4209)

    Co-authored-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>

commit 7ad9ce8
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Mon Oct 6 11:04:20 2025 +0200

    Remove tokenizer creation from `sft` example script (#4197)

commit 0c2dc14
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Mon Oct 6 08:31:58 2025 +0200

    Remove custome_container for building the docs (#4198)

commit ced8b33
Author: burtenshaw <ben.burtenshaw@gmail.com>
Date:   Mon Oct 6 08:23:11 2025 +0200

    [DOCS/FIX] lora without regrets - fix lr (#4207)
@qgallouedec
Merge branch 'main' into py3.14
cbb41f7
@qgallouedec
style
a33e642
@qgallouedec
rm prerelease
88eee87
@qgallouedec
rm whitespace
c97bb24
@qgallouedec
revert video change
d66ea24
@qgallouedec
some missing
cfcec4a
@qgallouedec
style
a5ca7d4
albertvillanova
albertvillanova reviewed Oct 7, 2025
View reviewed changes
Copy link
Member

@albertvillanova albertvillanova left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that you removed the CI tests for Python 3.9, which makes sense given its upcoming end of life at the end of October.

However, I noticed that you're also replacing typing.Optional and typing.Union. Since these are not deprecated, these changes don't seem strictly necessary. Are we intentionally introducing incompatibility with Python 3.9, even if we're no longer officially supporting it?

albertvillanova
albertvillanova reviewed Oct 7, 2025
View reviewed changes
pyproject.toml
"Programming Language :: Python :: 3.13"
]
requires-python = ">=3.9"
requires-python = ">=3.10"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I just saw this now! 😅

@qgallouedec
Copy link
Member Author

You're right, it's not strictly needed. And this is a result of upgrading the ruff target version to py310

This would indeed break the support for python3.9. Which I think is ok, only 1% of users are still using it, and I suspect that these users actually use an old version of TRL

@qgallouedec
Copy link
Member Author

Wdyt?

albertvillanova
albertvillanova reviewed Oct 7, 2025
View reviewed changes
Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree.

Additionally, with requires-python = ">=3.10" in the pyproject.toml file, trl will not be able to be installed in Python 3.9 environments anymore.

albertvillanova
albertvillanova reviewed Oct 8, 2025
View reviewed changes
Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, I think it might be slightly early to drop support before the end-of-life date itself: 31 October.

I would suggest merging (and releasing) this PR early November, so that users and downstream projects still get a full compatibility window until the end of the month. This follows standard open-source practice: drop in the first release after the end-of-life date.

What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@albertvillanova albertvillanova albertvillanova requested changes

@edbeeching edbeeching edbeeching approved these changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@qgallouedec @HuggingFaceDocBuilderDev @edbeeching @albertvillanova