Make actions happy #2311

matthewhughes934 · 2024-12-08T10:53:02Z

Ignore security issue with mkdocs-material

This requires handling upstream (see linked issue), trying to bump this
dependency errored with:

  Because mkdocs-material (9.5.32) depends on mkdocs (>=1.6,<2.0)
   and portray (1.8.0) depends on mkdocs (>=1.3.0,<1.4.0), mkdocs-material (9.5.32) is incompatible with portray (1.8.0).
  And because no versions of portray match >1.8.0, mkdocs-material (9.5.32) is incompatible with portray (>=1.8.0).
  So, because isort depends on both portray (>=1.8.0) and mkdocs-material (9.5.32), version solving failed.

Bump some dependencies for security fixes

Bump jinja

  -> Vulnerability found in jinja2 version 3.1.3
     Vulnerability ID: 71591
     Affected spec: <3.1.4
     ADVISORY: Jinja is an extensible templating engine. The `xmlattr`
     filter in affected versions of Jinja accepts keys containing non-attribute...
     CVE-2024-34064
     For more information, please visit
     https://data.safetycli.com/v/71591/f17

Bump anyio

  -> Vulnerability found in anyio version 4.1.0
     Vulnerability ID: 71199
     Affected spec: <4.4.0
     ADVISORY: Anyio version 4.4.0 addresses a thread race condition in
     `_eventloop.get_asynclib()` that caused crashes when multiple event loops...
     PVE-2024-71199
     For more information, please visit
     https://data.safetycli.com/v/71199/f17

Bump bandit

  -> Vulnerability found in bandit version 1.7.6
     Vulnerability ID: 64484
     Affected spec: <1.7.7
     ADVISORY: Bandit 1.7.7 identifies the str.replace method as a
     potential risk for SQL injection because it can be misused in constructing...
     PVE-2024-64484
     For more information, please visit
     https://data.safetycli.com/v/64484/f17

Bump certifi

  -> Vulnerability found in certifi version 2023.11.17
     Vulnerability ID: 72083
     Affected spec: >=2021.05.30,<2024.07.04
     ADVISORY: Certifi affected versions recognized root certificates from
     GLOBALTRUST. Certifi patch removes these root certificates from the root...
     CVE-2024-39689
     For more information, please visit
     https://data.safetycli.com/v/72083/f17

Bump idna

  -> Vulnerability found in idna version 3.6
     Vulnerability ID: 67895
     Affected spec: <3.7
     ADVISORY: Affected versions of Idna are vulnerable to Denial Of
     Service via the idna.encode(), where a specially crafted argument could...
     CVE-2024-3651
     For more information, please visit
     https://data.safetycli.com/v/67895/f17

Bump requests

  -> Vulnerability found in requests version 2.31.0
     Vulnerability ID: 71064
     Affected spec: <2.32.2
     ADVISORY: Affected versions of Requests, when making requests through
     a Requests `Session`, if the first request is made with `verify=False` to...
     CVE-2024-35195
     For more information, please visit
     https://data.safetycli.com/v/71064/f17

Bump setuptools

  -> Vulnerability found in requests version 2.31.0
     Vulnerability ID: 71064
     Affected spec: <2.32.2
     ADVISORY: Affected versions of Requests, when making requests through
     a Requests `Session`, if the first request is made with `verify=False` to...
     CVE-2024-35195
     For more information, please visit
     https://data.safetycli.com/v/71064/f17

Bump tornado

  -> Vulnerability found in tornado version 6.4
     Vulnerability ID: 71957
     Affected spec: <=6.4.0
     ADVISORY: When Tornado receives a request with two Transfer-Encoding:
     chunked headers, it ignores them both. This enables request smuggling when...
     PVE-2024-71957
     For more information, please visit
     https://data.safetycli.com/v/71957/f17

  -> Vulnerability found in tornado version 6.4
     Vulnerability ID: 71956
     Affected spec: <6.4.1
     ADVISORY: Tornado’s curl_httpclient.CurlAsyncHTTPClient class is
     vulnerable to CRLF (carriage return/line feed) injection in the request...
     PVE-2024-71956
     For more information, please visit
     https://data.safetycli.com/v/71956/f17

Bump urllib3

  -> Vulnerability found in urllib3 version 2.1.0
     Vulnerability ID: 71608
     Affected spec: >=2.0.0a1,<=2.2.1
     ADVISORY: Urllib3's ProxyManager ensures that the Proxy-Authorization
     header is correctly directed only to configured proxies. However, when...
     CVE-2024-37891
     For more information, please visit
     https://data.safetycli.com/v/71608/f17

Bump zipp

  -> Vulnerability found in zipp version 3.17.0
     Vulnerability ID: 72132
     Affected spec: <3.19.1
     ADVISORY: A Denial of Service (DoS) vulnerability exists in the
     jaraco/zipp library. The vulnerability is triggered when processing a...
     CVE-2024-5569
     For more information, please visit
     https://data.safetycli.com/v/72132/f17

Bump virutalenv

  -> Vulnerability found in virtualenv version 20.25.0
     Vulnerability ID: 73456
     Affected spec: <20.26.6
     ADVISORY: Affected versions of the virtualenv package are vulnerable to command injection. This vulnerability could allow an attacker to execute arbitrary commands by exploiting improperly quoted string placeholders in activation scripts. The vulnerable functions include
     various shell activation scripts where placeholders like __VIRTUAL_ENV__ are used. The exploitability depends on the ability to control the input to these placeholders. Users are advised to update to the version where a quoting mechanism has been implemented to mitigate this...
     PVE-2024-73456
     For more information, please visit https://data.safetycli.com/v/73456/f17

Update black

  -> Vulnerability found in black version 23.11.0
     Vulnerability ID: 66742
     Affected spec: <24.3.0
     ADVISORY: Affected versions of Black are vulnerable to Regular
     Expression Denial of Service (ReDoS) via the...
     CVE-2024-21503
     For more information, please visit
     https://data.safetycli.com/v/66742/f17

Also re-run black to pick up any changes from the new version and
update some unit test that relied on how black formats.

CI: use pip over `pipx for poetry install

pipx is installed on all the runners by default, but using this means
pipx is run with the system Python, and not the one installed with
steup-python. This was noticed when e.g. the MacOS Python 3.9 job
would report:
```
  creating virtual environment...
  creating shared libraries...
  upgrading shared libraries...
  installing poetry...
  done! ✨ 🌟 ✨
    installed package poetry 1.3.1, installed using Python 3.13.0
    These apps are now globally available
      - poetry
  Poetry (version 1.3.1)
```
Python 3.13.0 is the system version pre-installed on these runners[1],
and a similar pattern was seen on the Ubuntu and Windows runners. An
alternative would be to add an install step for pipx but this feels
simpler

Link: https://github.com/actions/runner-images/blob/de16eefce8361c24c716958843d8c87cb1c25990/images/macos/macos-14-Readme.md [1]

Update pip for GitHub runner

This is to address an error seen on some Python 3.12 runners:

  <-- SNIP -->
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2164, in <module>
    register_finder(pkgutil.ImpImporter, find_on_path)
                      ^^^^^^^^^^^^^^^^^^^
  AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
                  ^^^^^^^^^^^^^^^^^^^

This looks to be the issue[1] fixed in Pip 23.2 so use that verison

Link: pip's vendored pkg_resources should stop using pkgutil.ImpImporter pypa/pip#11501 [1]

Update code to address deepsource errors

It complained about an else-return issue[1] and some commented-out code

Link: https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/no-else-return.html [1]

This requires handling upstream (see linked issue), trying to bump this dependency errored with: Because mkdocs-material (9.5.32) depends on mkdocs (>=1.6,<2.0) and portray (1.8.0) depends on mkdocs (>=1.3.0,<1.4.0), mkdocs-material (9.5.32) is incompatible with portray (1.8.0). And because no versions of portray match >1.8.0, mkdocs-material (9.5.32) is incompatible with portray (>=1.8.0). So, because isort depends on both portray (>=1.8.0) and mkdocs-material (9.5.32), version solving failed.

Bump `jinja` -> Vulnerability found in jinja2 version 3.1.3 Vulnerability ID: 71591 Affected spec: <3.1.4 ADVISORY: Jinja is an extensible templating engine. The `xmlattr` filter in affected versions of Jinja accepts keys containing non-attribute... CVE-2024-34064 For more information, please visit https://data.safetycli.com/v/71591/f17 Bump `anyio` -> Vulnerability found in anyio version 4.1.0 Vulnerability ID: 71199 Affected spec: <4.4.0 ADVISORY: Anyio version 4.4.0 addresses a thread race condition in `_eventloop.get_asynclib()` that caused crashes when multiple event loops... PVE-2024-71199 For more information, please visit https://data.safetycli.com/v/71199/f17 Bump `bandit` -> Vulnerability found in bandit version 1.7.6 Vulnerability ID: 64484 Affected spec: <1.7.7 ADVISORY: Bandit 1.7.7 identifies the str.replace method as a potential risk for SQL injection because it can be misused in constructing... PVE-2024-64484 For more information, please visit https://data.safetycli.com/v/64484/f17 Bump `certifi` -> Vulnerability found in certifi version 2023.11.17 Vulnerability ID: 72083 Affected spec: >=2021.05.30,<2024.07.04 ADVISORY: Certifi affected versions recognized root certificates from GLOBALTRUST. Certifi patch removes these root certificates from the root... CVE-2024-39689 For more information, please visit https://data.safetycli.com/v/72083/f17 Bump `idna` -> Vulnerability found in idna version 3.6 Vulnerability ID: 67895 Affected spec: <3.7 ADVISORY: Affected versions of Idna are vulnerable to Denial Of Service via the idna.encode(), where a specially crafted argument could... CVE-2024-3651 For more information, please visit https://data.safetycli.com/v/67895/f17 Bump `requests` -> Vulnerability found in requests version 2.31.0 Vulnerability ID: 71064 Affected spec: <2.32.2 ADVISORY: Affected versions of Requests, when making requests through a Requests `Session`, if the first request is made with `verify=False` to... CVE-2024-35195 For more information, please visit https://data.safetycli.com/v/71064/f17 Bump `setuptools` -> Vulnerability found in requests version 2.31.0 Vulnerability ID: 71064 Affected spec: <2.32.2 ADVISORY: Affected versions of Requests, when making requests through a Requests `Session`, if the first request is made with `verify=False` to... CVE-2024-35195 For more information, please visit https://data.safetycli.com/v/71064/f17 Bump `tornado` -> Vulnerability found in tornado version 6.4 Vulnerability ID: 71957 Affected spec: <=6.4.0 ADVISORY: When Tornado receives a request with two Transfer-Encoding: chunked headers, it ignores them both. This enables request smuggling when... PVE-2024-71957 For more information, please visit https://data.safetycli.com/v/71957/f17 -> Vulnerability found in tornado version 6.4 Vulnerability ID: 71956 Affected spec: <6.4.1 ADVISORY: Tornado’s curl_httpclient.CurlAsyncHTTPClient class is vulnerable to CRLF (carriage return/line feed) injection in the request... PVE-2024-71956 For more information, please visit https://data.safetycli.com/v/71956/f17 Bump `urllib3` -> Vulnerability found in urllib3 version 2.1.0 Vulnerability ID: 71608 Affected spec: >=2.0.0a1,<=2.2.1 ADVISORY: Urllib3's ProxyManager ensures that the Proxy-Authorization header is correctly directed only to configured proxies. However, when... CVE-2024-37891 For more information, please visit https://data.safetycli.com/v/71608/f17 Bump `zipp` -> Vulnerability found in zipp version 3.17.0 Vulnerability ID: 72132 Affected spec: <3.19.1 ADVISORY: A Denial of Service (DoS) vulnerability exists in the jaraco/zipp library. The vulnerability is triggered when processing a... CVE-2024-5569 For more information, please visit https://data.safetycli.com/v/72132/f17 Bump `virutalenv` -> Vulnerability found in virtualenv version 20.25.0 Vulnerability ID: 73456 Affected spec: <20.26.6 ADVISORY: Affected versions of the virtualenv package are vulnerable to command injection. This vulnerability could allow an attacker to execute arbitrary commands by exploiting improperly quoted string placeholders in activation scripts. The vulnerable functions include various shell activation scripts where placeholders like __VIRTUAL_ENV__ are used. The exploitability depends on the ability to control the input to these placeholders. Users are advised to update to the version where a quoting mechanism has been implemented to mitigate this... PVE-2024-73456 For more information, please visit https://data.safetycli.com/v/73456/f17

-> Vulnerability found in black version 23.11.0 Vulnerability ID: 66742 Affected spec: <24.3.0 ADVISORY: Affected versions of Black are vulnerable to Regular Expression Denial of Service (ReDoS) via the... CVE-2024-21503 For more information, please visit https://data.safetycli.com/v/66742/f17 Also re-run `black` to pick up any changes from the new version and update some unit test that relied on how black formats.

`pipx` is installed on all the runners by default, but using this means `pipx` is run with the system Python, and not the one installed with `steup-python`. This was noticed when e.g. the MacOS Python 3.9 job would report: creating virtual environment... creating shared libraries... upgrading shared libraries... installing poetry... done! ✨ 🌟 ✨ installed package poetry 1.3.1, installed using Python 3.13.0 These apps are now globally available - poetry Poetry (version 1.3.1) Python 3.13.0 is the system version pre-installed on these runners[1], and a similar pattern was seen on the Ubuntu and Windows runners. An alternative would be to add an install step for `pipx` but this feels simpler Link: https://github.com/actions/runner-images/blob/de16eefce8361c24c716958843d8c87cb1c25990/images/macos/macos-14-Readme.md [1]

This is to address an error seen on some Python 3.12 runners: <-- SNIP --> File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2164, in <module> register_finder(pkgutil.ImpImporter, find_on_path) ^^^^^^^^^^^^^^^^^^^ AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'? ^^^^^^^^^^^^^^^^^^^ This looks to be the issue[1] fixed in Pip 23.2 so use that verison Link: pypa/pip#11501 [1]

It complained about an else-return issue[1] and some commented-out code Link: https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/no-else-return.html [1]

matthewhughes934 · 2024-12-23T18:58:56Z

I've also created a separate fork https://github.com/matthewhughes934/isort-fork where I've done some more work on dependencies: bringing everything up to date and including dependabot to keep things that way

kurtmckee · 2025-01-07T00:06:03Z

scripts/lint.sh

 poetry run cruft check
 poetry run mypy -p isort -p tests
-poetry run black --target-version py38 --check .
+poetry run black --target-version py38 .


This fundamentally changes how the lint script works. I recommend adding --check back in to maintain its behavior.

This fundamentally changes how the lint script works. I recommend adding --check back in to maintain its behavior.

whoops, I removed this to easily re-format everything with the new black version, but it shouldn't have been committed, adding the flag back: 5760cc9

kurtmckee · 2025-01-07T00:11:47Z

pyproject.toml

-bandit = ">=1.6"
-black = ">=22.6.0"
+bandit = ">=1.7.7"
+black = "24.3.0"


Did you intend to lock this to an specific, old version of black? Could it be >=24.3.0,<25?

Did you intend to lock this to an specific, old version of black? Could it be >=24.3.0,<25?

another mistake, also fixed with 5760cc9, I didn't both fixing a highes version as there wasn't one before

DanielNoord

Thanks!

DanielNoord · 2025-01-09T08:19:14Z

isort/deprecated/finders.py

            for line in f:
                import_name, _, pypi_name = line.strip().partition(":")
                mappings[pypi_name] = import_name
            return mappings


Added in 52c86da and indeed seems unnecessary

matthewhughes934 added 6 commits December 6, 2024 18:29

Update code to address deepsource errors

6a83152

It complained about an else-return issue[1] and some commented-out code Link: https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/no-else-return.html [1]

matthewhughes934 mentioned this pull request Dec 8, 2024

Make CI happy again #2310

Closed

eirnym approved these changes Dec 10, 2024

View reviewed changes

kurtmckee suggested changes Jan 7, 2025

View reviewed changes

fixup! Update black

5760cc9

kurtmckee approved these changes Jan 7, 2025

View reviewed changes

DanielNoord approved these changes Jan 9, 2025

View reviewed changes

DanielNoord merged commit 1d1e9f3 into PyCQA:main Jan 9, 2025
18 checks passed

DanielNoord mentioned this pull request Jan 11, 2025

PyCQA/isort triage PyCQA/meta#64

Closed

matthewhughes934 mentioned this pull request Jan 14, 2025

Updates round 3 #2334

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Make actions happy #2311

Make actions happy #2311

Uh oh!

matthewhughes934 commented Dec 8, 2024

Uh oh!

matthewhughes934 commented Dec 23, 2024

Uh oh!

kurtmckee Jan 7, 2025

Uh oh!

matthewhughes934 Jan 7, 2025

Uh oh!

kurtmckee Jan 7, 2025

Uh oh!

matthewhughes934 Jan 7, 2025

Uh oh!

DanielNoord left a comment

Uh oh!

DanielNoord Jan 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Make actions happy #2311

Make actions happy #2311

Uh oh!

Conversation

matthewhughes934 commented Dec 8, 2024

Uh oh!

matthewhughes934 commented Dec 23, 2024

Uh oh!

kurtmckee Jan 7, 2025

Choose a reason for hiding this comment

Uh oh!

matthewhughes934 Jan 7, 2025

Choose a reason for hiding this comment

Uh oh!

kurtmckee Jan 7, 2025

Choose a reason for hiding this comment

Uh oh!

matthewhughes934 Jan 7, 2025

Choose a reason for hiding this comment

Uh oh!

DanielNoord left a comment

Choose a reason for hiding this comment

Uh oh!

DanielNoord Jan 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!