Skip to content

Conversation

jhaigh0
Copy link
Contributor

@jhaigh0 jhaigh0 commented May 6, 2025

Description of work

Upgrade mantid to Python 3.11. See release notes

Things that have been done

  • Update python version pinning
  • PyFrameObject's attributes are private now, so in a few places I've switched to using the provided Get functions
  • Byte code changes which mean necessary updates to funcinspect.py :(
    • Introduction of CACHE instructions meant that our process to find the latest function-like call needed updating.
    • Changes to the names of opcodes
    • A strange bug with the new (in our code) PRECALL instruction where sometimes frame.f_lasti would be pointing to PRECALL instead of CALL (but only happened in one test), should be fixed now.
  • Remove the use of the U option when opening files (it's been the default for a while)
  • Fix a mock object which was mistakenly being constructed with spec set to another mock object (no longer allowed).
  • Set PYDEVD_DISABLE_FILE_VALIDATION=1 to avoid ipython debugger warning on startup
  • Remove lib2to3 usage to avoid deprecation warnings

Fixes #37780

To test:

Packages for testing: https://builds.mantidproject.org/job/Core_Team_test_pipeline/249/artifact/


Reviewer

Please comment on the points listed below (full description).
Your comments will be used as part of the gatekeeper process, so please comment clearly on what you have checked during your review. If changes are made to the PR during the review process then your final comment will be the most important for gatekeepers. In this comment you should make it clear why any earlier review is still valid, or confirm that all requested changes have been addressed.

Code Review

  • Is the code of an acceptable quality?
  • Does the code conform to the coding standards?
  • Are the unit tests small and test the class in isolation?
  • If there is GUI work does it follow the GUI standards?
  • If there are changes in the release notes then do they describe the changes appropriately?
  • Do the release notes conform to the release notes guide?

Functional Tests

  • Do changes function as described? Add comments below that describe the tests performed?
  • Do the changes handle unexpected situations, e.g. bad input?
  • Has the relevant (user and developer) documentation been added/updated?

Does everything look good? Mark the review as Approve. A member of @mantidproject/gatekeepers will take care of it.

Gatekeeper

If you need to request changes to a PR then please add a comment and set the review status to "Request changes". This will stop the PR from showing up in the list for other gatekeepers.

@jhaigh0 jhaigh0 added this to the Release 6.13 milestone May 6, 2025
@jhaigh0 jhaigh0 added Maintenance Unassigned issues to be addressed in the next maintenance period. ISIS: Core Issue and pull requests at ISIS that relate to Core features and functions labels May 6, 2025
@jhaigh0 jhaigh0 changed the title Move to python311 Move to python 3.11 May 6, 2025
@jhaigh0 jhaigh0 force-pushed the move_to_python311 branch from 2af5154 to b06db3b Compare May 6, 2025 14:23
@jhaigh0 jhaigh0 marked this pull request as ready for review May 7, 2025 08:13
@jclarkeSTFC jclarkeSTFC self-assigned this May 7, 2025
@jhaigh0 jhaigh0 force-pushed the move_to_python311 branch from b06db3b to 951ce65 Compare May 7, 2025 10:26
@jclarkeSTFC
Copy link
Contributor

Error in the log when launching the standalone on Windows:

0.01s - Debugger warning: It seems that frozen modules are being used, which may
0.15s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

@jclarkeSTFC
Copy link
Contributor

Btw I noticed that the installers are labelled '6.11.*', does this branch need rebasing?

@jclarkeSTFC
Copy link
Contributor

jclarkeSTFC commented May 9, 2025

Will be worth checking if this PR has any effect on #38412

jhaigh0 added 10 commits May 13, 2025 11:27
For some reason OSIRISDiffactionReductionTest was failing when running all of it's unit test.
The first three would pass, then the rest fail (no matter which you commented out etc).
Running them individually was fine.
I still don't understand why this was happening, but it was caused by line 374 of OSIRISDiffactionReduction
where some workspaces are summed. This calls the op_wrapper in _workspaceops which calls lhs_info()

Through debugging I worked out that when the tests were failing, frame.f_lasti was pointing to PRECALL instead of CALL.
So I've changed process_frame to change last_i to point at CALL when it is at PRECALL.
@jhaigh0 jhaigh0 force-pushed the move_to_python311 branch from 951ce65 to 2427ffd Compare May 13, 2025 13:41
Copy link
Contributor

@jclarkeSTFC jclarkeSTFC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux and Windows standalone packages work well, I tried running some algorithms, running a script, using the iPython tab (getting workspace name from variable name). Also tried a few different types of plot.

Copy link
Member

@mducle mducle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and I think these changes are all that's needed for Python 3.12 too!

@jclarkeSTFC
Copy link
Contributor

macOS standalone and Windows conda installs also both look good.

@jclarkeSTFC jclarkeSTFC merged commit 1909bf3 into mantidproject:main May 15, 2025
10 checks passed
peterfpeterson pushed a commit to peterfpeterson/mantid that referenced this pull request May 16, 2025
### Description of work

Upgrade mantid to Python 3.11. [See release
notes](https://docs.python.org/3/whatsnew/3.11.html)

Things that have been done

- Update python version pinning
- `PyFrameObject`'s attributes are private now, so in a few places I've
switched to using the provided Get functions
- Byte code changes which mean necessary updates to `funcinspect.py` :(
- Introduction of `CACHE` instructions meant that our process to find
the latest function-like call needed updating.
  - Changes to the names of opcodes
- A strange bug with the new (in our code) `PRECALL` instruction where
sometimes `frame.f_lasti` would be pointing to `PRECALL` instead of
`CALL` (but only happened in one test), should be fixed now.
- Remove the use of the `U` option when opening files (it's been the
default for a while)
- Fix a mock object which was mistakenly being constructed with `spec`
set to another mock object (no longer allowed).
- Set `PYDEVD_DISABLE_FILE_VALIDATION=1` to avoid ipython debugger
warning on startup
- Remove `lib2to3` usage to avoid deprecation warnings

Fixes mantidproject#37780 

### To test:

Packages for testing:
https://builds.mantidproject.org/job/Core_Team_test_pipeline/249/artifact/

<!-- Ensure the base of this PR is correct (e.g. release-next or main)
Finally, don't forget to add the appropriate labels, milestones, etc.!
-->

---

### Reviewer

Please comment on the points listed below ([full
description](http://developer.mantidproject.org/ReviewingAPullRequest.html)).
**Your comments will be used as part of the gatekeeper process, so
please comment clearly on what you have checked during your review.** If
changes are made to the PR during the review process then your final
comment will be the most important for gatekeepers. In this comment you
should make it clear why any earlier review is still valid, or confirm
that all requested changes have been addressed.

#### Code Review

- Is the code of an acceptable quality?
- Does the code conform to the [coding
standards](http://developer.mantidproject.org/Standards/)?
- Are the unit tests small and test the class in isolation?
- If there is GUI work does it follow the [GUI
standards](http://developer.mantidproject.org/Standards/GUIStandards.html)?
- If there are changes in the release notes then do they describe the
changes appropriately?
- Do the release notes conform to the [release notes
guide](https://developer.mantidproject.org/Standards/ReleaseNotesGuide.html)?

#### Functional Tests

- Do changes function as described? Add comments below that describe the
tests performed?
- Do the changes handle unexpected situations, e.g. bad input?
- Has the relevant (user and developer) documentation been
added/updated?

Does everything look good? Mark the review as **Approve**. A member of
`@mantidproject/gatekeepers` will take care of it.

### Gatekeeper

If you need to request changes to a PR then please add a comment and set
the review status to "Request changes". This will stop the PR from
showing up in the list for other gatekeepers.
peterfpeterson added a commit that referenced this pull request May 16, 2025
This pulls a bunch of maintenance and dependency management into
`ornl-next`
* #38996
* #39182
* #39221
* #39207
* #39308
* #38990
* #39283
* #39304
* #39330
* #39235
* #39216

---------

Co-authored-by: MialLewis <95620982+MialLewis@users.noreply.github.com>
Co-authored-by: thomashampson <thomas.hampson@stfc.ac.uk>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: James Clarke <139879523+jclarkeSTFC@users.noreply.github.com>
Co-authored-by: Jonathan Haigh <35813666+jhaigh0@users.noreply.github.com>
Co-authored-by: Ross Whitfield <whitfieldre@ornl.gov>
Co-authored-by: James Clarke <james.clarke@stfc.ac.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ISIS: Core Issue and pull requests at ISIS that relate to Core features and functions Maintenance Unassigned issues to be addressed in the next maintenance period.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move to Python 3.11
3 participants