changes to mantid's assert_almost_equal test fixture

[rboston628]

Description of work

Summary of work

This changes the internal logic of assert_almost_equal, in particular fixing the condition for it to run comparisons. It is no longer necessary to pass one of the tolerances (it defaults to an absolute comparison at 1.e-10).

Many more tests were added, including extensive negative tests.

Purpose of work

In PR #37708, the assert_almost_equal test fixture was updated to accept two possible tolerances, one absolute and one relative. To make both optional, they were given the default value Property.EMPTY_DBL. This value is more or less equivalent to DBL_MAX in C++, representing the largest possible double value.

The fixture would only run comparisons if one or the other tolerances was set. However, it checked if they were set by

if rtol > Property.EMPTY_DBL:

and

if atol > Property.EMPTY_DBL:

These conditions are never true, so it never ran any comparisons, meaning the fixture did nothing. Doing nothing is equivalent to the assertion passing. Therefore, assert_almost_equal passes for any workspace whatsoever.

That PR also removed a negative test that would have caught the issue. This adds many more negative tests.

There is no associated issue.

ORNL EWM 7016

See discussion on PR #37708, here and here.

Further detail of work

There is further validation of atol and rtol to make sure they are positive.

The code calculate two flags

use_relative = rtol != Property.EMPTY_DBL
use_absolute = atol != Property.EMPTY_DBL

by seeing if the values equal the defaults. If not, then the associated comparison is performed.

Previously, the function would fail early if the first comparison failed. Instead, if both comparisons are called for then both will be performed, and the error message thrown later. This allows better debugging, so users can see if both or only one comparison is failing.

There are now many more tests. There is an additional positive test, several validation tests, and negative tests of basically every situation.

To test:

In main, launch workbench and run the following:

from mantid.simpleapi import *
import matplotlib.pyplot as plt
import numpy as np

from mantid.testing import assert_almost_equal

ws1 = CreateSingleValuedWorkspace(DataValue=1.0)
ws2 = CreateSingleValuedWorkspace(DataValue=2.0)
assert_almost_equal(ws1, ws2, atol=0.0, rtol=0.0)
print("assert passes")

This will print "assert passes", even though the workspaces are not equal to within the tolerance.

Pull and build this branch and launch workbench. Run the same script. It should fail on assert_almost_equal.

Explore some other combinations of atol, rtol, and values of the workspaces. It should work as expected.

---

Reviewer

Please comment on the points listed below (full description).
Your comments will be used as part of the gatekeeper process, so please comment clearly on what you have checked during your review. If changes are made to the PR during the review process then your final comment will be the most important for gatekeepers. In this comment you should make it clear why any earlier review is still valid, or confirm that all requested changes have been addressed.

Code Review

• Is the code of an acceptable quality?
• Does the code conform to the coding standards?
• Are the unit tests small and test the class in isolation?
• If there is GUI work does it follow the GUI standards?
• If there are changes in the release notes then do they describe the changes appropriately?
• Do the release notes conform to the release notes guide?

Functional Tests

• Do changes function as described? Add comments below that describe the tests performed?
• Do the changes handle unexpected situations, e.g. bad input?
• Has the relevant (user and developer) documentation been added/updated?

Does everything look good? Mark the review as Approve. A member of @mantidproject/gatekeepers will take care of it.

Gatekeeper

If you need to request changes to a PR then please add a comment and set the review status to "Request changes". This will stop the PR from showing up in the list for other gatekeepers.

[rboston628]

This is currently blocked by problems inside CompareWorkspaces, as reported in Bug #37877.
Those problems should be fixed by PR #37889