Skip to content

Conversation

thomhurst
Copy link
Owner

This PR contains the following updates:

Package Type Update Change
MSTest.TestAdapter nuget major 3.11.0 -> 4.0.0
MSTest.TestFramework nuget major 3.11.0 -> 4.0.0

Release Notes

microsoft/testfx (MSTest.TestAdapter)

v4.0.0

What is new?
Assert.That

MSTest v4 adds a new type of assertion, that allows you to write any expression, and it will inspect the result to give you more information on failure. Providing a very flexible way to assert complicated expressions. Here a simple example:

[TestClass]
public sealed class Test1
{
    [TestMethod]
    public void TestMethod1()
    {
        var animal = "Tiger";
        var zoo = new Zoo();
        Assert.That(() => zoo.GetAnimal() == animal);
    }
}

public class Zoo
{
    public string GetAnimal()
    {
        return "Giraffe";
    }
}
Assert.That(() => zoo.GetAnimal() == animal) failed.
Details:
    animal = "Tiger"     
    zoo.GetAnimal() = "Giraffe"
CallerArgumentExpression

CallerArgumentExpression is consumed by all assertions, to make them aware of the expressions used in the assertion. In the example below we now know what both the expected and actual values are. But also what value they come from, giving us opportunity to provide better error messages:

[TestClass]
public sealed class Test1
{
    [TestMethod]
    public void TestMethod1()
    {
        string animal = null;
        Assert.AreEqual("Giraffe", animal);
    }
}
Error Message: Assert.AreEqual failed. Expected:<Giraffe>. Actual:<>. 
'expected' expression: '"Giraffe"', 'actual' expression: 'animal'.
Breaking changes

We hidden many types that should have never been public in the first place. We believe that most of the changes won't have any real user impact. For the changes where we expect user impact migration guide from v3 is provided: https://learn.microsoft.com/dotnet/core/testing/unit-testing-mstest-migration-v3-v4

Full changelog here: microsoft/testfx@v3.11.0...v4.0.0


Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

@thomhurst thomhurst added dependencies Pull requests that update a dependency file MAJOR renovate-bot labels Oct 7, 2025
@thomhurst thomhurst enabled auto-merge (squash) October 7, 2025 19:21
Copy link
Contributor

claude bot commented Oct 7, 2025

Claude finished @thomhurst's task —— View job


MSTest v4 Update Review ✅

Todo List:

  • Read the changed file (Directory.Packages.props)
  • Search codebase for MSTest usage patterns
  • Analyze breaking changes impact
  • Check test compatibility
  • Review security and performance implications
  • Provide final recommendations

Review Summary

This PR updates MSTest packages from v3.11.0 to v4.0.0 in the centralized package management file. The update appears safe and compatible with the current codebase.

✅ Code Quality & Compatibility

  • Low Impact: MSTest usage limited to performance comparison benchmarks in tools/speed-comparison/UnifiedTests/
  • Safe APIs: All usage relies on stable MSTest APIs that remain compatible in v4
  • Well-Isolated: MSTest code is conditionally compiled behind #if MSTEST directives
  • Standard Usage: Uses conventional [TestClass], [TestMethod], and Assert.* methods

✅ Breaking Changes Analysis

MSTest v4's breaking changes primarily affect internal/hidden types that were mistakenly public. The benchmark code only uses:

  • Microsoft.VisualStudio.TestTools.UnitTesting namespace
  • Standard assertion methods (Assert.AreEqual, Assert.IsTrue, etc.)
  • Collection assertions (CollectionAssert.Contains)
  • Basic attributes ([TestClass], [TestMethod])

None of these core APIs are affected by v4 breaking changes.

✅ Performance Considerations

  • Positive: MSTest v4 includes performance improvements
  • New Features: Assert.That() and CallerArgumentExpression enhance debugging capabilities
  • No Regression Risk: Benchmark suite will benefit from framework optimizations

✅ Security Assessment

  • Clean Update: Dependency upgrade from reputable Microsoft package
  • No New Attack Surface: No new code dependencies or API surface area changes
  • Maintained Support: MSTest v4 is actively maintained by Microsoft

✅ Test Coverage

  • Comprehensive: Existing benchmark tests cover all MSTest usage patterns
  • Framework Agnostic: Test logic identical across all supported frameworks (TUnit, xUnit, NUnit, MSTest)
  • Validation Ready: Performance benchmarks will validate v4 compatibility automatically

Recommendation: APPROVE

This is a low-risk, high-value update that:

  1. Brings performance improvements and better debugging features
  2. Maintains full backward compatibility for this codebase
  3. Keeps the project current with Microsoft's testing ecosystem
  4. Has minimal blast radius (only affects benchmark comparisons)

The MSTest v4 migration guide confirms no action needed for codebases using standard MSTest APIs like this project.


@thomhurst thomhurst merged commit 89d61b6 into main Oct 7, 2025
12 checks passed
@thomhurst thomhurst deleted the renovate/major-mstest branch October 7, 2025 19:45
This was referenced Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file MAJOR renovate-bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants