Windows automated builds have started failing, github have changed "something"?#! Need help to resolve.

[robertosfield]

Hi All,

For the last couple of days any commits to the VulkanSceneGraph repo has been resulting in build errors for the Windows automated build target. Nothing changed to the VulkanSceneGraph/.github/workflows/ci.jml that directs the automated buildsm it was working one day, then a couple of days ago it broke. The errors look like:

17s
Run if [ "$RUNNER_OS" == "Windows" ]; then
-- Building for: Visual Studio 15 2017
CMake Error at CMakeLists.txt:3 (project):
  Failed to run MSBuild command:
    MSBuild.exe
  to get the value of VCTargetsPath:
    MSBuild version 17.9.8+b34f75857 for .NET Framework
    Build started 5/17/2024 10:34:20 AM.
    
    Project "D:\a\VulkanSceneGraph\VulkanSceneGraph\CMakeFiles\3.10.3\VCTargetsPath.vcxproj" on node 1 (default targets).
    C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(456,5): error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found. To build using the v141 build tools, please install Visual Studio 2017 build tools.  Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [D:\a\VulkanSceneGraph\VulkanSceneGraph\CMakeFiles\3.10.3\VCTargetsPath.vcxproj]
    Done Building Project "D:\a\VulkanSceneGraph\VulkanSceneGraph\CMakeFiles\3.10.3\VCTargetsPath.vcxproj" (default targets) -- FAILED.
    
    Build FAILED.
    
    "D:\a\VulkanSceneGraph\VulkanSceneGraph\CMakeFiles\3.10.3\VCTargetsPath.vcxproj" (default target) (1) ->
    (PrepareForBuild target) -> 
      C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(456,5): error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found. To build using the v141 build tools, please install Visual Studio 2017 build tools.  Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [D:\a\VulkanSceneGraph\VulkanSceneGraph\CMakeFiles\3.10.3\VCTargetsPath.vcxproj]
    
        0 Warning(s)
        1 Error(s)
    
    Time Elapsed 00:00:08.56
    
  Exit code: 1
-- Configuring incomplete, errors occurred!

I didn't write our ci.yml files so I can't claim any expertise with automated builds let alone the Windows side. The ci.jml file was including microsoft/setup-msbuild@v1.1 so I looked up microsoft/setup-msbuild and found there is now a v2 version so I updated our ci.yml to this, and changed MSBUILD.exe to msbuild to be more consistent with the microsoft/setup-msbuild docs. This were done with these two commits: a827f2c & 1f8c285.

Unfortunately the errors remain the same. Has github broken something? Have folks come across other projects that their automated builds are failing?

I would appreciate help in resolving this issue as I've not yet come up with anything helpful on my own yet.

[Mikalai]

I've observed similar failures on azure pipelines recently. Maybe the issue is "os: windows-latest" which doesn't have required toolset. On azure explicit specifying windows-2019 fixed the build. But I don't know what options are available on github.

[Mikalai]

github yml available os

[robertosfield]

Thanks for the suggestion/link. I have tried windows-2019 but now get the error:

https://github.com/vsg-dev/VulkanSceneGraph/actions/runs/9128585117/job/25101231819

[Mikalai]

At least it was able to find VS 2017. Maybe now the issue is with "Add MSBuild to PATH" as there are mentions of VS 2019 but not 2017. Is it correct that VSG supposed to be build with VS 2017? Probably "Add MSBuild to PATH" should be extended with specific version like described here https://github.com/microsoft/setup-msbuild#specifying-specific-versions-of-visual-studio-optional

[Mikalai]

If I recall correctly VS 2017 is 15, probably 15.9 is the latest

[robertosfield]

I tried using cmake --build to build under Windows and this works so now things are compiling on all platforms with this commit: b1c6c10

I have now checked in reverting back to windows-latest to see if that works again now I'm just relying on cmake to do the build. I haven't attempted an install as well yet, but baby steps...

[AnyOldName3]

Actually, that's not quite the cause. Until the 13th, windows-latest would have old MSVC versions available in addition to the latest, and they changed that actions/runner-images#9701.

Before the 13th, you'd ask for the latest image and CMake 3.10, and CMake 3.10 would find and use MSVC 2017 on the latest image.

After the 13th, you'd ask for the same, but there'd be no MSVC 2017 for CMake to find, and it would fail.

[AnyOldName3]

I suspect a similar change might come to the windows-2019 image soon, so it's probably for the best to make this robust. That would mean asking for a newer version of CMake on Windows, and potentially also explicitly passing -G "Visual Studio 17 2022".

[sbrkopac]

I took a quick look at this and yeah, we have to bump the cmake version and should explicitly call the version of Visual Studio that we want. @robertosfield do you want to keep the lowest possible version on each of the runners? We can add some logic to hardcode a high version for the window runner.

[AnyOldName3]

I think it's pretty safe to use a newer version of CMake on Windows than other platforms. It's rare that Windows users don't keep their CMake at least vaguely up-to-date, so using 3.10 in CI isn't representative of the real world, and it's nominally backwards-compatible, so it shouldn't break anything. Using an older version is more important for Linux, where there's likely to be an older version provided by the package manager.

[sbrkopac]

Agreed. CI could probably use a little more TLC. I don't remember th exact reason I left it on windows-2019 but I think it had something to do with a weird migration or change to the images.

[robertosfield]

I like the simplicity of just bumping the CMake version for the automated builds, if it's straight forward to do this just for Windows then this would be fine. What version of CMake is required to make things work more smoothly for Windows? The oldest version that works would be the starting place if we are just going to use one CMake version.

[AnyOldName3]

3.21 for MSVC 2022, 3.14 for MSVC 2019, and 3.7.1 for MSVC 2017. There's already CMake new enough to work with the installed MSVC version preinstalled on the VM images, so if we simply don't run the Setup cmake step that's switching to the older version when the OS is Windows, everything should just work.

[sbrkopac]

Looks like some feedback on the image changes is less than stellar as well.

Im back from vacation on the 23rd and can commit to doing a more holistic review of the CI then.

[robertosfield]

Would it work fine if we didn't specify a cmake version? I presume window-latest would pull in CMake 3.21 or later.

[AnyOldName3]

All of GitHub's VM images have CMake 3.29 preinstalled. The CI for this repo has an explicit step where it installs an older version over the top. Disabling that step on Windows was what I proposed here #1192 (reply in thread)

[robertosfield]

To test out the possibility of just not specifying the CMake version I created a ci_yml_revisions branch and checked in the commit that removes the CMake version specification: c623fcc. This compiles fine on the automated build.

Next test to to try window-latest...

[robertosfield]

Yay, window-latest is back working again! 2f5bba2

[AnyOldName3]

The documentation for the action we're using says it defaults to the latest CMake version available on GitHub if given an empty string, and after a peek at their source code, it looks like they mean these https://github.com/Kitware/CMake/releases. I didn't realise Kitware had a GitHub mirror as CMake development happens on their own GitLab instance. Anyway, this means that you'll get an even newer version than the preinstalled one from the VM image, which may or may not be desirable.

[sbrkopac]

Yeah the checked-in commit will cause latest and greatest against all of the images in the matrix.

We can work out the details. I'll work on something when I'm back.

[robertosfield]

I have merged my latest changes to the ci.yml with VSG master: #1194.