MacOS and Windows images not running tests

[druzm]

Hi,

I'm not sure whether I should report this issue, but I think starting a discussion about it and seeing if other people are experiencing the same issue might help.

I work on a Python package that needs to be compatible with Linux (Ubuntu), MacOS, and Windows. Therefore, I have set up the CI accordingly. For about a year, everything worked well. I would push to the repository, and the CI would run tests for different Python versions and operating systems. However, approximately a month ago, the MacOS and Windows images never seem to finish running the tests. It doesn't matter whether I set the timeout to one or six hours; sometimes not even a single test is executed. However, everything continues to work with Ubuntu.

Here there's a link to one of the most recent runs: https://github.com/gaia-dpci/GaiaXPy/actions/runs/5289393980

I contacted GitHub support and was redirected here.

Thanks!

[vpolikarpov-akvelon]

Hi @druzm. I forked your repo and experimented a bit. I selected two subsequent commits from polish-finder-release branch and ran workflow for each of them several times. The results are:

• workflow always works with no problems for https://github.com/gaia-dpci/GaiaXPy/tree/8d93408ade
• windows and macOS jobs hang for https://github.com/gaia-dpci/GaiaXPy/tree/037cba7e84

Image versions were the same for both commits (windows-2022 20230612.1, macos-12 20230612.1). So it looks like something is wrong with your code but not runners.

[vpolikarpov-akvelon]

I created machine from runner image and tried to run your workflow manually. Well, you are right: tests open windows that never get closed so execution hangs. Here is how it looks on Windows:

I bet the same is on macOS. The reason for your workflow to work in Ubuntu is the absence of graphical UI on Ubuntu runners.

And speaking of ModuleNotFoundError, it's a known issue with python 3.7.17 that you can track here: actions/setup-python#682

[druzm]

At the moment, I've opted for skipping those tests by simply running a custom script that ignores the corresponding folder, but I'm not sure this is ideal. Are there any other alternatives you know of, like suppressing the graphical UI for the affected OSs, for example?

[vpolikarpov-akvelon]

As far as I know there is no easy solution for that. In general you have two options:

1. Write some script that would work in background and close all those windows
2. Restrict opening windows using group policy (on Windows) or remove execution permission from the binary that opens windows (on macOS)

Both options are quite touch to implement and are unreliable from my point of view. I think it would be easier to change tests somehow.

[druzm]

I can try passing a flag to my tests script, so I can save the plots without displaying them. It's not like anyone will see them either, but I can choose to display them when running the tests locally and not to when running them remotely.
Thanks!