Option to display markers in `--collect-only` output (in order to `grep` for skipped all skipped tests)

[nitrocode]

We're skipping many of our tests and I think this is due to some missing functionality.

In order to see all the skipped tests, I have to run all the tests in verbose mode, store the output, and grep -E 'xfail|xpass' in order to see all of the specific tests that are skipped.

If there isn't a better way to already do this. I was thinking what if in the --collect-only output we could also see markers like skip or even custom markers for each test. That way I could easily collect the tests and grep for skipped ones without having to run all of them again.

[The-Compiler]

I'm pretty sure you could implement some pytest hook to check this (perhaps pytest_collect_modifyitems) and print the skipped items.

However, note that a test can call pytest.skip() inside the test as well, and you can't know about this while collecting.

[Zac-HD]

You can select tests by markers with e.g. pytest -m skip.

[RonnyPfannschmidt]

Marker deselection doesn't match logic based deselection, thus reopening

[Zac-HD]

The question is about seeing which tests have mark.skip applied though? Both of the following work for me:

pytest --quiet --collect-only -m skip
pytest --quiet --collect-only -m "not xfail and not skip"

What more needs to work in order to close this issue? If it's a new or enhanced feature that should probably have a new issue to describe the current and desired functionality.

[RonnyPfannschmidt]

But this wont know if a skipif actually triggers

[nicoddemus]

If there isn't a better way to already do this. I was thinking what if in the --collect-only output we could also see markers like skip or even custom markers for each test.

So it seems this issue can relate to two separate behaviors: showing marks during --collect-only (any mark, not only skip or skipif) and actually only running tests which actually have been skipped.

@nitrocode could you chip in in on which behavior are you interested?

[nitrocode]

@nicoddemus Hi everyone. The --collect-only kind of seems like a --dry-run option so the implication is that when you run this, it will scan through all the tests, and show all the tests that will run. It would be nice to also default it to show tests that have been explicitly skipped.

If in addition to that, you can also pass in a specific mark, that would be a bonus.