`--deselect` doesn't work with absolute paths

[bstaletic]

• a detailed description of the bug or suggestion
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

I wanted to do something like pytest /path/to/project/tests/* --deselect /path/to/project/tests/test_needsToBeAvoided.py, but this didn't work. Instead I had to do pytest /path/to/project/tests/* --deselect tests/test_needsToBeAvoided.py.

bstaletic@Gallifrey pytest % pip list
Package               Version
--------------------- ----------
attrs                 19.3.0
beautifulsoup4        4.8.2
certifi               2019.11.28
chardet               3.0.4
codecov               2.0.15
coverage              5.0.3
entrypoints           0.3
flake8                3.7.9
flake8-comprehensions 3.2.2
flake8-ycm            0.1.0
idna                  2.8
mccabe                0.6.1
more-itertools        8.2.0
packaging             20.1
pip                   20.0.2
pluggy                0.13.1
psutil                5.6.7
ptvsd                 4.3.2
py                    1.8.1
pycodestyle           2.5.0
pyflakes              2.1.1
PyHamcrest            2.0.0
pyparsing             2.4.6
pytest                5.3.5
pytest-cov            2.8.1
requests              2.22.0
setuptools            41.2.0
six                   1.14.0
soupsieve             1.9.5
urllib3               1.25.8
waitress              1.4.3
wcwidth               0.1.8
WebOb                 1.8.6
WebTest               2.0.34
bstaletic@Gallifrey pytest % python -c 'print(__import__("pytest").__version__)'
5.3.5
bstaletic@Gallifrey pytest % cat /etc/os-release
NAME="Artix Linux"
PRETTY_NAME="Artix Linux"
ID=artix
BUILD_ID=rolling
ANSI_COLOR="0;36"
HOME_URL="https://www.artixlinux.org/"
DOCUMENTATION_URL="https://wiki.artixlinux.org/"
SUPPORT_URL="https://forum.artixlinux.org/"
BUG_REPORT_URL="https://gitea.artixlinux.org/"
LOGO=artixlinux

Steps to repro:

• mkdir ~/test
• cd ~/test
• echo 'def test_test():assert 1' > test_test.py
• pytest ~/test/*_test.py --deselect test_test.py - this works. The report says "1 test deselected".
• pytest ~/test/*_test.py --deselect ~/test/test_test.py - this one doesn't work. The test is being run.
• pytest . --deselect ~/test/test_test.py - this one doesn't work either. The test is being run.

Additional info:

I have encountered this with python 3.8.0 and 3.8.1. I have not tried any other version.

[LucasFunai]

I'd like to fix this, but this is my first time contributing to anything and I have no idea of where to start.

[bstaletic]

pytest/src/_pytest/main.py

Lines 326 to 341 in d79179a

	def pytest_collection_modifyitems(items, config):
	deselect_prefixes = tuple(config.getoption("deselect") or [])
	if not deselect_prefixes:
	return
	remaining = []
	deselected = []
	for colitem in items:
	if colitem.nodeid.startswith(deselect_prefixes):
	deselected.append(colitem)
	else:
	remaining.append(colitem)
	if deselected:
	config.hook.pytest_deselected(items=deselected)
	items[:] = remaining

That's the function that takes --deselect-produced tuple and filters the tests. colitem.nodeid is something like (based off the example in OP) test_test.py::test_test, which isn't the absolute path. Then pybind just says if nodeid.startswith(deselect_tuple), which clearly is false if --deselect contains an absolute path.

Note: nodeid doesn't depend on current working directory.

However, this logic is fine. What should be changed is how nodeid gets assigned.

[LucasFunai]

That helped me a lot. Thank you bstaletic. Now I will try to find a solution and hopefully get a pull request done.

[bstaletic]

I took another look. nodeid is intentionally relative. I think the root path is also stored somewhere, so maybe the correct thing to do is (in pseudo-code):

if nodeid.startswith(deselect_prefixes) or os.path.join(root, nodeid).startswith(deselect_prefixes):
    # deselect here

[blueyed]

Try/see session._node_location_to_relpath.
Ref: https://github.com/pytest-dev/pytest/pull/6701/files#diff-adadbb411b38936fe8a465a285a5dd0fR428

I.e. I think the given argument should be converted to a nodeid (i.e. a path relative to the root), and then compared with the nodeid (which should have that format already).

[bstaletic]

I may have misunderstood something, because the following seemed to have no effect to the absolute/relative paths.

def pytest_collection_modifyitems(items, config, session):
    deselect_prefixes = tuple(config.getoption("deselect") or [])
    if not deselect_prefixes:
        return

    remaining = []
    deselected = []
    for colitem in items:
        for dp in deselect_prefixes:
            relative_dp = session._node_location_to_relpath(dp)
            if colitem.nodeid.startswith(relative_dp):
                deselected.append(colitem)
        else:
            remaining.append(colitem)

    if deselected:
        config.hook.pytest_deselected(items=deselected)
        items[:] = remaining

[blueyed]

Use --ignore / --ignore-glob maybe then? (ref: #3201 (comment)) As per docs only nodeids are supported.

[bstaletic]

--ignore is what I've tried first, but if I use a glob to select a bunch of files to run the tests, pytest then seems to disregard --ignore that points to one of the previously selected files.

bstaletic@Gallifrey ycmd  (git)-[valgrind]-% pytest ycmd/tests/s* --ignore shutdown_test.py --co
===================================================================== test session starts =====================================================================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /home/bstaletic/work/ycmd, inifile: pytest.ini
collected 8 items
<Package /home/bstaletic/work/ycmd/ycmd/tests>
  <Module shutdown_test.py>
    <Class Shutdown_test>
        <Function FromHandlerWithoutSubserver_test>
        <Function FromHandlerWithSubservers_test>
        <Function FromWatchdogWithoutSubserver_test>
        <Function FromWatchdogWithSubservers_test>
  <Module signature_help_test.py>
    <Function SignatureHelp_IdentifierCompleter_test[]>
    <Function SignatureHelp_IdentifierCompleter_disabled_test[]>
  <Module subcommands_test.py>
    <Function Subcommands_Basic_test[]>
    <Function Subcommands_NoExplicitCompleterTargetSpecified_test[]>

==================================================================== no tests ran in 0.03s ====================================================================

In this case --ignore doesn't work regardless of whether the paths are relative or absolute.

[blueyed]

@bstaletic yeah, it's bad.. maybe something can be done via #3201 (comment).
Another workaround might be to use -k "not …" additionally/instead.
I've not checked your pytest_collection_modifyitems example - have you debugged / looked into what session._node_location_to_relpath does?

[bstaletic]

-k 'not shutdown_test.py works. I've also tried making a custom mark and using -m "not ...", which also works. Both of which are fine workarounds for me, so the issue we're discussing isn't a blocker.

relative_dp = session._node_location_to_relpath(dp)

After that line, relative_dp had the same value as dp - the absolute path to the test file.

[blueyed]

relative_dp = session._node_location_to_relpath(dp)

After that line, relative_dp had the same value as dp - the absolute path to the test file.

Make sure to use a py.path.local - it should result in a relative path to the rootdir.
I recommend to just look at what it does using pdb.

[LucasFunai]

Edit : I tried to format this comment but it did not work. Sorry for the crappy code.

Hello. I was trying to fix this by converting deselect_prefixes to an absolute path,
and instead of doing
if colitem.nodeid.startswith(deselect_prefixes)
do something like
if os.path.isabs(colitem.nodeid): if colitem.nodeid.startswith(deselect_prefixes): deselected.append(colitem) else: if os.path.abspath(colitem.nodeid).startswith(deselect_prefixes): deselected.append(colitem)
But I don't know the format of colitem.nodeid and I could not find it on the documentation.
Is nodeid just the path or is there something else I should know?