Description
I'm observing a very odd behavior of tests that are imported from other files getting collected only for one test file when they have parametrized arguments that are generated through itertools.
Example with 3 files:
common.py:
import pytest
import itertools
class CommonTestSuite:
@pytest.mark.parametrize(
"m,n",
itertools.product(
[100, 1_000],
[100, 1_000],
)
)
def test_no_negative_numbers(self, generator, m, n):
assert generator(m,n).min() >= 0.test_poisson.py:
from common import CommonTestSuite
import pytest
import numpy as np
class PoissonGenerator:
def __init__(self):
self.rng = np.random.default_rng(seed=123)
def generate(self, m, n):
return self.rng.poisson(1, size=(m,n))
class TestPoisson(CommonTestSuite):
obj = PoissonGenerator()
@pytest.fixture
def generator(self):
return self.obj.generatetest_gamma.py:
from common import CommonTestSuite
import pytest
import numpy as np
class GammaGenerator:
def __init__(self):
self.rng = np.random.default_rng(seed=123)
def generate(self, m, n):
return self.rng.standard_gamma(1, size=(m,n))
class TestGamma(CommonTestSuite):
obj = GammaGenerator()
@pytest.fixture
def generator(self):
return self.obj.generateIf I run pytest . --co, this is what I get:
============================================ test session starts ============================================
platform linux -- Python 3.11.0, pytest-7.4.0, pluggy-1.0.0
rootdir: /home/david/del/pytest_issue
plugins: anyio-3.5.0, mypy-0.10.3
collected 5 items
<Module test_gamma.py>
<Class TestGamma>
<Function test_no_negative_numbers[100-100]>
<Function test_no_negative_numbers[100-1000]>
<Function test_no_negative_numbers[1000-100]>
<Function test_no_negative_numbers[1000-1000]>
<Module test_poisson.py>
<Class TestPoisson>
<Function test_no_negative_numbers[m0-n0]>
That is: the tests from the common class were only collected for the first file that is executed when running pytest .. Same behavior happens if I add more files importing from common.py.
If I modify the file common.py as follows (no itertools-generated parametrization):
import pytest
class CommonTestSuite:
@pytest.mark.parametrize(
"m,n",
[
(100, 10),
(10, 100)
]
)
def test_no_negative_numbers(self, generator, m, n):
assert generator(m,n).min() >= 0.Then the tests get collected as expected:
platform linux -- Python 3.11.0, pytest-7.4.0, pluggy-1.0.0
rootdir: /home/david/del/pytest_issue
plugins: anyio-3.5.0, mypy-0.10.3
collected 4 items
<Module test_gamma.py>
<Class TestGamma>
<Function test_no_negative_numbers[100-10]>
<Function test_no_negative_numbers[10-100]>
<Module test_poisson.py>
<Class TestPoisson>
<Function test_no_negative_numbers[100-10]>
<Function test_no_negative_numbers[10-100]>
If instead of importing the common tests from the same file in both test_gamma.py and test_poisson.py I copy-paste the code with itertools or "import" the file by using file.read() + exec(), then the tests are collected correctly as expected.