Fixture executed even if all tests are skipped

[lazka]

I just now updated an old application from pytest 7 to 8 and the following started failing:

# Using pytest 8.4.2
from unittest import TestCase, skipIf
import pytest

@skipIf(True, "reason")
class Foo(TestCase):
    @pytest.fixture(autouse=True)
    def something(self):
        assert 0

    def test_bar(self):
        pass

• Expected: The assert not being hit since all tests are skipped
• Actual: The assert is hit despite no tests being run.

I found that using pytest.mark.skipif instead of the unittest skipIf works around the issue. The context is that this is a Linux only test and skipped anywhere else, and the fixture also only works on Linux

[The-Compiler]

I don't think this is a regression, I can reproduce with your example and pytest 7.4.4 (also 7.0.0 and 6.2.5):

============================= test session starts ==============================
platform linux -- Python 3.13.7, pytest-7.4.4, pluggy-1.6.0
rootdir: /home/florian/tmp/pytest-13885
collected 1 item                                                               

test_13885.py E                                                          [100%]

==================================== ERRORS ====================================
________________________ ERROR at setup of Foo.test_bar ________________________

    @pytest.fixture(autouse=True)
    def something():
>       assert 0
E       assert 0

test_13885.py:7: AssertionError
=========================== short test summary info ============================
ERROR test_13885.py::Foo::test_bar - assert 0
=============================== 1 error in 0.02s ===============================

[lazka]

Hm, weird:

================================ test session starts ================================
platform linux -- Python 3.13.7, pytest-7.4.4, pluggy-1.6.0
rootdir: /home/lazka/Desktop/git/quodlibet
configfile: setup.cfg
collected 1 item                                                                    

a.py s                                                                        [100%]

================================ 1 skipped in 0.01s =================================

let me try in a fresh venv.

edit: same in a fresh venv with just "pip install pytest==7.4.4"

[The-Compiler]

configfile: setup.cfg - have you also tried in a clean directory outside of your project dir so that no config is picked up?

[lazka]

yes, I did, to remove any doubts:

FROM python:3.13-slim

RUN pip install --no-cache-dir pytest==7.4.4

RUN echo 'from unittest import TestCase, skipIf\n\
import pytest\n\
@skipIf(True, "reason")\n\
class Foo(TestCase):\n\
    @pytest.fixture(autouse=True)\n\
    def something(self):\n\
        assert 0\n\
    def test_bar(self):\n\
        pass\n' > /test_file.py

RUN pytest /test_file.py

This also reveals that it actually changed between 8.0.2 and 8.1.0.

[The-Compiler]

Oh, it looks like you edited your example (to move the fixture inside of the class) while I was looking at the issue and already had the code saved... With the same code I indeed get the same behavior you describe, that makes more sense now.

Bisected to c8792bd:

• Add an internal "register fixture" API and use it replace object patching for fixture injection #11780

cc @bluetech

[The-Compiler]

That being said, with OPs original example:

from unittest import TestCase, skipIf
import pytest


@pytest.fixture(autouse=True)
def something():
    assert 0


# @pytest.mark.skip
@skipIf(True, "reason")
class Foo(TestCase):
    def test_bar(self):
        pass

this is not a regression, but I'm still surprised somewhat because the behavior of @unittest.skipIf and @pytest.mark.skip is different here.

[RonnyPfannschmidt]

I believe the implementation location matters

Afair our skip happens before setup
I believe unittest skipif is fixture triggered by the integration and ordered after the autouse fixture

[lazka]

Oh, it looks like you edited your example

oh, sorry, I tried to make it match my case better after simplifying.

[bluetech]

Need to dig into the unittest plugin code to see how skipIf is handled. Hopefully will do it soon.