Rule that suggests explicit strict= in @pytest.mark.xfail

[lengau]

Rule request

Description

Detect use of the @pytest.mark.xfail decorator and ensure it explicitly sets a strict=True or strict=False rather than allowing silent (and possibly accidental) use of the default False configuration.

Rationale

The pytest.mark.xfail decorator has a strict parameter that defaults to False, making xfail in CI roughly equivalent to a skip rather than failing if the test passes. This would prevent accidentally committing xfail tests that get hidden in CI.

[m-burst]

Hi @lengau,

Thanks for the issue!
Unfortunately I'm not sure about it.
I agree that strict xfail should always be preferred to non-strict, but it is probably better to set it globally in the config file, instead of bugging developers with lint errors on each individual test.

There already is issue #24 which suggests to verify that the config file follows the best practices (including xfail_strict = True)

[RonnyPfannschmidt]

the plan in pytest is to make xfail strict by default again