Start using a `coverage` plugin to handle conditional coverage omissions

[sirosen]

In #2253, it came up that we wanted to conditionally exclude a block from coverage reporting on Python 3.8.

In order to support this, we'd need some mechanism beyond the baseline suite of features in coverage. Either our own workarounds (e.g., multiple configs) or a coverage plugin. A plugin seems preferable, as it keeps all of our config in one place and builds upon open and prior art.

Existing Plugins

Going off of the plugins which have gotten themselves added to the coverage docs here, there are primarily two options.

coverage-conditional-plugin offers conditional configs, which encompasses this use case. The project has few updates and appears mostly inactive, but not necessarily dead.

coverage-simple-excludes appears to be a perfect fit for our needs. It's newer and more active, but appears to be a solo effort without community traction as of yet.

Our Own Plugin

Writing our own plugin for this purpose, especially an in-tree one (e.g., piptools._internal.extensions.coverage) would be very easy. It would not give back to the community necessarily, expect in the form of showing how this can be done. Using the existing plugins and filing issues if we have trouble would potentially feed forward, and I therefore treat it as a preferable path.

I consider this an excellent backup option, if using the existing plugins is problematic.

Action Items

• Experiment with one or both of the listed plugins
• Configure one, of the implementer's choosing, for use in pip-tools
• Examine any existing no-cover pragmas to see if they can be updated to be conditional
• Report back on your findings, here or in a new PR

---

I have tried to phrase this as a good exploratory issue for any motivated new contributor. However, we should discuss this idea -- to ensure maintainers are aligned -- before anyone implements.

[webknjaz]

Here's a few plugin references: python-trio/trio#3163. I don't necessarily want to choose the same plugins (supply chain suspicion) but there's a few to look into.

[webknjaz]

coverage-conditional-plugin

This one is not being tested past Python 3.11

coverage-simple-excludes

And this dropped Python 3.8 already (although, using an older version for that env is fine too).

---

Though, the currently integrated covdefaults supports conditional coverage pragmas already. So we could just start using them for now.

[webknjaz]

I started playing with the pragmas that covdefaults already supports and found out that using them is difficult because many conditionals are tied to the version of pip and not Python. So this is something we'd have to take into consideration when looking at other plugins.