coverage with lcov problem: meson hard-codes "--ignore-errors unused" which invalidates .lcovrc directives

[markgalassi]

Describe the bug
[pre-scriptum: I love meson and use it for all my C/C++ projects and some others]

In complicated c++ code you often need to use several ignore_errors entries in a .lcovrc file. Unfortunately these are ignored by meson's invocation of lcov because meson hard-codes a "--ignore-errors unread" at line (approx.) 146 of coverage.py

"man lcovrc" states the following

Note that the lcovrc file message  list
is   not   applied   (those   messages   NOT   ignored)  if  the
'--ignore-errors' command line option is specified.: 

To Reproduce
I wish I could make a MWE for this, but the only way I know to reach the level of complexity which brings out the need for those --ignore-errors entries is to use my massive c++ codebase. The problem might be obvious enough that looking at the '--missing' option in coverage.py will make it clear, but here is a neat trick to give you an inverse MWE :-)

1. Take any lcov coverage setup.
2. In project source dir put this .lcovrc file (just one line is enough):

ignore_errors = bogus

3. Run the usual lcov sequence:

meson setup _mbuild_lcov -Db_coverage=true --wipe
meson test -C _mbuild_lcov
ninja -C _mbuild_lcov coverage-html

4. Since you have a bogus value for ignore_errors, you should get an error from lcov. Instead meson will just keep going because it specified --ignore-errors on the command line, which suppresses that entry. If you fix the problem by removing that line

	                           '--ignore-errors', 'unused',

in the context of:

            subprocess.check_call([lcov_exe,
                                   '--remove', covinfo,
                                   *lcov_subpoject_exclude,
                                   *lcov_exe_rc_branch_coverage,
	                           '--ignore-errors', 'unused',
                                   '--output-file', covinfo] + lcov_config)

then you will get the expected behavior of meson exiting with an error message from lcov saying that "bogus" is not a valid value.

Expected behavior
You should see the .lcovrc entries for ignore_missing honored by meson, instead of ignored.

A brief discussion of how to fix this: clearly removing the line I showed ('--ignore-errors', 'unused') will do it, but I don't know if that was really needed for other reasons. The other option would be to lobby the lcov people to have a --add-to-ignore-errors option which does not suppress what was in .lcovrc, but that seems like a long shot.

system parameters

• Is this a cross build or just a plain native build (for the same computer)?
  • just a native build
• what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
  • linux, Rocky Linux 8.0 (el8)
• what Python version are you using e.g. 3.8.0
  • python3.12
• what meson --version
  • 1.6.1 (installed with pipx)
• what ninja --version if it's a Ninja build
  • 1.8.1

[eli-schwartz]

A brief discussion of how to fix this: clearly removing the line I showed ('--ignore-errors', 'unused') will do it, but I don't know if that was really needed for other reasons. The other option would be to lobby the lcov people to have a --add-to-ignore-errors option which does not suppress what was in .lcovrc, but that seems like a long shot.

Getting proper lcov tool support upstream for this would probably be ideal.

The thing that springs to mind as a second-place solution is for mesonbuild/scripts/coverage.py to not only check if .lcovrc exists, but also to open it and check if it specifies any existing ignore_errors values -- if it does, we can assume the project has configured their ignores appropriately, and if not, then there is nothing for --ignore-errors on the command line to override.

[eli-schwartz]

            if os.path.exists(subproject_root):
                lcov_subpoject_exclude.append(os.path.join(subproject_root, '*'))

We remove this from coverage info automatically. My understanding is we then pass the requisite option to lcov to prevent it from fatally erroring in the event that there is no coverage info matching the subproject root, which can be excluded. So simply removing the ignore-errors option is probably going to break people -- we do need the more elaborate solution.

Possibly we could assume in the presence of .lcovrc that the project has configured the excludes properly?

[markgalassi]

[...] check if .lcovrc exists, but also to open it and check if it specifies any existing ignore_errors values -- if it does, we can assume the project has configured their ignores appropriately, and if not, then there is nothing for --ignore-errors on the command line to override.

That works, but you might be close to an even better and quite robust approach: if ignore_errors is in .lcovrc you can add all those fields to --ignore-missing (with a line.split(',') kind of thing) and also keep the "unused" field that meson needs.