CodeCoverage

I have a patch that I will be submitting to add to the build system the capability of reporting on test code coverage metrics for the test suite. Actually it can show coverage for any application run against PostgreSQL. Download File:Coverage.tar.gz to see an example report. Gunzip and un-tar the file and click on coverage/index.html. I had to delete most of the files to decrease the file size for upload, so only the links for access work.

gcov reports line, branch, and function coverage, but lcov only reports on line coverage. I've added a link to the html to the gcov output that shows summary statistics for each file for line, branch, and function calls.

The report gives a very clear and browseable view of what parts of the system might benefit from more extensive testing. It's obviously useful for planning future testing, but also can be used in conjunction with debugging to see what lines and functions are being exercised or missed by existing tests of the functionality under investigation. It could even be helpful to give a static view of lines hit by a bug test case in lieue of using a debugger. Also, when you're writing a unit test for new functionality, it would be good to check what you're actually hitting with the test.

It uses gcov together with gcc to generate the statistics, and the lcov suite to create the html report. Both of these would obviously have to be installed to get a coverage report, but this would be an optional feature of the build. It only works with gcc.

To generate coverage statistics, you run configure with --enable-coverage and after building and running tests, you do make coverage. The process generates data files in the same directories as source & object files and produces a coverage directory at the top level with the html files. I've also set it up so a tar file with the html is generated. That is what I've attached.

The current coverage for the master branch can be viewed at https://coverage.postgresql.org/. We turned off branch coverage because gcov's version seems uncapable of understanding out code completely; see https://www.postgresql.org/message-id/flat/30022.1520642188@sss.pgh.pa.us for background. We'll probably turn it on again once we have gcc-8.

More information on PostgreSQL test coverage at https://www.postgresql.org/docs/current/regress-coverage.html.

More information on gcov at http://gcc.gnu.org/onlinedocs/gcc/Gcov.html, lcov at http://ltp.sourceforge.net/documentation/how-to/ltp.php (coverage/lcov tabs).