Cfbot

From PostgreSQL wiki
Jump to: navigation, search

Frequently Asked Questions

What is this?

In short, Cfbot is a bridge between traditional mailing list-based open source software development and a modern continuous integration system.

PostgreSQL is developed on the pgsql-hackers mailing list. Email threads that contain patches proposed for commit are registered in a Commitfest entry, analogous in some ways to a "pull request". The Cfbot checks all registered patches for bitrot and triggers continuous integration testing by maintaining its own set of branches on Github.

Can I have CI without Cfbot?

Yes! As you can see here, any Github account that has Cirrus CI enabled (free for open source projects) will trigger CI testing on every commit (see the instructions in src/tools/ci/README). (Other public git hosts and CI systems could also be supported, if someone does the work to propose the control files to make that happen.) The Cfbot is just a way to do the same for email threads, and tabulate the results in one place, to make it easier to keep track of the state of 200-300 active patches at a time.

Who is it for?

Individual developers can look at their personal patch list to watch out for bitrot and failures.

Reviewers, Commitfest managers and committers can look at the whole result table to find things to investigate.

Why not use Github pull requests?

This suggestion has so far not had much success in the community, and is a bigger topic than the author of this humble tool can address. Cfbot is an attempt to give similar benefits while maintaining the traditional email-based workflow.

Which attachments are considered to be patches?

First, it ignores emails unless they have a .diff[.gz], .patch[.gz], .tar.gz, .tar.bz2, .tgz attachment. It can be useful to post an alternative or unrelated patch with a .txt ending if it is not intended to replace the main proposed patch. (Code: interesting emails are noticed here).

When it sees an interesting email, it decompresses and expands as required first, and then applies .diff and .patch files after sorting by name them so that 0001, 0002 etc prefixes are respected. Everything else is ignored, so if you send an email with attachments 0001-foo.patch, 0002-bar.patch and 0001-baz.txt only the first two are applied. This can be useful for extra material that might be a patch but shouldn't be considered part of the submission. (Code: interesting patches are applied here).

Each email should contain a self-sufficient set of patches to be applied the current master branch; there is no way to say "this is incremental on top of the patches already attached to some other email".

Patches are applied with GNU patch for now. It might be better to use git am at some point, but the goal was to support existing practices, and many contributors post raw diffs rather than git format-patch files.

When are patches tested?

Whenever a new patch is posted as an attachment (within a few minutes typically), and also periodically to check against changes in the master branch. The period is currently 48 hours, but this is subject to change depending on resource consumption. (Code: deciding which patch to process next happens here.)

What do the icons mean?

A hollow checkmark (tick) or cross means that the result has not changed recently. A solid checkmark or cross means that the result is a change; for example, it was previously failing, and now passing, or vice versa. The "alt" text (visible in most browsers by hovering with a mouse) shows the name and status of a "task". If you click on the icon, Cirrus shows the output and artifacts such as logs and regression test diffs.

What are there multiple icons for each patch?

Broadly, they correspond to different operating systems that are tested concurrently. The tasks are not defined by Cfbot (though in earlier versions they were, before .cirrus.yml was promoted to being part of the PostgreSQL source tree for everyone to use and maintain) and may change, but here's a list as of the time of writing:

  • CompilerWarnings: check for warnings with both GCC and Clang, with asserts and without, and try to build the documentation.
  • FreeBSD
  • Linux
  • Windows
  • macOS

Why Cirrus CI?

There are many competitors in the CI space and Cfbot did previously use a couple of others, but at the time of writing Cirrus is the only system we can find with these properties:

  • allows test results to be viewed by anyone without some kind of account
    • users who don't want to open an account on a commercial service benefit from CI merely by sending patches to our mailing list
    • URLs can be shared in our mailing lists for all to read, as part of public discussions
  • has resource limits high enough to handle testing ~300 branches every couple of days
  • has support for a good range of operating systems and raw virtual machines, not just Linux and not just containers
  • has super responsive, proactive and helpful support staff

What are the future plans for Cfbot?

In no particular order:

  • Integrate with commitfest.postgresql.org (ie show results there)
  • Become webhook based (commitfest pokes cfbot with new patch, cfbot pushes branch, cirrus pokes cfbot with results, cfbot pokes commitfest with results), rather than the current polling/webscraping/cron system
  • Migrate to postgresql.org infrastructure (a Debian machine)
  • Use a separate Cirrus task to apply patches, instead of the current scheme (applying inside a FreeBSD jail, a bit too magical)

How do I change/improve the tests?

Modify .cirrus.yml, test in your own Github account, and propose changes to the pgsql-hackers list.

Who do I contact?

If you would like to discuss CI for PostgreSQL in general, the pgsql-hackers mailing list. For issues relating to the way Cfbot makes branches or collects results, please contact thomas.munro-at-gmail.com.