CommitFest Checklist

From PostgreSQL wiki
(Redirected from Running a CommitFest)
Jump to: navigation, search

CommitFest Checklist and Timeline

This page contains a list of things to do and when to do them for the CommitFest Manager. It's based on the current CommitFest app (written by Robert Haas), and will change once the new CF App is done.

Three to Four Weeks before CF

With luck, you will know at least this far out that you are going to be CommitFest Manager (CFM). If so, do the following things:

  • announce to pgsql-hackers that you are the next CFM and when the CF will be beginning
  • attempt to recruit an Assistant CFM from -hackers and other places
  • begin saving all pgsql-hackers email in your local mail store for searching, if you don't do this regularly
  • check that you have administrative permissions on the CF. If not, request permissions fix from pgsql-www.

5 to 7 days before CF

  • Post a reminder to pgsql-hackers (and your blog?) that the CF is beginning on the date, and that they should get their patches registered/submitted.
  • Do a patch sweep (see below).
  • Go over the pending patches and assign them categories or change their categories if they're wrong.
  • Recruit people to pgsql-rrreviewers to be assigned random patches (see below).
  • Post to pgsql-rrreviewers and ask who will be available for the upcoming CF
  • Check each and every patch in the CF, and find/update:
    • patches which already received some review prior to the CF, and make sure this review is linked in the comments on the patch.
    • patches which have been already committed/rejected/returned before the CF, and update their status accordingly

Patch Sweep: search the pgsql-hackers archives from the prior 2 months for messages with the attachment "patch". Make sure that all patches submitted to -hackers are in the CommitFest (or in the prior CommitFest). If they are not, email the patch author asking if they want to submit their patch to the CF (see template below). New contributors should probably be added to the CF even if they are slow to respond, but veteran contributors should not be.

pgsql-rrreviewers: This mailing list includes the "Round Robin Reviewers" (RRR) who are regular contributors who want to help with reviewing patches, but aren't sure what they want to review. As CFM, you will be assigning patches to review to the RRR to keep them busy.

2 days before CF

  • Send reminder to -hackers that the CF is beginning in 2 days.

First day of CF

  • Change CommitFest status from "Open" to "In Progress"
  • Change status of next CommitFest if needed.
  • Announce that the CF has begun on -hackers and that any new patches will need to be put on the next commitfest.
  • Assign patches to review to all RRR.
  • Send a reminder to all patch submitters that they have a patch in the commitfest, and that they will be expected to review at least one patch by someone else.
  • If anyone signed up to review a patch more than 4 days before the start of the CF, send them a reminder that they're reviewing a patch.

5 days after the beginning of the CF

  • Check all assigned reviewers. If the reviewer was assigned before the beginning of the CF, and they haven't posted anything, take them off the patch and send them a nice note (see below).
  • Make a list of all patch submitters who have not volunteered to review someone else's patch. Send each one a private note reminding them that they need to do that (see template).
  • Check "waiting on author" patches per "Every 5-7 days" below.

10 days after the beginning of the CF

  • First, do the stuff under "every 5-7 days" below.
  • Make a list of all patch submitters who have submitted a feature patch (i.e. not bugfix or docs), have not claimed any patch for review or posted a review, and have not responded in some what to the private email sent on Day 5. Publish this list to -hackers (see template below)

Each 5 to 7 days of the CF

  • Check patches marked "waiting on author". If they've been waiting for more than 5 days, and there's been no status update, do the following:
    • If you're clearly waiting on a new patch update, then just mark them "Returned with Feedback".
    • If they're waiting on discussion on -hackers, and the discussion has died out, ask for a status update on the list.
  • Check patches that are marked "waiting for review" and have a review assigned.
    • If there's been a review, make sure it's logged. Add the review as a comment if it hasn't been.
      • If the review says it's "ready for committer", change it to that status.
    • If there's ongoing discussion, or if the review found stuff to fix, mark the patch as "waiting on author" and add a comment. For really huge patches, you may wish to wait a little longer than the strict 5-7 days.
    • If there hasn't been any review activity/discussion by the reviewer for 5 days or more (since the beginning of the CF), send the reviewer a reminder and take their name off the patch.

Constantly During CF

  • Monitor the CF app Activity Log page.
    • Watch for unpaired log entries which may indicate a patch or review without an associated status change. Update the status where appropriate.

5 to 7 days before end of CF

  • For any open patches:
    • If there hasn't been recent discussion, send an email to the reviewer reminding them CF is almost over and asking if they're doing ok or could use some help.
    • If there is recent discussion, post to the discussion thread asking if it'll likely be ready by the end of CF.

3 days before end of CF

  • Post to -hackers with a reminder of what will happen to open patches on the last day of CF.

Last day of CF

  • For patches marked "Waiting for Author" and having at least one review, set to "Returned with Feedback" and send the appropriate email.
  • For patches marked "Needs review"
    • If it received at least one good review, move it to the next CF (removing the current reviewer reservation)
    • Otherwise, leave them pending

Sudden Death Overtime

Once the closure date for the CF has passed, how patches get dealt with changes:

  • As soon as a patch gets a solid review, if it needs any changes at all it's marked "returned with feedback".
  • Patches which were marked "ready for committer" and turn out to need further work also get "returned" instead of going to "waiting on author"
  • Patches which require further spec discussion, performance testing, etc., get "returned" immediately.
  • Patches from submitters who have not, themselves, done any review, and are still "needs review" get pushed to the next Commitfest.

The goal is to close out the CF as soon as possible, without skipping review for any patch which still deserves a review.

After CF

  • Close CF in app (but make sure all patches are moved first; otherwise build machinery like cfbot will drop them)
  • Make the next CF current
  • Send reminder email to authors with patches that were Returned with Feedback reminding them to add their patch to the next CF if they plan to continue working on it.
  • Send "thank you" email to reviewers, reminding them they are more than welcome review in the next CF even if they haven't submitted a patch.
  • Assemble two lists of reviewers for the eventual release notes:
    • Reviewers who did substantial work on the patch (usually code work) to make it go in and the patches they worked on, and
    • Reviewers who did a regular review, as one big group
  • Have a beer or three - you've earned it. :-)

Email Templates

Patch Sweep

For new contributors:

   [contributor name]:
   You sent in your patch, [name of patch file] to pgsql-hackers on [date posted].  
   You may not be aware, but in order for your patch to be included in PostgreSQL, 
   it needs to be part of the next CommitFest.  Accordingly, I have added your 
   patch to the list here [link to CF entry].  If you want to withdraw your patch, 
   or if you simply won't be around to answer questions during the 30 days of 
   this CommitFest, please let me know and I'll remove it.  Thanks!

For experienced contributors:

   [contributor name]:
   You sent in your patch, [name of patch file] to pgsql-hackers on [date posted], 
   but you did not post it to the next CommitFest [link to CF page].  If this was 
   intentional, then you need take no action.  However, if you want your patch to 
   be reviewed as part of the upcoming CommitFest, then you need to add it yourself
   before [start date].  Thanks for your contributions.

Patch Reminder

   [contributor name]:
   As a reminder, you have the following patches in the current CommitFest:
   [list of links to their patches]
   If you are not prepared to follow up on review to these patches during this 
   CommitFest, please let me know and I will remove them now.  I also remind
   you that each patch submitter is expected to review at least one patch from
   another submitter during the CommitFest, so please pick someone else's
   patch to review as soon as you can.  
   Thanks for your contributions to PostgreSQL!

Reviewer Clear

   [reviewer name]:
   Since you haven't had time to write a review of [patch] in the last 5 days, 
   we've taken your name off the reviewer list for this patch.  Of course, you 
   are still very welcome to review it if you can find time.  We're just removing 
   your name so that other reviewers know the patch still needs attention.  
   We understand that day jobs and other things get in the way of doing patch review 
   when you want to, so please come back and review a patch or two later 
   when you have more time.  

Returned with Feedback

   [author name]:
   [patch], which you submitted to this CommitFest, has been awaiting your attention 
   for more than five days.  As such, we have moved it to "Returned with Feedback" 
   and removed it from the reviewing queue.  Depending on timing, this may be reversable, 
   so let us know if there are extenuating circumstances.  In any case, you are welcome 
   to address the feedback you have received, and resubmit the patch to the next CommitFest.  
   Thank you for contributing to PostgreSQL.

List of Non-Reviewing Submitters

   Subject: Submitters who have not reviewed
   Per the Developer Meeting of 2012, each patch submitter is required 
   to do at least one patch review for each submitted patch.  The following
   people have submitted one or more patches to this CommitFest, and have 
   neither claimed any patch for review, nor responded to a private reminder
   to do so.  This list is being posted in hopes that these submitters will 
   see it and do a review soon.  Without each submitter doing a review, 
   the PostgreSQL project cannot promise to review all patches.
   [list of names]