PgCon 2016 Developer Meeting

A meeting of the interested PostgreSQL developers is being planned for Tuesday 17 May, 2016 at the University of Ottawa, prior to pgCon 2016. In order to keep the numbers manageable, this meeting is by invitation only. Unfortunately it is quite possible that we've overlooked important individuals during the planning of the event - if you feel you fall into this category and would like to attend, please contact Dave Page (dpage@pgadmin.org).

Please note that the attendee numbers have been kept low in order to keep the meeting more productive. Invitations have been sent only to developers that have been highly active on the database server over the 9.6 release cycle. We have not invited any contributors based on their contributions to related projects, or seniority in regional user groups or sponsoring companies.

As at last years event, a Developer/Hacker Unconference will be held on Wednesday for in-depth discussion of technical topics.

This is a PostgreSQL Community event.

Meeting Goals

Define the schedule for the 9.7 release cycle
Address any proposed timing, policy, or procedure issues
Address any proposed Wicked problems

Time & Location

The meeting will be:

9:00AM to 12PM
DMS 3105 (3rd floor)
University of Ottawa.

Coffee, tea and snacks will be served starting at 8:45am. Lunch will be after the meeting.

RSVPs

The following people have RSVPed to the meeting (in alphabetical order, by surname):

Oleg Bartunov
Josh Berkus
Joe Conway
Jeff Davis
Andrew Dunstan
Peter Eisentraut
Andres Freund
Stephen Frost
Etsuro Fujita
Kevin Grittner
Robert Haas
Magnus Hagander
Heikki Linnakangas
Amit Kapila
Alexander Korotkov
Tom Lane
Noah Misch
Dave Page
Michael Paquier
Simon Riggs
Masahiko Sawada
Teodor Sigaev

Agenda Items

Please add suggestions for agenda items here.

Minutes. Why is this meeting minuted? Tasks and actions are OK to record, but minutes prevent discussion of certain topics. Discuss stopping taking of minutes, as it used to be.

(Major) Contributors. The lists of contributors and major contributors on the web site are not always up to speed with who is currently contributing. I think these lists should be updated (both as to adds and removals) more aggressively. The list of invitees to the developer meeting has tends to stagnate somewhat. Can we come up with a better way to keep this information up to date? [Robert Haas]

Core Team. Should core team members continue to hold indefinite tenure and be chosen solely by the existing core team? Is the current membership of the core team the best set of people for its current purposes? It started as a release group, but that function has somewhat been taken over by pgsql-release. [Robert Haas]

Postgres Professional roadmap and interacting with community and other companies. It's not a news that large open source projects like PostgreSQL are not driven by enthusiasts only. Significant part of development and other efforts is supplied by commercial companies. PostgresPro is a new company which contributes PostgreSQL. Despite some PostgresPro developers are community members many years they are also only starting to contribute on behalf of company. PostgresPro now has resources for contributing PostgreSQL and will have more resources for that in future. The aim of this topic is find a way to use these resources most efficiently for both community and company. We share our development roadmap. We'd like to find a way to act most coordinated, evade duplicated efforts and contradictions of interests. [Alexander Korotkov]

Feedback for PostgreSQL 9.6 Release Management Team [Robert Haas]

Review of PostgreSQL 9.6 Release Management Team [Simon Riggs]

Schedule for 9.7 [Robert Haas, Simon Riggs]

Sharding/Clustering. Should there be a preferred approach? Are certain approaches blocked? [Simon Riggs]

Cross-node transactional consistency. Do we want it in core at all? [Simon Riggs]

Logical replication [Simon Riggs]

Shared testing infrastructure [Amit Kapila]

Future Version numbers: we have three credible proposals: (a) keep things as they have been; (b) always increment the first digit and turn the second digit into update number, discarding the third digit; (c) always update the first digit, using the third digit for updates and keeping the middle digit as 0 except for special circumstances. If we choose (a) we have the follow-on question of determining whether the next version will be 9.7 or 10.0.

Agenda

Time	Item	Presenter
09:00 - 09:10	Welcome and introductions	Dave Page
9:10 - 9:25	Contributors	Robert Haas
9:25 - 9:40	Core team	Robert Haas
9:40 - 9:55	Release management team	Robert Haas/Simon Riggs
9:55 - 10:05	9.7 Schedule	All
10:05 - 10:20	Shared testing infrastructure	Amit Kapila
10:20 - 10:30	Postgres Pro Roadmap	Alexander Korotkov
10:30 - 10:45	Coffee break	All
10:45 - 11:00	Sharding/clustering	Simon Riggs
11:00 - 11:15	Cross-node transactional consistency	Simon Riggs
11:15 - 11:30	Logical replication	Simon Riggs
11:30 - 12:00	Any other business	Dave Page
12:00	Lunch

Minutes

09:00 - 09:10 Welcome and introductions Dave Page

Attending:

Oleg Bartunov, Postgres Professional
Josh Berkus, Red Hat
Joe Conway, Crunchy Data
Jeff Davis, AWS
Andrew Dunstan
Peter Eisentraut, 2nd Quadrant
Andres Freund, Citus Data
Stephen Frost, Crunchy Data
Etsuro Fujita, NTT
Kevin Grittner, EnterpriseDB
Robert Haas, EnterpriseDB
Magnus Hagander, Redpill Linpro
Heikki Linnakangas, Pivotal
Amit Kapila, EnterpriseDB
Alexander Korotkov, Postgres Professional
Tom Lane, Crunchy Data
Noah Misch
Dave Page, EnterpriseDB
Michael Paquier, VMWare
Simon Riggs, 2nd Quadrant
Masahiko Sawada, NTT
Teodor Sigaev, Postgres Professional

9:10 - 9:25 Contributors Robert Haas

Robert Haas raised the issue that the contributors list isn't promptly updated. The process has been a "spare time" activity for the core team. Simon suggested a process. Josh pointed out that the once-a-year process wasn't working. Stephen suggested a small team. Simon thinks it should be part of the release process.

Dave points out that the major contributor promotion is an issue. Robert Haas suggested something for the mailing lists. Noah wants some objective criteria. Simon worries about people gaming the system. Heikki said two different issues: who's going to do the work, and the list of people. Who should be on the team? Discussion ensued. Robert suggested we don't necessarily want top contributors doing this.

Requested for volunteers: Josh, Stephen, Joe, Dave. Stephen to follow up on creating a team. There were some questions about an emeritus list. Results of committee will be reviewed by privcommiters. Do other open source projects get this right? Not sure. Josh to ask other projects who gets this right. The team should also decide on rules for promotion/demotion. Some people go away for long periods of time and come back. Michael brought up people who blog a lot.

9:25 - 9:40 Core team Robert Haas

Robert's concern is that there's a very slow turnover of core team members. However, he thinks that it's a bit too slow. No mechanism he knows of to make sure that the right people get on the team. Who would we select. The roles of the core team have shifted over the years, the core team used to be the release team, but not it's more about project governance. Heikki said that things are OK because the core team doesn't do much. Stephen said that the role makeup of the core team is pretty good, even if we change the people, some committers, some non-committers. The project steering committee is what the core team is today.

Dave pointed out the role of the core team on the website. Some of those items have now been delegated. Discussion about appointing core team members ensued, we don't have a system. There is a bit of a diversity issue, which was discussed. Andrew suggested that we could consider the value of "outside directors" ala corporate boards. The core team may appoint a wider selection committee than core when we need to replace a member.

9:40 - 9:55 Release management team Robert Haas/Simon Riggs

This is feedback for the RMT experiment. Josh feels that beta coming out on time trumped everything. Stephen is not entirely convinced that maybe beta wasn't ready. The deadline for beta came out of the Brussels developer meeting. Tom would have liked to see the RMT be more aggressive about rolling back patches. Amit felt that the commitfest management vs. the RMT was confusing, maybe there should be the same role? Robert said the pestering role which David Steele took on was invaluable. Joe remarked how it used to be before the commitfests. Robert would have liked to have a longer feature freeze to beta gap, there wasn't enough time to fix patches; Andres disagreed, said that having a deadline sped things up.

There were some precipitous commits at the end. We think we know which those were. The RMT can't guarantee the quality of the release, they can just be gatekeepers. Committers have a responsibilty to not commit buggy code, that's on them.

The general feedback on the RMT is positive. We think we want to keep the RMT as part of the process. Amit suggested the RMT get involved in commitfest management. Robert said no; the RMT should be focused on the release.

Simon suggested that we need a better process for picking the CFM. Simon suggested using the release list. Discuss later.

9:55 - 10:05 Next Release Schedule All

Stephen wants the feature freeze sooner. Robert says that we don't need a much bigger gap, just two more weeks. The final CF ended with record speed. The last CF could start 15 days earlier. But moving up the schedule would cause patches to be more half-baked. The RMT found time pressure too much though, they were working 7 days a week. Tom suggested that the RMT not include people who have really large patches, but others thought this was impractical. Simon pointed out the importance of rotating people through roles. Noah pointed out that we don't actually know the quality of the release yet.

Robert would like to keep the beta date, but do feature freeze at Feb 15th. Simon says this at the expense of developers, who will use the time to stabilize patches themselves. Argument about dates ensued. The question is whether we're targeting the beta release date or the open the tree date. Various date schemes were discussed.

The dates agreed were:

CF1: September 1 to 30th
CF2: November 1 to 30th
CF3: January 1 to 31st.
CF4: March 1 to 31st.
Feature Freeze: March 31st
May 16th: Beta

10:05 - 10:20 Shared testing infrastructure Amit Kapila

Amit started with the "wicked bugs" from 9.5. Right now there is no way to share test machines in order to collaborate on bugs, because the authors can't reproduce the tests. We also don't have any way to publish performance numbers. We need some shared testing infrastructure. Dave brought up the performance machines hosted in Portland. Amit suggested buying some machines. We're not going to buy the expensive machines EDB has, like the 8-socket machine they have.

Can we see if there are ways that hardware owned by companies can be made available to others? People didn't know about the shared machines. We should edit the wiki page. There's issues of administrative control. Oleg mentioned access which we're given temporarily on donated machines.

Josh to look for the existing wiki page and send it to Amit.

10:20 - 10:30 Postgres Pro Roadmap Alexander Korotkov

Alexander talked about the new contributors they're training. He wants to coordinate roadmaps for various companies who are contributing to PostgreSQL. They've added a wiki page with what they're working on. Stephen said that Crunchy would be happy to share theirs, can use the wiki if that works for people. Folks need to understand that roadmap information is provisional, though.

Alexander stressed the importance of keeping information updated. Simon suggested that we have a collaborative roadmap, with individual's names. It would be good to know what people are working on. A combined roadmap would need to be a consensus document, and individual company roadmaps can be kept up to date better. A consensus roadmap would be inaccurate, and would encourage self-promotion. Andrew suggested having a single combined page. Robert is worried about the downside of having a "roadmap" which forces unrealistic development priorities.

Discussion ensued. People and companies will post individual roadmaps. Simon will create a unifying wiki page with links to all the others.

10:45 - 11:00 Sharding/clustering Simon Riggs

Simon asked about a roadmap for sharding. He thinks we need a clear statement for advocacy reasons, people ask about it all the time. We should have an unconference session on sharding, or maybe a wiki page. Simon questioned the FDW stategy; he says no details have been published, so it's impossible to judge the plan. Robert pointed out that no plan survives actual development. Discussion of the FDW sharding ensued.

Fujita is not focusing on sharding, he is just improving FDWs more. Various people are working on various projects, and somehow those will come together to form sharding and clustering. EnterpriseDB isn't specifically working on FDW Sharding, that's a Bruce project; EDB is working on some specific features, like async execution and aggregate push-down.

If people have alternate proposals, then propose them. None of the current projects are definitely so good they should block something else. Bruce speaks only for Bruce. Several people agreed that having a bunch of building blocks like distributed transactions useful. But Robert thinks that we're too far away from usable sharding to have a spec.

11:00 - 11:15 Cross-node transactional consistency Simon Riggs

This will be a discussion with the PostgresXL team and 2nd Quadrant. Will be a session at unconference tommorrow, the Pro team will have a session on multi-master.

11:15 - 11:30 Logical replication Simon Riggs

Simon mentioned that they need a node registry for logical replication. This is something which will be useful for sharding. It's a roadmap item for 2nd Quadrant. Theirs will have a graph of connections between machines. We need some concrete specifics about this. There's some stuff in PostgresXL. Everybody seemed to think a node registry was a good idea, but nobody had a model.

pglogical will be submitted as smaller pieces in order to get it into core next year. It was too hard to have it be a contribution and an external working system. Petr has said that the next submission will be different, he'll be forking pglogical. August 1st seems likely for submission.

11:30 - 12:00 Any other business Dave Page

Poll on Version Numbering

Josh called for a straw poll on version numbering. There are four proposals currently on the table:

1. Keeping things the way they are, in which we have three parts to the version number, a second "major" digit, a third "minor" digit, and a first digit which is about "marketing", or "breakage", and we argue about it. Eight people supported this. 2. Moving to a two-digit number, where the first digit advances every year, and the second digit is for minor releases. Thirteen people supported this proposal. 3. The same as 2, but with three digits, the middle one of which is almost always "0". Nobody supported this proposal. 4. That we shift to a date-based release name. Nobody supported this proposal.

One developer abstained.

Dave proposed that we run this by packagers, and ask for feedback on the breakage it would cause. Depending on that, we will move to two-digit numbers. Josh also to put up a blog post looking for breakage information. Peter will drive the follow-up process.

Simon called for a vote on whether the next release will be 10.0. There was already consensus that the next release is 10.0.

Select CFM

Amit suggested that similar to the RMT, we should have a designated team for the year. There should be one committer in that group of two or three people. Robert and Stephen pointed out some problems with that; it's hard to have a team, and it's a big time commitment. Heikki suggested that we get four volunteers for CFs next year. We could just solicit on the list, as long as we ask well before the CF starts.

Core was tasked with making sure that there is a CFM three weeks in advance of each CF.