FOSDEM/PGDay 2018 Developer Meeting

A meeting of the interested PostgreSQL developers is being planned for Thursday 1st February, 2018 at the Brussels Marriott Hotel, prior to FOSDEM/PGDay 2018. In order to keep the numbers manageable, this meeting is by invitation only. Unfortunately it is quite possible that we've overlooked important individuals during the planning of the event - if you feel you fall into this category and would like to attend, please contact Dave Page (dpage@pgadmin.org).

Please note that the attendee numbers have been kept low in order to keep the meeting more productive. Invitations have been sent only to developers that have been highly active on the database server over the 10 and 11 release cycles. We have not invited any contributors based on their contributions to related projects, or seniority in regional user groups or sponsoring companies.

This is a PostgreSQL Community event.

Meeting Goals

• Review the progress of the 11.0 schedule, and formulate plans to address any issues
• Address any proposed timing, policy, or procedure issues
• Address any proposed Wicked problems

Time & Location

The meeting will be:

• 9:00AM to 5:00PM
• Brussels Marriott Hotel

Coffee, tea and snacks will be served starting at 8:45am. Lunch will be provided.

RSVPs

The following people have RSVPed to the meeting (in alphabetical order, by surname) and will be attending:

• Oleg Bartunov
• Joe Conway
• Andres Freund
• Stephen Frost
• Magnus Hagander
• Petr Jelinek
• Bruce Momjian
• Alexander Korotkov
• Dave Page
• Simon Riggs
• Andreas Seltenreich
• Tomas Vondra

The following people have sent their apologies:

• Andrew Dunstan
• Peter Eisentraut
• Etsuro Fujita
• Andrew Gierth
• Peter Geoghegan
• Robert Haas
• Alvaro Herrera
• Kyotaro Horiguchi
• Amit Kapila
• KaiGai Kohei
• Tom Lane
• Fujii Masao
• Noah Misch
• Thomas Munro
• Michael Paquier
• Dean Rasheed
• Craig Ringer
• David Rowley
• Masahiko Sawada
• Pavel Stehule

Agenda Items

Please add agenda items here!

• Tools for Commitfest process management - changes and future requests (Simon Riggs)
• 11.0 Release Review
• Commitfest item review
• Pluggable storages (Alexander Korotkov)
• CSN & 64-bit xids (Alexander Korotkov)
• Built-in sharding (Bruce Momjian)
• MERGE syntax, sqlsmith and concurrency (Simon Riggs)

Agenda

Time	Item	Presenter
09:00 - 09:10	Welcome and introductions	Dave
09:10 - 09:20	11.0 Release Review	All
09:20 - 10:00	Pluggable storage	Alexander
10:00 - 10:30	CSN & 64-bit xids	Alexander
10:30 - 11:00	Coffee break	All
11:00 - 11:45	MERGE syntax, sqlsmith and concurrency	Simon
11:45 - 12:30	Built-in sharding	Bruce
12:30 - 13:30	Lunch	All
13:30 - 15:00	Open CommitFest Item Review/Hacking time/Additional discussion	All
15:00 - 15:30	Tea break	All
15:30 - 16:35	Open CommitFest Item Review/Hacking time/Additional discussion	All
16:35 - 16:45	Future developer meetings	Dave
16:45 - 17:00	Any other business	Dave
17:00	Finish

Minutes

09:00 - 09:10   Welcome and Introductions
=========================================

Present:

Joe Conway
Dave Page
Stephen Frost
Tomas Vondra
Petr Jelinek
Bruce Momjian
Simon Riggs
Oleg Bartunov
Alexander Korotkov

09:10 - 09:20	11.0 Release Review
===================================

Stephen: Some large items outstanding in CF.
Dave: We're pretty good at not delaying the release for outstanding patches now,
 so don't expect any blockers.
Oleg: We have a very large SQL standard JSON patch outstanding. Andrew Dunstan
 is reviewing, but we need help.
Oleg: In particular we need help writing docs as it's extremely hard for
 non-native English speakers.
Tomas: It's hard to review patches without some docs.
Alexander: You have the SQL standard to describe it
Tomas: Yeah, I'm not going to read that
Dave: We don't need full polished docs to do a review
Joe: Yeah, a README or similar
Oleg: The lead dev is always online and responsive. We do have comprehensive
 reference docs, but not user docs.
Oleg: We also have a patch for opclass parameters. Useful for (for example)
 length of signature in GIST.
Tomas: We have a rule about no large patches in the last commitfest
Oleg: It's not too big, maybe 1000 lines and not very invasive
Stephen: You can just commit it (and revert later)
Simon: We need a revertfest
Simon: I can't help thinking we're doing this commitfest thing wrong. We have a
 big queue of patches, and your proposing quite reasonable additions that we're
 unlikely to get to. We need more reviewers.
Oleg: Maybe we need a development version
Simon: No, I think the issue is lack of primary review
Tomas: I don't think a developer version is the way. We'll get large,
 unfinished patches committed, and then what? What does it solve? You could just
 share a branch in your private repo.
Stephen: One thing that Andres does well is to include a big description and
 latest summary of large patches when emailing them.
Tomas: One way to get reviews is to solicit them directly. We also need people
 to review as well as submit patches.
Stephen: Reviewing patches is a great way to learn
Tomas: If you invest time in writing patches, you need to also invest time in
 reviewing.
Stephem: Companies need to bake in time to work with the community
Oleg: We try! But find it hard to work in the lists.

09:20 - 10:00	Pluggable storage
=================================

Alexander: We have a thread about pluggable storage, with a patch originally
 from Alvaro. I've joined the development and made some improvements. There is a
 new storage type with a different implementation of MVCC that can do update in
 place, and use an undo log. It's very important that these work together. The
 important thing for me is even if we retain our tuple identifier, it really
 becomes a row identifier pointing to a number of tuples. To implement we need
 to add params to our heap storage method, to provide the snapshot which can be
 used to find the correct tuple version in the undo chain.
Simon: I don't have much input on the API, but in terms of accepting it into PG,
 we don't have an alternative storage method at the moment. I would like this as
 I think it's needed for columnar storage, but we don't have such an Open Source
 engine, only commercial. We need a purpose for the API as we have to support
 it.
Tomas: I think the right word is proprietary not commercial, but without an API,
 how can we even test anything? I think the other problem is that the storage is
 just a small part of the problem.
Alexander: Maybe storage isn't the right word. Maybe "table engine". In PG Pro
 we have an in-memory storage engine that we're looking to make Open Source, but
 it uses FDW and is very different. Also, the EDB guys said they're going to
 publish zHeap as Open Source, but haven't done so yet as it's not ready.

[Discussion on EDB's Open Source vs. proprietary decision making and why zHeap
is likely to be open sourced, and why it might be bad if Oracle used such an API
for "InnoDB for Postgres"]

Stephen: It's a chicken and egg thing - we won't accept all the work for the API
 without a storage engine. Both are huge pieces of work.
Simon: We're always changing index APIs
Stephen: We need to ensure we can always extend and enhance our API, without
 having to worry about breaking properietary extensions.
Tomas: We don't break stuff needlessly - but if it is needed, we'll do so in
 major releases.
Alexander: I'd like to note that table engines for Postgres will be very
 different from MySQL - they will be tightly integrated, unlike MySQL where they
 are basically different DBMSs under the same roof. We shouldn't make an API
 just for proprietary, but for Open Source and our users first, and additionally
 for proprietary extensions.
Dave: I think that should be our default position for everything.
Joe: We should include the ability to prevent certain operations (e.g. BTREEs on
 columnar storage as noted by Simon)
Tomas: I'd actually like the blackhole storage engine from MySQL
Simon: Hannu wanted that, so data could be transmitted through WAL, but not
 stored locally.
Oleg: We need a reference storage engine.
Tomas: I'd like to work on that, so if you have any ideas.
Oleg: We need a team for v12. Maybe EDB with zHeap, PG Pro with in-memory and
 others on the API etc.
Simon: I can support that. If we say this is the API we're writing for v12,
 please work on storage engines for that release with that API.

Oleg: (Recap): for PG 12 we need to support new storage types, in memory,
 columnar, zHeap etc. We need an API.
Dave: We need someone to spearhead the effort like Bruce did with sharding
Oleg: I want commitments from the contrbuting companies to work on parts of
 this.
Bruce: This is like sharding (as Dave noted). I needed to re-assure Postgres
 users that in the future we will have a multi-node solution, and that took a
 lot of PR and organization.
Bruce: Let me think about how I'd PR that
Oleg: It's not about PR, it's about how we get things done.
Tomas: It's probably the right time of the year to do this now as we're right
 before the last commitfest. We should do this now, and have a meeting at pgCon.

Dave: All in favour of proposing to -hackers that we create a roadmap item for
 a pluggable storage API and engines being developed by various companies.

[All voted in favour, except Andreas who abstained]

TODO: Alexander to discuss with the various parties involved and prepare the
 proposal. Meeting to be setup at PGCon.


10:00 - 10:30	CSN & 64-bit xids
=================================

Alexander: CSN is commit sequence number. This is an alternative idea to our
 current snapshots which are an array of xmin, xmin and xids. The problem with
 this is that it was designed for single core processors, but for servers with
 hundreds of cores this approach is of quadratic complexity with the number of
 backends. CSN gives a number which orders the commits so we can find a snapshot
 with a single number. Original patch by Ants, with work from Heikki and now one
 of the PG Pro guys. Unfortuntely most of the current community work seems to be
 on improving the current model. I think we need to eventually switch to CSN to
 avoid significant efforts on micro optimisations of the current approach.
Simon: I'm confused about the difference between CSN and 64bit XIDs. Why are 
 they the same thing?
Alexander: No, they're not the same thing, but they could be overlapped in some
 implementation details.
Tomas: How large is the CSN?
Alexander: 64bit. There are some cases of regression over the current
 implementation, for example tables with lots of random seeks. One option is 
 instead of writing a hint bit, we write the CSN in the tuple header.
 Discussion with Andres and Heikki who suggested it's hard to guarantee atomic
 writes.
Simon: We already use 64bit atomic values in the code and clearly that works.
Alexander: It's not about writing in memory but writing to disk
Simon: There was a suggestion from Robert to switch to a different heap, using
the storage API. We could support both then deprecate the old one.
Simon: What's stopping you from having a hint bit and CSN on one tuple? That
doesn't have the benefit of avoiding lookups, but it does avoid WAL
Alexander: On some workloads you will have regression (but on others, benefits)
Simon: It's not going to happen in this commitfest is it?
Oleg: No!

[Discussion on whether we have to support 32bit machines in the future (yes) and 
how 64bit XIDs would affect that]

Alexander: The plan for 64bit would be to provide a patch to replace 32bit XIDs
 with 64bit XIDs in memory, and then an alternate heap with 64bit XIDs on disk.
Simon: That works - if you get the first part done, all the other patches become
 much smaller.
Oleg: Also, the CSN patch is big, but removes a lot of code from Postgres,
making it cleaner.
Alexander: Would you like to review CSN in light of logical decoding (Petr).
Petr: Yes.

SIDE TOPIC:

Release date for PostgreSQL 13 agreed: Friday 13th September 2019!!


11:00 - 11:45	MERGE syntax, sqlsmith and concurrency
======================================================

Simon: MERGE is an SQL standard command. The reason I'm working on it is that I 
 started in 2008 and would really like to finish it! There's been a lot of
 misunderstanding over the years, mostly around concurrency. I think it's
 possible to get something into Postgres 11. The current version is 14, but as
 of 10a, all standard functionality works. As of 13, all concurrency works.
Simon: SQLsmith. I've come here primarily to talk to you as MERGE is crushingly
 complex in it's syntax. I'd like to propose working with you to add MERGE
 support to SQLsmith, then we'll run it for 20 minutes to generate the next
 months work for me! I think fuzz testing is very important for this. The reason
 that fuzz testing is important is that MERGE works and there are no bugs in it
 at this time. Most people think it's half-baked crap, but it's actually not -
 it's clean code and it works. The code is cleanly distributed in a few areas, 
 and all the magic happens confined to the executor. Current status is at 
 https://wiki.postgresql.org/wiki/SQL_MERGE_Patch_Status.
Simon: I consider it committable. It has extensive docs, regression and 
 isolation tests. Works with everything we can think of.
Simon: Not supported features include RLS. It will error out on tables with RLS 
 at present, to prevent the first version potentially having security bugs.
Stephen: What concerns me is that we don't normally do that with new features.  
 Eg. Peter put a lot of effort into INSERT ON CONFLICT. I understand your point,
 but I don't really see this as being any different from other situations.
Simon: We have done this before. Partitioning was committed without docs and
 without support for INSERT ON CONFLICT. It may not be possible to support
 MERGE with partitioning in this release as partitioning still isn't finished.
Tomas: My question is, which part of the code will we have to fix to make MERGE
 work well with partitioning? Will we automatically get better plans with
 improved partitioning?
Simon: I can't answer that.

[Discussion about whether to include partitioning support despite likely
generation of poor plans. Simon seems against, others less so]

Stephen swings the conversation back to the similar issue of RLS support and
agrees to a request from Simon to review for possible security risks.

Alexander: Does MERGE use the same speculative insertion internal machinery as 
 INSERT ON CONFLICT?
Simon: Peter Eisentraut asked me that, and I said yes and got a one-word 
 response of "good". Others, including Peter Geogehan seem less keen. There is
 ongoing discussion about whether or not to attempt an INSERT following an 
 update failure (e.g. because a snapshot has gone away). Originally it was 
 suggested that it throw an error which is the code Simon wrote and works. 
 Others are now saying that was a misunderstanding and the INSERT should be 
 attempted.

[Discussion about the expected behaviour of MERGE in certain circumstances, 
what seems logical and what other databases do]
[Further discussion about making behaviour controllable via GUC or syntax. Simon
is currently preferring to throw an error until correct behaviour is clear]

Bruce: I haven't heard this issue discussed this clearly on the mailing lists.
 As was the case with SSI, it neeeded a simplified example to help people
 understand the issues.
Joe: This is the part where the spec says "implementation defined" right? Has
 anyone tabulated what the other DBMSs do so we can see if there's a clear
 answer?
Simon: Not really. I was told to throw an error on the mailing list, so that's 
 what I did.
Joe: This could be a problem for people migrating.

11:45 - 12:30	Built-in sharding
=================================

Bruce: Not a huge amount to report. We're in the multi-year approach to sharding
 and we're continuing to improve FDWs and partitioning. I've been looking at 
 what is committed so far - PG10 almost has enough for read-only sharding. PG11
 has a patch for parallel foreign scans that is important.
Oleg: We have a different approach for sharding.
Alexander: We have an extension that uses FDWs and our tsDTM to provide 
 transaction atomicity. It's on Github. It basically works but doesn't yet 
 support automatic recovery.
Oleg: We've tested it on 64 nodes and it works. It's Open Source. I don't know
 what to do about this. Maybe it's possible to include in pg_contrib?
Alexander: It also uses logical replication for redundancy of shards.
Stephen: It can parallelise queries?
Alexander: Yes, you need several patches to the FDW/core code.
Simon: It would be nice if there was a web page somewhere that explained what 
 the missing pieces are and the overarching architecture. If we knew what 
 patches were holding it up, we might be able to get more eyes on them.
Stephen: Can you break it up into smaller parts for commit?
Tomas: There was a patch for the TM API a couple of years ago
Oleg: Yes, there was a problem with the API
Alexander: The problem is lack of review
Tomas: I think the problem was that there was no extension to show use of the
 TM API
Oleg: There were two extensions.
Tomas: I do remember that one of the problems with the patch was lack of docs
Oleg: We have a wiki page on it.
Tomas: It's not something that has a chance to get in before the last commitfest
 - I would revive it before PGCon so we can discuss there. I think over the last
 two years we've got a lot of infrastruture that might make it more viable.

Commitfest review
==============

Commitfest item review, with your host, Stephen Frost...

Future dev meetings
- Dave- What do people want me to do?
- Simon- I don't want more than 1 face-to-face per year; want it announced when/where it's going to be, if there'll be another Brussels meeting then announce soon
- Dave- We don't know exact time
- Simon- Does have to be an exact date
- Stephen- just talking timeframe
- Dave- I can't guarantee that it'll happen ahead of time
- Simon- Main thing is about planning, not about booking flights
- Dave- We will have one at pgCon
- Bruce- Just one per year?  But which one, not at pgcon?
- Simon- Not being specific, just only want one per year
- Dave- Should be moved around?
- Simon- could be moved around but still be only one
- Dave- issue with that is not everyone travels, and the timing changes - which impacts what the meeting is about and for
- Simon- if there's more than one meeting then they end up being split
- Dave- if only one then it'll be pgCon, probably remain as admin/procedural meeting + unconference
- Tomas- Having just one would be easier to plan, will mean it's always in US or Europe and reduces number of attendees too
- Dave- that's true, doesn't disagree but there are pros/cons
- Simon- What was changed is that with multiple meetings we couldn't get everyone to all of them, more meetings means more cost or fewer people
- Dave- of those who have been to Japan or Pgcon or here, which seems like the best timing/most useful
- Tomas- Best timing for CF review/patches for next meeting, but probability of getting people here for this meeting is tough
- Dave- Probably won't get Tom here
- Bruce- only one dev meeting then there's no experimenting, only one means no failure possible, either pgEu or Ottawa
- Tomas- Prefer 2 dev meetings, one here or pgConf.Eu & pgCon Ottawa
- Dave- those two confs will likely attract people
- Tomas- Would be extra cost but likely to have good number of people at both, up to Simon to some extent
- Simon- Depends on people too and who wants to travel
- Dave- people who don't want to travel may not go to either
- Petr- big difference based on distance for travel
- Dave- discussed doing one in Asia and did one previously
- Tomas- Asia was a bit of a strange meeting, would keep one in Europe and one in US
- Dave- just did an unconf in Asia instead of a dev meeting
- Bruce- are you saying meeting like this isn't useful with 10 people..?
- Dave- Not useful if no one else finds it useful, happy to continue carrying on with it or is it a waste of time?  Fewer people than ever here and every year there is badgering people to get here and to have an agenda worked out
- Bruce- Might not be sure what to do until they get there
- Dave- suggested maybe do this as an unconference style instead of as a dev meeting
- Bruce- kinda done that today
- Tomas- thought it was useful and more developer patch-oriented meeting here and governance, et al, meeting at PgCon then it seems reasonable
- Dave- who thinks we should carry on doing something here
- Nearly everyone votes yes
- Andres- Not cheap to get flights on short notice, please send out notice earlier
- Tomas- here or at pgconf.eu?  like it here
- Dave- timing is good here, but may be fewer people
- Tomas- chance that PG hackers come to fosdem is lower
- Dave- stats for pgconf.eu very biased by what country it is in
- Tomas- if we want to have a meeting where there are more people from US, pgconf.Eu is more likely
- Dave- Europe one at FOSDEM or at pgconfEu?
- General favor for pgConfEu
- Dave- Will talk to Magnus about making it happen
- Dave- last question- dev meeting style, or unconf or?
- General favor for dev meeting style

Any other business
- Bruce- any comments about companies working together?
- Simon- Concrete suggestions regarding meetings and CFM app improvements
- Dave- Concrete suggestions?  Being able to vote to help prioritize work on patches?
- Simon- Need a way to say what the items are that we each care about and what things we're each going to do.  If I know that something I care about is being done then I can look at reviewing another patch, otherwise can't
  really look at other items
- Tomas- Not really a general agreement
- Simon- Some form of organized teamwork would be great
- Dave- need further discussion about the CF app in general
- Tomas- generally feel like it works pretty well
- Dave- One problem is that it takes a long time to figure out what the current status is, maybe add more metadata to the patch that anyone can update
- Tomas- When there is a huge thread, once in a while the person who is working on it should provide a summary email when the new patch is submitted
- Dave- Maybe that should be in metadata on the app somehow
- Joe- Or a way to flag the summary message or current point in the thread
- Tomas- Usually go through the thread anyway, would like to see the info embedded in the thread
- Dave- if you're looking for something to spend a couple hours on, better to have a summary available in the app or linked from the app
- Tomas- Should be for items which are in needs review
- Petr- Could match one which is the summary one in the app, maybe with a way to mark it in the app
- Tomas- That could be done, just don't want to have to maintain both the CF app and the thread
- Dave- Probably no specific change to make at this time but things to discuss on the list
- Tomas- issue with picking patch to review, patch in needs review with no reviewers, so go look but turns out there's a huge thread on it
- Dave- out of time, need to defer discussion but with concrete suggestions we can post something to the thread