PgCon 2024 Developer Meeting

Minutes

Rough, I (Andres) was delegated this because I was a few minutes late :)

Operations and Governance - Jonathan Katz

• Lots of committees aren't known even by people involved in the project
• Joe Conway plans to create a committee website
• Lots of independently moving parts of the project
  • partially intentionally done that way, for resilience to takeover
  • also an obstruction to transparency
• Teams:
  • Core Team - "last resort", conflicts etc
  • Committers - code obviously, plenty policy around that
  • Pginfra - manages all the postgres infrastructure, servers for git etc
  • Code of Conduct Committee - Deal with violations of the CoC
  • Security 0 deal with security issues in our code
    • CNA we can now assign our own CVEs
  • Release - manage releases of postgres
    • RMT
    • Press
  • Translators
  • Packaging - build binary packages of postgres
  • Press
  • Planet
  • pg.org moderation
  • pgweb
  • twitter etc
  • Contributors team
  • Funds
  • Google Summer of Code (GSoC)
  • PGCA - Trademark etc
  • Postgres Women
  • Funds Groups
  • Events Canada, organizing this conference
  • I missed transcribing some, Jonathan missed listing some
• Amazing that it works, but it also often doesn't
• Robert: Really important to have list public, for volunteers to be able to oin
• Karen Jax: staffing, not manning teams
• Joe: Quick recap of the plans for listing of the teams on the website
  • some teams already have pages, but there's no coherent way to find those
  • have index with brief description and links to pages

Core Team - Tom Lane

• At Berkeley, Postgres then Postgres 95, a few outside users
• Marc Fournier volunteered to host development
• Initial Core Team:
  • 1996:
    • Mark Fournier
    • Bruce Mojian
    • Bryan Henderson
  • 1998:
    • Jan Wiek
    • Tom Lane
• Initially did nearly all the work
• Early additional responsibility:
  • releases
  • infrastructure
  • committter selection
• Additional responsibility
  • Liaison
  • Security
  • Trademark
  • really because ther wasn't anybody else
• Core never did:
  • Website
  • Advocacy
  • Fundraising
• 2000: Landmark / Great Bridge
  • started a postgres services company
  • closed in .com bust
  • Asked: How can this be done well with the community?
    • Not enough bandwidth / experience in core *> delegate
  • Delegate:
    • security, in 2007
    • pginfra, in 2011
      • took over more incrementally
      • Tom doesn't remember all the details
      • some small preexisting responsibilities
    • Trademark issues
      • PgCAC, formally incorporated NPO
    • Release, in 2015
      • are we ready
      • do we need to release
      • about 2017 we switch to time based releases, for both minor and major releases
    • RMT, in 2016
    • Committter selection, in 2015
      • private committers list
    • CoC, in 2016
      • initial idea was for core to do enforcement
      • now CoC does that
  • One would think core doesn't have work after all that delegation
  • Still lots of coordination / "cat herding"
  • Important teams overlap with Core, CoCC is explicitly separate
  • largely independent teams
    • Advocacy
    • Funds
  • Questions:
    • Stacey: Core is self selecting, plans to change that?
      • Tom:
        • Not currently
        • Should definitely be long-term members for institutional memory
      • Bruce: Takes a while to get there, we were looking for people a few times, didn't work out
      • Robert: Self selecting is an issue, but not clear what the solution is
      • Dave: It's hard to model something resilient against takeover
      • Karen: At least we should be honest about this issue toward the community
      • Heikki: Community is very concensus drive, hard to make decision, but also good to not piss off people. Core team has been decent at that.
      • Tom: Policy that there is not more than 50% of one company
      • Robert: Feels awkward to be "steered" by a something as opaque

Slides: Core Team Update

Security Team - Noah

• 17 people
• receives reports of security issues
• issues come from three groups
  • postgres devs
  • people we haven't heard of, typically something operating automated scanner
    • our mailing lists are public, you know?!?
  • occasionally reports by "real" security researchers
• fixes developed by
  • reports
  • security team members
  • "outsourcing program", to non-security member committers
• 7 security issues in the last year, one high risk one
• We are now a CNA
  • CVEs are assigned for most vulnerabilities
  • new: encouragement to become our own numbering authority
  • why worthwhile: CVEs assigned by outside numbering authorities by other numbering authorities
  • with CNA we're supposed to be asked first
  • now can update metadata for old CVEs
  • also now responsible for assigning CVEs in the wider postgres space
  • also get asked about filling out security questionaire
• Question to audience: Problems with security model?
  • Dave Cramer: Process more complicated than github's
    • tarballs are released, commit messages mails are suppressed, but everyone can look at commits
    • in theory we could make fix in private repo, like we did once before

PostgreSQL Community Association - Steve Singer

Media:Pgconfdev-leadership-meeting.pdf

• Special purpose entitity to focus on intelectual property matters
• Maintain and protect assets (trademarks, domain names, not code)
• does not hold copyright on postgres code, held by individuals
• Canadian Not for Profit
• Board:
  • Dave Page
  • Steve Siunger
  • Jonathan Katz
  • Jaime Casanova
  • Peter Eisentraut
  • Marc Fournier
• self selecting membership, by board (past and present members)
  • aware of issues around that
• Why:
  • ensure fair use of PostgreSQL brand assets
  • otherwise somebody else can call something postgres
  • requires enforcement, otherwise trademark is abandoned and can't be enforced
  • keep PG fre and open source
• Where:
  • Canada
  • US
  • Europe
• How:
  • controlled fair use allowed
  • example: Brand names involving postgres need to be licensed
  • standard license for templates, but often companies want modified terms
• Key Activities:
  • Oppose Trademark Registrations
    • some conflicts around that are public, others not
  • Ask that the trademarks be used properly
  • License where needed
• Highlights
  • Fundacion PostgreSQL conclusion (after three years and lots of money)
  • Trademark oppositions, cease & desists
  • Donor Recognition Program
• Why private?
  • public statement can have legal consequences, need to be reviewed by lawyers, costs money for review
  • rather work amiciably, rather than public shaming, often succeeds
  • some unlicensed uses are because we didn't have the time/money to do something about them
• Financial highlights:
  • Net loss: CAD 11k
  • Goal: Improve financials via Donor recognition program
  • Various levels that grant public recognition
• Question:

Code of Conduct Committee - Eliza Benett

• Eliza is chair since 2014
• Overview of the last year
  • Representation:
    • 5 members, 3 male identifying, 2 female, plus chair
    • geographic spread: 2 from Canada, 1 from Asial, one american in Asia, 1 from Europe
    • Volunteers, selected by current members
    • How can group size be increased
  • Activity
    • 8 complaints in 2023/24 (inappropriate conduct on community channels (2), at community events (2), non-community channel (3))
    • Outcomes: no violation (3), out of scope (2), corrective recommended and taken (2), corrective action recommended (1)
    • processing done by lead investigator, always with at least one other person
    • [couldn't keep up]
  • Trends:
    • 2023/24 have been busy
    • Complaints on social media, [couldn't keep up]
  • Opportunities:
    • need new members
    • update CoC policy, hasn't bene updated significantly since introduction
    • improve enforcement, appeals process
    • inform community events about CoC existance / use of CoC[C]
  • Revisions:
    • Roles & responsibilities
    • Scope clarification
    • Appeals process
    • Record sharing and retention, records exist since 2019 (unsure)
    • Communication processes
    • ....
    • proposed timeline: Summer 2024, engegement on policy revisions, September: 2024, in effect

Slides: Code of Conduct Committee Update

Port report - Thomas Munro

• slides
• lots of platform that postgres used to run on, but doesn't anymore
• still support about 11 platforms
• most users on linux, some on windows, the rest probably miniscule
• Robert Haas and Devrim Gunduz think there are way more on windows than Thomas guessed
• Dave Page says some windows apps embed postgres
• Estimate about problem per platform, overwhelmingly windows
  • Robert: Lots of it just perl crap
• Highlights:
  • AIX was dropped in 2017, suddenly IBM folks appeared, rely on Postgres
    • AIX is weird, unusual linker/object format
    • Old OS versions in buildfarm
    • Vendor compiler requires extra work
      • some bugs got fixed
  • HP-UX was dropped in 16
    • no real complaints, one mention
  • Solaris <= 11.3 in 17 was dropped
    • solaris is less painful
    • user interest, solaris buildfarm animals by users
    • also illumos
    • they stopped increasing versions after 11.4
      • should we drop support for the sunpro compiler?
    • won't support newer things like direct-io
  • OpenBSD:
    • Short supported version window, buildfarm animals exist that are very old
    • uses libressl
  • Linux+musl
    • different libc, a few incompatibilities that we have to deal with
    • Have a buildfarm animal now
  • Windows:
    • Removed old perl based build system, did make things easier
    • Thomas never used windows, despite working for Microsoft
    • Locale support is very unstable, breaks frequently
      • particularly with Turkish locale, breaks all PG databases with the locale, but also some others
    • truncations fail occasionally and causes corruptions
    • Socket event loss, causes weird problems
    • Thomas tries to fix things, but doesn't use windows, hard to find testing
    • Thomas finds our state of windows support embarrassing
    • We ought to make some decisions to improve the situation, requires making dropping some filesystems
      • who makes decision?

Group 1

• Action item: bugtracker
  • Presented by Matthias Van de Meent
  • would be useful to have a bug-manager responsibility, likely longer than a CFM
  • no agreement: add feature to track a commitfest entry for threads without patches
  • Same topic was also discussed by Group 4 (Peter Eisentraut)
    • We agreed that we should have one
    • Who would curate? Hire somebody?
    • What software to use? Gitlab?
  • Melanie: Is it it really a technical problem?
  • Andres: We seem to have started having agreement on having a bugtracker, that's different than in the past
  • Heikki: Some tracking would be useful, even if dumping ground
  • Robert: Relying on bugtracking / CF tracking person doing all the work on summarizing huge threads can't scale
  • Melanie: Fixing bugs is more important than new features
  • Bharat: Authors should summarize state in commit messages, several agreeing with that (Noah, Steve, Robert)
  • Robert: How to balance responsiblity to deal with bugreports and feature work
  • Melanie: Responsiblity of somebody doing triage would not be in-depth work, just high level triage
  • Alvaro: Just wishing there weren't bug doesn't work
  • Jelte: Knowing about many people are affected is useful

• Action item: tooling
  • More developer oriented tooling are required, including things like pull request
  • Platforms like github are problematic due to restrictions imposed by companies (e.g. github was restricted in Iran for a while)
  • we could use a hosted gitlab, or hosted by other open source project
  • Matthias: Platform notifications are hard to use
  • Robert: Real problem is changing the workflow
  • Andreas: We could have patch review events, to get more people involved?
  • Jonathan: Similar to python sprints
  • Robert: That's likely not solving problems with complicated patches
  • Andreas: Could do something like docs on a separate platform

Group 4

• Presented by Peter Eisentraut

• Action item: Windows port
  • Few / no core developers really care about windows port
  • Users are less sophisticated and communicate less
  • Andrew Dunstan would look at one of the pending technical issues
  • Problem areas
    • locale support - default to windows?
    • what's with refs - it doesn't support posix mode
  • Heikki: Should we just drop windows, similar to AIX
  • Robert: Bugs like the windows corruption issue is not enough reason to drop support, sometimes we just have bugs

• Action item: Freeze policy
  • Peter and Matthias
  • Group thought that this year was an anomaly, doesn't necessarily need a large scale change
  • Encourage more incremental work, don't require perfection, leading to pushing very late in cycle
    • Prioritize architecture of details like default values
  • Impact on feature freeze on release cycle
    • Branch earlier, to not have to postpone as much, group didn't really have agreement
    • Melanie: Issues with backporting, focus on stability
    • Matthias: Doesn't see a point reviewing during feature freeze
    • Robert: Committer authored patches are part of the problem, authors have control over time
    • Vik Faering: Once stable branch is forked, nobody cares about stable branch
    • Tom: Agrees
    • Matthias: Can strike balance, by moving branch up a month or such
    • Robert: One question is if there are more reverts, that's bad after the branch
    • Jonathan: Not convinced that that's really the issue, we had important fixes late
    • Robert: Not really addressing the issue of large stuff late in the cycle
    • Matthias: Still a problem not being able to get anything in
    • Melanie: Don't think many agree with you

• Action item: yubikeys for committing
  • no agreement on forcing, but encouraging is ok
  • laptop security etc would be good

• Action item: Tooling
  • Should be easier to find patches that e.g. have tests that never worked, similar tooling things

Group 2

• Presented by: Andreas Scherbaum

• Action item: Exhibition Effort
  • we're not showing up at enough non-PG conferences
  • Jonathan: PGUS has some funding, but are not covering that much
  • More efforts to support groups like pgWomen
  • consider having an application developer oriented conferences

• • • Stacey: At conferences like djangocon people say they like postgres, but feel like postgres conferes aren't "for them".
    • Stacey: Perhaps collaborating with conferences like djangocon would help reach people and then get them to postgres
  • People should submit talks at more conferes

• Action item: SPI

• • Jonathan: SPI is opaque, we don't know how much money we have, we get money from donations
  • Heikki: Why is SPI not funding pgca
  • Steve: SPI is very complicated to deal with, partially due to being out of the country, partially just SPI being SPI
  • Jonathan: SPI didn't give money for pgconf.dev, that should have been easy
  • Alvaro: Could SPI pay for a bug manager, agreement in audience
  • Robert: Companies could also provide staff tracking for

• Action item: Benchmarking
  • TPC is still relevant
  • Better testing needed
  • Find way to run meaningful test with every relevant patch
  • Jonathan: There's lots of money in the postgres world, there should be money for better testing

• Action item: Commitfest
  • Not enough reviewers, need more
  • Idea: review events

Notes from Andreas Scherbaum

• Exhibition Effort
  • Go to other conferences with PostgreSQL presence and content (have a booth)
  • Ask community to submit talks
  • Support different efforts to grow the community (like pgWomen)
  • Consider a joint postgres.app event with another community, such as `django`
• Separate discussion about SPI membership
  • And how to finance the community
  • Come up with ideas to use the funds, and organize them into useful and doable form
• Tooling
  • Consider in-person events for recruiting new people
    • For example, a post-conference Community Day similar to sprints to give people a chance to become involved
  • Consider keeping documentation in GitHub/GitLab
• Benchmarks
  • TPC is still relevant
  • Update TPC tooling or get tooling
  • Find a way to run meaningful tests with every relevant patch
  • Create a test plan which can be presented to test sponsors
• Commitfest
  • Not enough reviews, and reviewers
  • Try to test patches on buildfarm before commit
  • Have patch review sessions at conferences

Group 3

• Presented by: Jeff Davis

• Action item: Protocol changes
  • Which features do we want?
  • How do we introduce changes, dealing with breakages?
    • new protocol version?
    • optional features?
  • Desired features
    • Compression
    • Proxies and conneciton poolers
    • Transparent column encryption
    • Pgpool would like to have sequence numbers
  • Required vs optional features
    • Depends on implementation complexity on the driver side?
  • Concerns around
    • too partial solutions
    • genericity introducing unnecessary roundtrips
    • concerns about breakages
  • Robert: How do we put various efforts into something coherent

• Action item: Surprising behavior around roles and privileges
  • Some of this is user hostile
  • Users can't control what code is executed with their privileges
  • Function authors don't know what their function will do
    • requires setting search_path, but most won't
  • There's no way to drop privileges
    • SET SESSION AUTHORIZATION and SET ROLE can be undone via RESET SESSION AUTHORIZATON
  • Hacks and special cases
    • e.g. around table maintenance tasks like REINDEX and logical replication
      • it's safe to reindex, but not safe to insert
  • We define a lot of these to not be security issues, but they often are for users.
    • We don't even document much of this
  • Noah: Is it really a problem, we document this?
  • Jeff: Yes, we document this just for security definer, but it's also a problem in other situations
  • Noah: Now agree
  • Concensus: People other than Robert and Jeff need to get involved, including at some point agreeing to do something