PgCon 2019 Developer Unconference

Unconference schedule by Time and Room

Time	DMS 1160	DMS 1110	DMS 1120
Tue 11:15-12:15	Zed Store	Test Coverage	Parallel Grouping Sets
Tue 12:15-13:15	Lunch
Tue 13:15-14:15	DirectIO & AsyncIO	PostGIS 3.0	Locale apocalypse, collation, ICU stuff
Tue 14:15-14:30	Coffee Break
Tue 14:30-15:30	Problems / Possible solutions for high volume / latency links w/replication	Fault Injection Framework	SLRU to Buffer Cache but without on disk changes
Tue 15:30-16:00	Break
Tue 16:00-17:00	WAL Archiving extensions	Constrain Session memory	SQL standard stuff

Notes/Minutes

Test Coverage

• We talked through various details of the existing coverage report.
• David Steele talked about testing in pgbackrest project: https://github.com/pgbackrest/pgbackrest/blob/master/test/src/test.c
• Interested in fault injection to get coverage for hard to test code paths.
• Actions:
  • Write more tests. Everyone is welcome.
  • Let's get all function coverage scores to green.
  • Look into reactivating branch coverage.
  • Look into generating coverage diffs via cfbot.

Locale apocalypse

• Upgrade from glibc 2.27 to 2.28 will break indexes for many users.
• Details on this page: Locale_data_changes
  • Point users to that page.
  • Update that page with information on different OS versions over time.
• Plan: Send out a community announcement about this soon.
• Various technical mitigation options were discussed, but none seemed feasible to help existing users in the near future.
• Plan for PostgreSQL 13: Make ICU an option for the global collation. Make it the default in the more distant future.
• Concern: ICU might be slower. Should benchmark.

SQL standard stuff

• Peter Eisentraut reported that he has joined the SQL standard working group and gave some insights into its inner workings.
• slides of a presentation by Keith Hare that gives some information
• For any more questions, contact Peter directly.
• SQL:2020 will contain a built-in JSON data type; concerns about how this will interoperate with PostgreSQL's existing json/jsonb types; Peter and Oleg will investigate

Direct I/O & Async I/O

Overview of fsync problem

Goals:

• Better error handling
• Better performance
• More predictability
• Observibility
• Remove double-buffering

Tasks:

• Background writer needs to actually write pages out, as well as trying to maintain a set of "free" pages
• Write throughput
• Write combining
• Smarter buffer eviction

Fault Injection

Requirements

• system call error scenarios
• protocol errors
• hard to hit errors / fine-grained control
• easy to write/read
• deliberate corruption
• concurrency
• slow down execution
• stress testing (ie: XID burn)
• uncommon combos
• SQL combination testing
• fuzz testing
• granular sequencing
• Mutation testing

Notes

• Why isn't Linux Fault Injection implemented?

Frameworks / solutions

• pg_crash
• pg_panic
• Jeff Jane's framework
• pgBackRest's framework
• Dtrace can over-write returns calls of system calls
• pg_swap_pages
• ptrace
• Microsoft test framework

https://github.com/greenplum-db/gpdb/tree/master/gpcontrib/gp_inject_fault

• Add fault points to core
• Extension that allows for triggering faults via SQL

   * Place of fault
   * Type of fault

• Extension communicates fault requests to core code via shared memory