PgCon 2019 Developer Unconference
From PostgreSQL wiki
- 1 Unconference schedule by Time and Room
- 2 Notes/Minutes
Unconference schedule by Time and Room
|Time||DMS 1160||DMS 1110||DMS 1120|
|Tue 11:15-12:15||Zed Store||Test Coverage||Parallel Grouping Sets|
|Tue 13:15-14:15||DirectIO & AsyncIO||PostGIS 3.0||Locale apocalypse, collation, ICU stuff|
|Tue 14:15-14:30||Coffee Break|
|Tue 14:30-15:30||Problems / Possible solutions for high volume / latency links w/replication||Fault Injection Framework||SLRU to Buffer Cache but without on disk changes|
|Tue 16:00-17:00||WAL Archiving extensions||Constrain Session memory||SQL standard stuff|
- We talked through various details of the existing coverage report.
- David Steele talked about testing in pgbackrest project: https://github.com/pgbackrest/pgbackrest/blob/master/test/src/test.c
- Interested in fault injection to get coverage for hard to test code paths.
- Write more tests. Everyone is welcome.
- Let's get all function coverage scores to green.
- Look into reactivating branch coverage.
- Look into generating coverage diffs via cfbot.
- Upgrade from glibc 2.27 to 2.28 will break indexes for many users.
- Details on this page: Locale_data_changes
- Point users to that page.
- Update that page with information on different OS versions over time.
- Plan: Send out a community announcement about this soon.
- Various technical mitigation options were discussed, but none seemed feasible to help existing users in the near future.
- Plan for PostgreSQL 13: Make ICU an option for the global collation. Make it the default in the more distant future.
- Concern: ICU might be slower. Should benchmark.
SQL standard stuff
- Peter Eisentraut reported that he has joined the SQL standard working group and gave some insights into its inner workings.
- slides of a presentation by Keith Hare that gives some information
- For any more questions, contact Peter directly.
- SQL:2020 will contain a built-in JSON data type; concerns about how this will interoperate with PostgreSQL's existing json/jsonb types; Peter and Oleg will investigate
Direct I/O & Async I/O
Overview of fsync problem
- Better error handling
- Better performance
- More predictability
- Remove double-buffering
- Background writer needs to actually write pages out, as well as trying to maintain a set of "free" pages
- Write throughput
- Write combining
- Smarter buffer eviction
- system call error scenarios
- protocol errors
- hard to hit errors / fine-grained control
- easy to write/read
- deliberate corruption
- slow down execution
- stress testing (ie: XID burn)
- uncommon combos
- SQL combination testing
- fuzz testing
- granular sequencing
- Mutation testing
- Why isn't Linux Fault Injection implemented?
Frameworks / solutions
- Jeff Jane's framework
- pgBackRest's framework
- Dtrace can over-write returns calls of system calls
- Microsoft test framework
- Add fault points to core
- Extension that allows for triggering faults via SQL
* Place of fault * Type of fault
- Extension communicates fault requests to core code via shared memory