PgCon 2019 Developer Unconference

From PostgreSQL wiki

Jump to: navigation, search

Contents

Unconference schedule by Time and Room

Time DMS 1160 DMS 1110 DMS 1120
Tue 11:15-12:15 Zed Store Test Coverage Parallel Grouping Sets
Tue 12:15-13:15 Lunch
Tue 13:15-14:15 DirectIO & AsyncIO PostGIS 3.0 Locale apocalypse, collation, ICU stuff
Tue 14:15-14:30 Coffee Break
Tue 14:30-15:30 Problems / Possible solutions for high volume / latency links w/replication Fault Injection Framework SLRU to Buffer Cache but without on disk changes
Tue 15:30-16:00 Break
Tue 16:00-17:00 WAL Archiving extensions Constrain Session memory SQL standard stuff

Notes/Minutes

Test Coverage

  • We talked through various details of the existing coverage report.
  • David Steele talked about testing in pgbackrest project: https://github.com/pgbackrest/pgbackrest/blob/master/test/src/test.c
  • Interested in fault injection to get coverage for hard to test code paths.
  • Actions:
    • Write more tests. Everyone is welcome.
    • Let's get all function coverage scores to green.
    • Look into reactivating branch coverage.
    • Look into generating coverage diffs via cfbot.

Locale apocalypse

  • Upgrade from glibc 2.27 to 2.28 will break indexes for many users.
  • Details on this page: Locale_data_changes
    • Point users to that page.
    • Update that page with information on different OS versions over time.
  • Plan: Send out a community announcement about this soon.
  • Various technical mitigation options were discussed, but none seemed feasible to help existing users in the near future.
  • Plan for PostgreSQL 13: Make ICU an option for the global collation. Make it the default in the more distant future.
  • Concern: ICU might be slower. Should benchmark.

SQL standard stuff

  • Peter Eisentraut reported that he has joined the SQL standard working group and gave some insights into its inner workings.
  • slides of a presentation by Keith Hare that gives some information
  • For any more questions, contact Peter directly.
  • SQL:2020 will contain a built-in JSON data type; concerns about how this will interoperate with PostgreSQL's existing json/jsonb types; Peter and Oleg will investigate

Direct I/O & Async I/O

Overview of fsync problem

Goals:

  • Better error handling
  • Better performance
  • More predictability
  • Observibility
  • Remove double-buffering

Tasks:

  • Background writer needs to actually write pages out, as well as trying to maintain a set of "free" pages
  • Write throughput
  • Write combining
  • Smarter buffer eviction

Fault Injection

Requirements

  • system call error scenarios
  • protocol errors
  • hard to hit errors / fine-grained control
  • easy to write/read
  • deliberate corruption
  • concurrency
  • slow down execution
  • stress testing (ie: XID burn)
  • uncommon combos
  • SQL combination testing
  • fuzz testing
  • granular sequencing
  • Mutation testing

Notes

  • Why isn't Linux Fault Injection implemented?

Frameworks / solutions

  • pg_crash
  • pg_panic
  • Jeff Jane's framework
  • pgBackRest's framework
  • Dtrace can over-write returns calls of system calls
  • pg_swap_pages
  • ptrace
  • Microsoft test framework

https://github.com/greenplum*db/gpdb/tree/master/gpcontrib/gp_inject_fault

  • Add fault points to core
  • Extension that allows for triggering faults via SQL
   * Place of fault
   * Type of fault
  • Extension communicates fault requests to core code via shared memory
Personal tools