PostgreSQL 9.6 Open Items

Open Issues

• Oddity in EXPLAIN for foreign/custom join pushdown plans
  • postgres_fdw produces incorrect aliases for joining relations shown in EXPLAIN for some join pushdown queries
  • low-priority issue; let's leave this for v10

Resolved Issues

resolved after 9.6rc1

• Scan of 2PC state files is broken at recovery
  • Fixed in ec253de1fd2e6002122de80815ac5b963af8277c

resolved after 9.6beta4

• Add SQL-accessible functions for inspecting index AM properties. (ed0097e)

• Fix assorted bugs in contrib/bloom (d6c9e05)

• What should the default max_parallel_degree be?
  • We chose to disable it by default for 9.6; done in f85b1a84152f7bf019fd7a2c5eede97867dcddbb

• unstable regression test for speculative-insertion-with-TOAST bug
  • initial indications are that 9595383bc6fc24d25970374e2eddd5ce6f977f9e was sufficient

• parallel query error handling fails in non-English locales
  • fixed by e796d0abab0f9508326c0aa721fe1475548fbe4e

resolved after 9.6beta3

• citext 1.1 to 1.2 upgrade script broken
  • fixed in fe5e3fce798dccf3f298b65c5d9a132e9646712a

• pg_dump broken for extensions including sequences
  • owner: Stephen Frost notified
  • patch sent for review
  • Fixed in f9e439b1ca81e3305b677d93c67299549625370c

• fix signature of pg_replication_origin_xact_reset()?
  • It's already wrong in 9.5, but that seems like a bad reason for not fixing it now
  • fixed in dd5eb805d5e5384963f09c9986845a544ef41810

• heap_lock_tuple can panic the server
  • original commit: eca0f1d (principal author: Masahiko Sawada; owner: Andres Freund notified)
  • fixed in e7caacf733f3ee77a555aa715ab6fbf4434e6b52

• crash in ts_delete
  • it looks like this example also exposes an independent bug in array_to_tsvector, which is also new in 9.6.
  • fixed in c50d192ce33c10fa06411306f8644b4f47ce9a06 and f10eab73df2b94c860dea4a906c54e3c903f42e2

• old_snapshot_threshold allows heap:toast disagreement
  • original commit: 848ef42 (principal author: Kevin Grittner; owner: Robert Haas [1])
  • ownership transferred from Kevin Grittner to RMT and then to Robert Haas
  • patch v1
  • fixed in 3e2f3c2e423b3ae906668c186bac79522b8e3e29
  • 3e2f3c2e has caused a new regression
  • Addressed regression with 07a601eedab7c5fa4d62055fa3efacef2f38e446 and 9ee1cf04ab6bcefe03a11837b53f29ca9dc24c7a

resolved after 9.6beta2

• change definition of phrase search to not match a single word?
  • Several issues here, fixed in commits 028350f61, 3dbbd0f02, 6734a1cac

• consider whether MinMaxAggPath could be be parallel-safe
  • owner: Robert Haas notified
  • analysis and status
  • no changes required

• parallel aggregation is quite broken with polymorphic aggregates
  • Multiple issues here, fixed in commits 915b703e1, f8ace5477, bd1693e89, 59a3795c2, 19e972d55, f1993038a

• parallel workers and client encoding
  • owner: Robert Haas notified
  • fixed in 10c0558ffefcd12bf1d3dc35587eba41d1ce4571

• set consider_parallel correctly for upper rels
  • owner: Robert Haas notified
  • and then remove wholePlanParallelSafe flag
  • rhaas needs help
  • fixed in 5ce5e4a12ee7175cd3fc356d9d38307e1d715827

• crashes in RestoreSnapshot on hot standby
  • fixed in 4f9f495889d3d410195c9891b58228727b340189

• Broken handling of lwlocknames.h
  • fixed in 548af97fcec5543603c20b61fec60f8147a05b29

• fix possible confusion between subqueries and subplans
  • old thread
  • Fixed in 110a6dbdebebac9401b43a8fc223e6ec43cd4d10 and 4ea9948e58a57316330acb1701f979bc1e872f50

• primary_conninfo missing from pg_stat_wal_receiver
  • original commit: b1a9bad (principal author: Michael Paquier; owner: Alvaro Herrera notified)
  • There have been concerns about making potentially password values visible at SQL level, patch v3 adds a routine to libpqwalreceiver to return back to caller an obfuscated connection string that gets saved in shared memory. Then pg_stat_wal_receiver makes use of it.
  • CLOSE_WAIT for now due to not-quite-finished discussion about security implications

• temp_file_limit + parallel query issue
  • The temp_file_limit documentation should be updated.
  • fixed in d1f822e58597cac5000bf69b893cc236c5ef5fcb

• Bug in batch tuplesort memory CLUSTER case
  • patch available
  • original commit: 0011c00 (principal author: Peter Geoghegan; owner: Robert Haas notified)
  • fixed in 1b0fc85077aadb3f4a965aff35c136398d859624

• Non-exclusive backups on standbys use incorrect timeline number for WAL segments names
  • original commit: 7117685 (principal author: Magnus Hagander; owner: Magnus Hagander notified)
  • patch, reviewed by Michael.
  • fixed in 87d84d67bb15752c79a1c07e603126830642ac84

• \df* does not show parallel mode for functions
  • fixed in a670c24c382693c4f75e99c9292b2ed0f0d40a72

• to_tsquery segfault with negation of a stopword
  • fixed in 00f304ce2dd86f4b76606225b41e0854a3362628

• Document that vacuum can't truncate if old_snapshot_threshold bigger than 0
  • fixed in 1c15aac53f3421fd38ae145118d3204f055ba955

• dumping database privileges broken in 9.6
  • original commit: 23f34fa (principal author: Stephen Frost; owner: Stephen Frost status)
  • fixed in 47f5bb9f539a7fff089724b1cbacc31613031895

• heap_update sets xmax and infomask without logging the changes or clearing PD_ALL_VISIBLE or visibility map bits
  • volunteer owner wanted
  • This has been buggy essentially forever but the freeze map changes give it new urgency.
  • fixed in bfa2ab56bb8c512dc8613ee3ff0936568a1c8418

• heap_lock_tuple failed to clear all-frozen bit
  • fixed in eca0f1db14ac92d91d54eca8eeff2d15ccd797fa

resolved after 9.6beta1

• bloom extension does not dump and restore properly
  • original commit: 5d58999 (principal author: Stephen Frost; owner: Stephen Frost notified)
  • (Michael): Not limited to bloom but to all access methods. Some code lookup is pointing out that checkExtensionMembership() in selectDumpableAccessMethod does not consider the fact that access methods do not have ACLs.
  • (Stephen): Fixed in 562f06f, which corrected dumpAccessMethod() to check which bits are set in the dump bitfield and to only dump out the components requested.

• In pg_dump, include pg_catalog and extension ACLs: pg_dumpall got slow; query hygiene
  • original commit: 23f34fa (principal author: Stephen Frost; committer: Stephen Frost notified)
  • Points (2) and (3) constitute this open item. Point (1) is just questions.
  • 5d589993cad212f7d556d52cc1e42fe18f65b057 made improvements, but is that enough?

• Default roles: regression when dumping from 9.5; ALTER ROLE x USER pg_signal_backend is unrestricted
  • original commit: 2930078 (principal author: Stephen Frost; committer: Stephen Frost)
  • Restrictions on roles in the pg_ namespace were removed, per discussion, turning this into an intentionally allowed operation.

• Access methods lack support for COMMENT and have no psql-level meta command
  • original commit: 473b932 (principal author: Alexander Korotkov; owner: Alvaro Herrera notified)
  • fixed in 4f04b66f97f0e0265489f0fe0373ea44c9ad11bf

• pg_dump dump catalog ACLs incorrect queries against catalogs
  • original commit: 23f34fa4ba358671adab16773e79c17c92cbc870 (committer: Stephen Frost)
  • fixed in 2e8b4bf80473d0e4a4254b417424e79195a9ce6a

• DROP ACCESS METHOD IF EXISTS isn't implemented
  • fixed in 83dbde94f726f2517a79b1cea59e57452c36e734

• bloom index error with unlogged table
  • fixed in abaffa907588283f7673fc223857e6966421b8ca

• need to address freeze map review comments
  • original commit: 7087166 (principal author: Masahiko Sawada; owner: Robert Haas notified)
  • fixed in fdfaccfa798c1c9993feae1fac6e0f79d72aa7b7, 6436a853f11952495f10e62d8b52b465e119155c, aba8943082f1ccbfb19f2e7ff02ba3be0fcb6c9d, e191a6900520a28ece9393eec2fdd69f292f12c4

• Failed assertions on parallel worker shutdown
  • owner: Robert Haas notified
  • fixed in 44339b892a04e94bbb472235882dc6f7023bdc65, c6dbf1fe79287291bc260cbc06b0de419d2a198c

• should we revert "Use Foreign Key relationships to infer multi-column join selectivity" entirely?
  • original commit: 137805f (principal author: Tomas Vondra and David Rowley; owner: Tom Lane status)
  • Resolved by revert (77ba610)

• bloom documentation should be reworked
  • fixed in cfd4804b1ebed8108ef6caedc76fb6eb37bc9724

• Basic test coverage of parallelism
  • owner: Noah Misch status
  • patch available, committed as 14a254f

• rename max_parallel_degree to max_parallel_workers_per_gather
  • original commit: 3bd909b (principal author: Amit Kapila; owner: Robert Haas notified)
  • fixed in c9ce4a1c61ebf39c03885cc19fe7c32edc04a300

• Failed assertion in parallel worker (ExecInitSubPlan)
  • owner: Robert Haas notified
  • fixed in b12fd41c695b43c76b0a9a4d19ba43b05536440c, although we suspect other bugs in this area

• "snapshot too old" feature considered not ready for 9.6 by some
  • original commit: 848ef42bb8c7909c9d7baa38178d4a209906e7c1 (principal author: Kevin Grittner; owner: Kevin Grittner status)
  • Andres considering revert, Bruce considering revert, Tom considering revert
  • CREATE INDEX may need work to avoid missing rows fixed in bf9a60e
  • ANALYZE of functional index that takes longer than old_snapshot_threshold may error out and fail to update statistics Analysis suggesting that this is not a problem
  • there is a not a consensus to revert this; whatever specific defects exist should be separately tracked

• Failed assertion in postgres_fdw/deparse.c
  • original commit: e4106b2 (principal author: Shigeru Hanada; owner: Robert Haas notified)
  • proposal of patch
  • fixed in 131c7e70b4596027992a2f72bfd3765f0fff1b7c
• idle_in_transaction_session_timeout patch overlooked pg_dump
  • fixed in 8383486f108c650b187358bfe811060627c751c9

• data-corrupting bug in lazy_scan_heap()
  • original commit: a892234 (principal author: Masahiko Sawada; owner: Robert Haas status)

• add integrity-checking functions to pg_visibility extension
  • fixed in e472ce9624e0f2083c8fd25ea1acb081be908f8f

• what to do with parallel worker errcontext()?
  • Should we have it at all? Maybe only for debugging? (If so, controlled how?) Unless we remove it entirely, the misbehavior shown in above message needs to be fixed.
  • analysis and explanation by rhaas
  • fixed in 292794f82b4ebde33ec7f2a572ddd1dedba8ce37

• provide a way to force a rebuild of the visibility map

• ERROR: ORDER/GROUP BY expression not found in targetlist
  • git bisect is pointing out to 04ae11f6 as the culprit
  • six separate issues have been raised on this thread
    • #1 fixed in 54f5c5150fa05d7ad15f8406debd5a2b394885b5, 598aa194af2fb7f294ae4b029494a134a44be333
    • #2 fixed in 8c1d9d56e9a00680a035b8b333a98ea16b121eb7, 103512cee95b5bd0feb83c225eeff61c58874413
    • #3 fixed in 75be66464cb1bffa1e5757907b9a04ad5afc7859
    • separate open items added for #4, #5, #6

• flush control causes performance regressions
  • owner: Andres Freund notified
  • parallel copy 9x slower
  • pgbench 45% slower
  • Fixed in 4bc0f165cb4fbd660648c0153485b3d6f55d80ea

• commit ac1d794 hurts performance
  • original commit: ac1d7945f866b1928c2554c0f80fd52d7f977772 (principal author: Rajeev Rastogi; owner: Robert Haas notified)
  • hopefully resolved by Andres, per [2]
  • no reply from original poster as of 2016-03-25
  • Mithun CY re-reported this problem 2016-05-06 - apparently, it's not fixed yet
  • possibly fixed after all?

resolved before 9.6beta1

• fd.c: flush data problems on osx
  • original commit: 428b1d6b29ca599c5700d4bc4f4ce4c5880369bf (principal author: Fabien Coelho; committer: Andres Freund notified)

• checkpoint fails with could not fsync file "...": No such file or directory
  • original commit: 72a98a6395 (committer: Andres Freund)
  • fixed in a712487087c73eee880ff1a7c50528cbab2f1b90

• Logical decoding timeline following (commits were reverted, feature no longer in tree)

• crash/assertion failure in postgres_fdw with three-table joins, fixed by 3151f16e.

• Some messages of pg_rewind --debug not getting translated, fixed by cad3edef.

• incorrect doc for pgbench log format. The part about latency limit description in the log format should be backported to 9.5.
  • fixed by d797bf7 and 7f0a2c8

• Recovery test failure for recovery_min_apply_delay
  • original commit: 49148645f7f30f461b09618b5342b37425f33b22 (principal author: Michael Paquier; committer: Alvaro Herrera notified)

• Parallel aggregates can't be declared safe
  • original commit: e06a38965b3bcdaa881e7e06892d4d8ab6c2c980 (principal author: David Rowley; committer: Robert Haas)
  • fixed by 41ea0c23

• Parallel Aggregate costs don't consider combine/serial/deserial funcs
  • original commits e06a38965b3bcdaa881e7e06892d4d8ab6c2c980, 5fe5a2cee91117673e04617aeb1a38e305dcd783 (principal author: David Rowley; committer: Robert Haas notified)
  • The above link contains a patch to fix this.

• Logical WAL Messages are missing database and origin filtering
  • original commit: 3fe3511d05127cc024b221040db2eeb352e7d716 (pricipal author: Petr Jelinek; committer: Simon Riggs)
  • fix available in the same thread
  • fixed by be65ed

• commit 6150a1b0 hurts performance
  • original commit: 6150a1b08a9fe7ead2b25240be46dddeae9d98e1 (principal author: Andres Freund; committer: Robert Haas notified)
  • testing after the atomic pin/unpin patch and other recent work seems to show that we've recovered the performance

• sporadic regression tests failures in syncrep suite
  • Per report from hamster here
  • fixed by 36c1c91

• Odd system-column handling in postgres_fdw join pushdown patch
  • original commit: aa09cd242fa7e3a694a31f8aed521e80d1e626a4 (principal author: Ashutosh Bapat; committer: Robert Haas notified)

• Crash on CREATE EXTENSION
  • original commit: 6c268df1276e9dd73e4d (principal author: Stephen Frost; committer: Stephen Frost)
  • Fixed in 99f2f3c19ae7d6aa2950a9bdb549217c5a60d941

• Update of backup documentation (per Brussels Developer Meeting, reorder options to focus on "safe and easy" rather than "difficult and risky")
  • 7117685461af50f50c03f43e6a622284c8d54694 makes that documentation incomplete since it wasn't updated, due to the restructure being planned. If restructure not done, then at least additional notes based on this commit should be added.
  • original commit: 7117685461af50f50c03f43e6a622284c8d54694 (principal author: Magnus Hagander; committer: Magnus Hagander notified)

• Parallel full and right join compute wrong result
  • original commit: 45be99f8cd5d606086e0a458c9c72910ba8a613d (principal author: Robert Haas; committer: Robert Haas)
  • fixed in 9c75e1a36b6b2f3ad9f76ae661f42586c92c6f7c

• Can't cancel DML pushdown queries
  • original commit: 0bf3ae88af330496517722e391e7c975e6bad219 (principal author: Etsuro Fujita; committer: Robert Haas notified)
  • Patch available here making more queries than those DMLs interruptible by client.
  • fixed in f039eaac7131ef2a4cf63a10cf98486f8bcd09d2

• pg_stat_activity related segfaults
  • original commit: 53be0b1add7064ca5db3cd884302dfc3268d884e (principal author: Amit Kapila; committer: Robert Haas)
  • fixed in c4a586c4860477ddae6d4f9cef88486f0e37c37e

• LWLocks API refactoring breaks extensions, there are no documented way, how to fix it.
  • suggested way to fix it
  • Pavel confirmed that the suggestion worked

• assorted checkpoint_flush_after issues
  • fixed in 8f91d87d43d021db92c6edd966a4bb8c3a81ae39

• asind(0.5) gives wrong answer on $mystery_Debian_host
  • fixed in 82311bcdd76904b2cee7567e14e9fb0cf6c6178c

• EXPLAIN VERBOSE with parallel Aggregate displays bogus output
  • original commit: e06a389 (principal author: David Rowley; committer: Robert Haas notified)
  • fixed in 59eb55127906b943ff155240eebc161df8edb62f

• configurable writeback causes or reveals mysterious invalidation-related failures
  • original commit: 428b1d6b29ca599c5700d4bc4f4ce4c5880369bf (principal author: Fabien Coelho; committer: Andres Freund notified)
  • fixed in 72a98a639574d2e25ed94652848555900c81a799

• add_partial_path can recycle Paths that are still referenced
  • fixed in c45bf5751b6338488bd79ce777210285531da373

• system_views.sql wrongly removes PARALLEL SAFE markings
  • original commit: 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b (principal author: Robert Haas; committer: Robert Haas)
  • fixed in 9888b34fdb169c1f0982ad700fc6d43e8b7aec14

• psql's \ev and \sv missing support for view options
  • These commands lack proper support for security barrier views or WITH CHECK OPTION.
  • Fixed in 9b66aa006f81b2705337ca223daeeabf4db6453a

• Segfault with simple query in planner, related to use of foreign keys in join estimates
  • Fixed in 68d704edbfe14fd8d21545b244c3f6824e0fbb8a

• old_snapshot_threshold's interaction with hash index
  • original commit: 848ef42bb8c7909c9d7baa38178d4a209906e7c1 (principal author: Kevin Grittner; committer: Kevin Grittner)
  • Patch to fix it
  • Fixed in 2cc41acd8fa3ebb8f0501c6102a253fb7053cf46

• should we revert the changes to restrict assuming a pre-created system role?
  • Fixed in a89505fd21da337b81172871d8f65d9a4fa22a8b

• "snapshot too old" commit causes performance regression at high concurrency, even when disabled
  • original commit: 848ef42bb8c7909c9d7baa38178d4a209906e7c1 (principal author: Kevin Grittner; committer: Kevin Grittner)
  • hopefully resolved by commit: 2201d801b03c2d1b0bce4d6580b718dc34d38b3e but testing on a big machine needed to confirm
  • some testing by rhaas on a big machine suggests that it is fixed
  • but some testing from Andres says maybe not
  • new commit 11e178d0dc4bc2328ae4759090b3c48b07023fab is another attempt to fix this
  • benchmarks by Tomas Vondra show slight gain (within the noise) with the patch (disabled) versus reverted

• ALTER TABLE lock downgrades have broken pg_upgrade
  • Fixed in 1a2c17f8e221b0c3c63cb2b1be1f862444f62516

• GIN data corruption bug(s) in 9.6devel
  • original commit: e95680832854cf300e64c10de9cc2f586df558e8 (principal author: Jeff Janes; committer: Teodor Sigaev notified)
  • A pre-existing theoretical bug, which becomes quite reachable in 9.6 due to FSM changes in commit e95680832854cf300e64c10d
  • fixed for 9.6 in e2c79e14d998cd31f860854bc9210b37b457bb01; back-branch fixes pending

Older Bugs

• attempted to delete invisible tuple and unexpected chunk number in crash recovery stress test
  • Symptoms suggest a data corruption bug.
  • Test procedure, Bisection (not reproducible)
  • original suspected commit: 48354581a49c30f5757c203415aa8412d85b0f70 (principal author: Alexander Korotkov; committer: Andres Freund)
  • now appears to be a longstanding issue

• Suspicious behaviour on applying XLOG_HEAP2_VISIBLE (9.2 and later)
  • c6ff84b06a68b71719aa1aaa5f6704d8db1b51f8 fixes this, sorta by accident, in master only

• Truncation of new relfilenodes broken with wal_level = minimal (9.4 and later)
• OOM in libpq and infinite loop with getCopyStart() (9.1 and later)
  • Patch available using some emergency message structure. Need review, and surely more discussion
• Logic to skip checkpoint is broken (9.1~9.5 if wal_level >= hot_standby, 9.6 if wal_level >= replica)
  • Patch available for review, preferred approach with XLogInsertExtended() (Michael, Robert, Simon at least)
• pgwin32_is_service not checking if SECURITY_SERVICE_SID is disabled
  • Patch available for review.
• Deadlock issue with pg_restore in WIN32 with urgent exit code path
  • Patch available for review
• pg_xlogdump fails to handle WAL file with multi-page XLP_FIRST_IS_CONTRECORD data
  • Patch available for review
• pg_receivexlog missing fsync calls of data directory
  • Patch available for review
• pg_basebackup and pg_dump should fsync dumped data and make it persistent.

Non-bugs

• Speedup 2-phase commit on slaves (reason for non-bug status)
  • Replication lag
  • original commit: 78b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e (principal author: Stas Kelvich; committer: Simon Riggs) (speedup on master node)
  • current patch: Link
  • Failure at replay because of locks taken at PREPARE phase
  • Too late to be considered for 9.6.

• parallel query vs extensions
  • Need to update extensions with parallel-safety markings
  • This an opportunity for further optimization, not a defect.

• does GIN's accumulatedMemory counter need to be signed?
  • original commit: 30bb26b5 (principal author: Robert Abraham; owner: Teodor Sigaev notified)
  • No; no subtraction will make it wrap. One could add an assertion about that.

• add max_parallel_workers GUC
  • this would be a system-wide limit, to keep parallel workers from consuming more of max_worker_processes than desired
  • there's support for slipping this into 9.6, but rhaas may not have time, and it's not a stop-ship issue