84Beta2Changes
List of significant fixes and changes between 8.4 Beta 1 and 8.4 Beta 2
2009-05-14 16:31 heikki
* contrib/pg_standby/pg_standby.c, doc/src/sgml/backup.sgml, doc/src/sgml/pgstandby.sgml, src/backend/access/transam/xlog.c: Add recovery_end_command option to recovery.conf. recovery_end_command is run at the end of archive recovery, providing a chance to do external cleanup. Modify pg_standby so that it no longer removes the trigger file, that is to be done using the recovery_end_command now.
Provide a "smart" failover mode in pg_standby, where we don't fail over immediately, but only after recovering all unapplied WAL from the archive. That gives you zero data loss assuming all WAL was archived before failover, which is what most users of pg_standby actually want.
recovery_end_command by Simon Riggs, pg_standby changes by Fujii Masao and myself.
2009-05-13 16:27 tgl
* src/: backend/access/transam/xact.c, backend/utils/adt/xml.c, include/utils/xml.h: Rewrite xml.c's memory management (yet again). Give up on the idea of redirecting libxml's allocations into a Postgres context. Instead, just let it use malloc directly, and add PG_TRY blocks as needed to be sure we release libxml data structures in error recovery code paths. This is ugly but seems much more likely to play nicely with third-party uses of libxml, as seen in recent trouble reports about using Perl XML facilities in pl/perl and bug #4774 about contrib/xml2.
I left the code for allocation redirection in place, but it's only built/used if you #define USE_LIBXMLCONTEXT. This is because I found it useful to corral libxml's allocations in a palloc context when hunting for libxml memory leaks, and we're surely going to have more of those in the future with this type of approach. But we don't want it turned on in a normal build because it breaks exactly what we need to fix.
I have not re-indented most of the code sections that are now wrapped by PG_TRY(); that's for ease of review. pg_indent will fix it.
This is a pre-existing bug in 8.3, but I don't dare back-patch this change until it's gotten a reasonable amount of field testing.
2009-05-12 12:43 tgl
* src/: backend/access/heap/heapam.c, backend/commands/lockcmds.c, include/access/heapam.h: Fix LOCK TABLE to eliminate the race condition that could make it give weird errors when tables are concurrently dropped. To do this we must take lock on each relation before we check its privileges. The old code was trying to do that the other way around, which is a bit pointless when there are lots of other commands that lock relations before checking privileges. I did keep it checking each relation's privilege before locking the next relation, which is a detail that ALTER TABLE isn't too picky about.
2009-05-11 23:11 tgl
* src/: backend/catalog/pg_inherits.c, backend/commands/lockcmds.c, backend/commands/tablecmds.c, backend/optimizer/prep/prepunion.c, backend/parser/parse_coerce.c, include/catalog/pg_inherits.h, include/catalog/pg_inherits_fn.h: Modify find_inheritance_children() and find_all_inheritors() to add the ability to lock relations as they scan pg_inherits, and to ignore any relations that have disappeared by the time we get lock on them. This makes uses of these functions safe against concurrent DROP operations on child tables: we will effectively ignore any just-dropped child, rather than possibly throwing an error as in recent bug report from Thomas Johansson (and similar past complaints). The behavior should not change otherwise, since the code was acquiring those same locks anyway, just a little bit later.
An exception is LockTableCommand(), which is still behaving unsafely; but that seems to require some more discussion before we change it.
2009-05-11 04:06 mha
* src/backend/libpq/be-secure.c: Support SSL certificate chains in the server certificate file.
Andrew Gierth
2009-05-09 22:51 tgl
* src/bin/pg_dump/pg_dumpall.c: Adjust pg_dumpall so that it emits ENCODING, LC_COLLATE, and LC_CTYPE options in its CREATE DATABASE commands only for databases that have settings different from the installation defaults. This is a low-tech method of avoiding unnecessary platform dependencies in dump files. Eventually we ought to have a platform-independent way of specifying LC_COLLATE and LC_CTYPE, but that's not going to happen for 8.4, and this patch at least avoids the issue for people who aren't setting up per-database locales. ENCODING doesn't have the platform dependency problem, but it seems consistent to make it act the same as the locale settings.
2009-05-09 18:51 tgl
* src/: backend/optimizer/path/costsize.c, backend/optimizer/plan/createplan.c, backend/optimizer/util/restrictinfo.c, include/optimizer/restrictinfo.h: Fix cost_nestloop and cost_hashjoin to model the behavior of semi and anti joins a bit better, ie, understand the differing cost functions for matched and unmatched outer tuples. There is more that could be done in cost_hashjoin but this already helps a great deal. Per discussions with Robert Haas.
2009-05-07 18:01 tgl
* contrib/pgbench/pgbench.c, doc/src/sgml/pgbench.sgml: Change pgbench to use the table names pgbench_accounts, pgbench_branches, pgbench_history, and pgbench_tellers, rather than just accounts, branches, history, and tellers. This is to prevent accidental conflicts with real application tables, as has been reported to happen at least once. Also remove the automatic "SET search_path = public" that it did at startup, as this seems to restrict testing flexibility without actually buying much. Per proposal by Joshua Drake and ensuing discussion.
Joshua Drake and Tom Lane
2009-05-06 12:15 tgl
* doc/src/sgml/charset.sgml, doc/src/sgml/manage-ag.sgml, doc/src/sgml/ref/create_database.sgml, src/backend/commands/dbcommands.c, src/test/mb/README, src/test/mb/mbregress.sh: Modify CREATE DATABASE to enforce that the source database's encoding setting must be used for the new database, except when copying from template0. This is the same rule that we now enforce for locale settings, and it has the same motivation: databases other than template0 might contain data that would be invalid according to a different setting. This represents another step in a continuing process of locking down ways in which encoding violations could occur inside the backend. Per discussion of a few days ago.
In passing, fix pre-existing breakage of mbregress.sh, and fix up a couple of ereport() calls in dbcommands.c that failed to specify sqlstate codes.
2009-05-05 16:06 tgl
* src/backend/storage/ipc/ipc.c: Install an atexit(2) callback that ensures that proc_exit's cleanup processing will still be performed if something in a backend process calls exit() directly, instead of going through proc_exit() as we prefer. This is a second response to the issue that we might load third-party code that doesn't know it should not call exit(). Such a call will now cause a reasonably graceful backend shutdown, if possible. (Of course, if the reason for the exit() call is out-of-memory or some such, we might not be able to recover, but at least we will try.)
2009-05-05 15:59 tgl
* src/: backend/postmaster/postmaster.c, backend/storage/ipc/ipci.c, backend/storage/ipc/pmsignal.c, backend/storage/lmgr/proc.c, backend/utils/init/globals.c, include/miscadmin.h, include/postmaster/postmaster.h, include/storage/pmsignal.h: Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash.
The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway.
This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs.
Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 14:32 petere
* doc/src/sgml/syntax.sgml, src/backend/parser/scan.l, src/test/regress/expected/strings.out, src/test/regress/sql/strings.sql: Disable the use of Unicode escapes in string constants (U&) when standard_conforming_strings is not on, for security reasons.
2009-05-02 21:17 ishii
* src/backend/utils/mb/Unicode/: euc_kr_to_utf8.map, johab_to_utf8.map, uhc_to_utf8.map, utf8_to_euc_kr.map, utf8_to_johab.map, utf8_to_uhc.map: Update UTF-8 <--> EUC_KR, JOHAB, UHC mappings. Patch contributed by Chuck McDevitt
2009-05-02 13:27 tgl
* doc/src/sgml/plpgsql.sgml, src/pl/plpgsql/src/pl_exec.c: Fix plpgsql's EXIT so that an EXIT without a label only matches a loop, never a BEGIN block. This is required for Oracle compatibility and is also plainly stated to be the behavior by our original documentation (up until 8.1, in which the docs were adjusted to match the code's behavior; but actually the old docs said the correct thing and the code was wrong).
Not back-patched because this introduces an incompatibility that could break working applications. Requires release note.
2009-04-24 05:43 mha
* doc/src/sgml/libpq.sgml, doc/src/sgml/runtime.sgml, src/interfaces/libpq/fe-connect.c, src/interfaces/libpq/fe-secure.c, src/interfaces/libpq/libpq-int.h: Remove sslverify parameter again, replacing it with two new sslmode values: "verify-ca" and "verify-full".
Since "prefer" remains the default, this will make certificate validation off by default, which should lead to less upgrade issues.
2009-04-22 20:23 tgl
* doc/src/sgml/config.sgml, doc/src/sgml/ref/prepare_transaction.sgml, src/backend/access/transam/twophase.c, src/backend/access/transam/varsup.c, src/backend/utils/misc/guc.c, src/backend/utils/misc/postgresql.conf.sample, src/test/regress/expected/prepared_xacts.out, src/test/regress/expected/prepared_xacts_1.out, src/test/regress/pg_regress.c, src/test/regress/sql/prepared_xacts.sql: Change the default value of max_prepared_transactions to zero, and add documentation warnings against setting it nonzero unless active use of prepared transactions is intended and a suitable transaction manager has been installed. This should help to prevent the type of scenario we've seen several times now where a prepared transaction is forgotten and eventually causes severe maintenance problems (or even anti-wraparound shutdown).
The only real reason we had the default be nonzero in the first place was to support regression testing of the feature. To still be able to do that, tweak pg_regress to force a nonzero value during "make check". Since we cannot force a nonzero value in "make installcheck", add a variant regression test "expected" file that shows the results that will be obtained when max_prepared_transactions is zero.
Also, extend the HINT messages for transaction wraparound warnings to mention the possibility that old prepared transactions are causing the problem.
All per today's discussion.
2009-04-21 11:49 momjian
* doc/src/sgml/ref/psql-ref.sgml, src/bin/psql/command.c, src/bin/psql/describe.c, src/bin/psql/describe.h, src/bin/psql/help.c, src/test/regress/expected/polymorphism.out: Add agg/normal/trigger/window flags for psql \df and in \df output.
David Fetter
2009-04-19 17:08 tgl
* src/backend/: commands/copy.c, parser/scan.l: Fix de-escaping checks so that we will reject \000 as well as other invalidly encoded sequences. Per discussion of a couple of days ago.
2009-04-19 14:52 tgl
* doc/src/sgml/plpgsql.sgml, src/pl/plpgsql/src/gram.y, src/pl/plpgsql/src/pl_comp.c, src/pl/plpgsql/src/pl_funcs.c, src/pl/plpgsql/src/plpgsql.h, src/pl/plpgsql/src/scan.l, src/test/regress/expected/plpgsql.out, src/test/regress/sql/plpgsql.sql: Revise plpgsql's scanner to process comments and string literals in a way more nearly matching the core SQL scanner. The user-visible effects are:
* Block comments (slash-star comments) now nest, as per SQL spec.
* In standard_conforming_strings mode, backslash as the last character of a non-E string literal is now correctly taken as an ordinary character; formerly it was misinterpreted as escaping the ending quote. (Since the string also had to pass through the core scanner, this invariably led to syntax errors.)
* Formerly, backslashes in the format string of RAISE were always treated as quoting the next character, regardless of mode. Now, they are ordinary characters with standard_conforming_strings on, while with it off, they introduce the same set of escapes as in the core SQL scanner. Also, escape_string_warning is now effective for RAISE format strings. These changes make RAISE format strings work just like any other string literal.
This is implemented by copying and pasting a lot of logic from the core scanner. It would be a good idea to look into getting rid of plpgsql's scanner entirely in favor of using the core scanner. However, that involves more change than I can justify making during beta --- in particular, the core scanner would have to become re-entrant.
In passing, remove the kluge that made the plpgsql scanner emit T_FUNCTION or T_TRIGGER as a made-up first token. That presumably had some value once upon a time, but now it's just useless complication for both the scanner and the grammar.
2009-04-12 17:02 adunstan
* src/bin/pg_dump/pg_backup_archiver.c: Fix logic to detect conflicts or blocks involving exclusive locks in parallel restore items. If a currently running item needs an exclusive lock on any item that the candidate items needs any sort of lock on, or vice versa, then the candidate item is not allowed to run now, and must wait till later.
2009-04-11 16:23 tgl
* src/bin/pg_dump/pg_dumpall.c: Fix pg_dumpall so that when --clean is specified, it drops roles and tablespaces in an order that has some chance of working. Per a complaint from Kevin Bailey.
This is a pre-existing bug, but given the lack of prior complaints I'm not sure it's worth back-patching. In most cases failure of the DROP commands wouldn't be that important anyway.
In passing, fix syntax errors in dumpCreateDB()'s queries for old servers; these were apparently introduced in recent binary_upgrade patch.
