https://wiki.postgresql.org/api.php?action=feedcontributions&user=Schmiddy&feedformat=atomPostgreSQL wiki - User contributions [en]2024-03-19T10:00:55ZUser contributionsMediaWiki 1.35.13https://wiki.postgresql.org/index.php?title=What%27s_new_in_PostgreSQL_9.3&diff=20786What's new in PostgreSQL 9.32013-09-18T14:49:14Z<p>Schmiddy: grammar</p>
<hr />
<div>This page contains an overview of PostgreSQL Version 9.3's features, including descriptions, testing and usage information, and links to blog posts containing further imformation. See also the [http://www.postgresql.org/docs/9.3/static/release-9-3.html Release Notes] and [[PostgreSQL 9.3 Open Items]].<br />
<br />
== Configuration directive 'include_dir' ==<br />
<br />
In addition to including separate configuration files via the 'include' directive, postgresql.conf now also provides the 'include_dir' directive which reads all files ending in ".conf" in the specified directory or directories.<br />
<br />
Directories can be specified either as an absolute path or relative from the location of the main configuration file. Directories will be read in the order they occur, while files will be read sorted by C locale rules. It is possible for included files to contain their own 'include_dir' directives. <br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/config-setting.html#CONFIG-INCLUDES Documentation]<br />
<br />
== COPY FREEZE for more efficient bulk loading ==<br />
<br />
To improve initial bulk loading of tables, a ''FREEZE'' parameter has been added to the COPY command to enable data to be copied with rows already frozen. See the documentation for usage and caveats.<br />
<br />
'''Links'''<br />
* [http://www.postgresql.org/docs/9.3/static/sql-copy.html Documentation] - see the ''FREEZE'' parameter<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-copy-freeze/ Postgres 9.3 feature highlight: COPY FREEZE]<br />
<br />
== Custom Background Workers ==<br />
<br />
This functionality enables modules to register themselves as "background worker processes", effectively operating as customised server processes. This is a powerful new feature with a wide variety of possible use cases, such as monitoring server activity, performing tasks at pre-defined intervals, customised logging etc.<br />
<br />
Background worker processes can attach to PostgreSQL's shared memory area and to connect to databases internally; by linking to libpq they can also connect to the server in the same way as a regular client application. Background worker processes are written in C, and as server processes they have unrestricted access to all data and can potentially impact other server processes, meaning they represent a potential security / stability risk. Consequently background worker processes should be developed and deployed with appropriate caution.<br />
<br />
Providing an example would go beyond the scope of this article; please refer to the blogs linked below, which provide annotated sample code. The PostgreSQL source also contains a sample background worker process in contrib/worker_spi.<br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/bgworker.html Documentation]<br />
* [http://www.depesz.com/2012/12/07/waiting-for-9-3-background-worker-processes/ Background worker processes] <br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-handling-signals-with-custom-bgworkers/ Postgres 9.3 feature highlight: handling signals with custom bgworkers] <br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-custom-background-workers/ Custom background workers] <br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-hello-world-with-custom-bgworkers/ "Hello World" with custom bgworkers]<br />
* [http://sql-info.de/postgresql/notes/custom-background-worker-bgw-practical-example.html Custom Background Workers - a practical example]<br />
<br />
== Data Checksums ==<br />
<br />
It is now possible for PostgreSQL to checksum data pages and report corruption. This is a cluster-wide setting and cannot be applied to individual databases or objects. Also be aware that this facility may incur a noticeable performance penalty. This option must be enabled during initdb and cannot be changed (although there is a new GUC parameter "[http://www.postgresql.org/docs/9.3/static/runtime-config-developer.html#GUC-IGNORE-CHECKSUM-FAILURE ignore_checksum_failure]" which will force PostgreSQL to continue processing a transaction even if corruption is detected). <br />
<br />
'''Links'''<br />
<br />
* Documentation<br />
** [http://www.postgresql.org/docs/9.3/static/app-initdb.html#APP-INITDB-DATA-CHECKSUMS initdb -k/--data-checksums]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-data-checksums/ Postgres 9.3 feature highlight: Data Checksums]<br />
<br />
== JSON: Additional functionality ==<br />
<br />
The [http://www.postgresql.org/docs/9.2/static/functions-json.html JSON datatype] and [http://www.postgresql.org/docs/9.2/static/functions-json.html two supporting functions] for converting rows and arrays were introduced in PostgreSQL 9.2. With PostgreSQL 9.3, dedicated JSON operators have been introduced and the number of functions expanded to 12, including JSON parsing support. The JSON parser has exposed for use by other modules such as extensions as an API.<br />
<br />
Additionally, the [http://www.postgresql.org/docs/9.3/static/hstore.html hstore] extension has gained two JSON-related functions, ''hstore_to_json(hstore)'' and ''hstore_to_json_loose(hstore)''. The former is used when an hstore value is cast to json.<br />
<br />
'''Links'''<br />
<br />
* Documentation<br />
** [http://www.postgresql.org/docs/9.3/static/datatype-json.html Documentation: JSON Datatype]<br />
** [http://www.postgresql.org/docs/9.3/static/functions-json.html Documentation: JSON Functions and Operators]<br />
* [http://www.depesz.com/2013/03/11/waiting-for-9-3-json-generation-improvements/ Waiting for 9.3 – JSON generation improvements] <br />
* [http://www.depesz.com/2013/03/30/waiting-for-9-3-add-new-json-processing-functions-and-parser-api/ Waiting for 9.3 – Add new JSON processing functions and parser API]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-json-data-generation/ Postgres 9.3 feature highlight: JSON data generation] <br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-json-operators/ Postgres 9.3 feature highlight: JSON operators] <br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-json-parsing-functions/ Postgres 9.3 feature highlight: JSON parsing functions]<br />
<br />
== LATERAL JOIN ==<br />
<br />
Put simply, a <tt>LATERAL JOIN</tt> enables a subquery in the <tt>FROM</tt> part of a clause to reference columns from preceding items in the FROM list.<br />
<br />
The following is a self-contained (if quite pointless) example of the kind of clause it is sometimes useful to be able to write:<br />
<br />
SELECT base.nr,<br />
multiples.multiple<br />
FROM (SELECT generate_series(1,10) AS nr) base<br />
JOIN (SELECT generate_series(1,10) AS b_nr, base.nr * 2 AS multiple) multiples<br />
ON multiples.b_nr = base.nr<br />
<br />
but which produces an error message like the following:<br />
<br />
<pre> LINE 4: JOIN (SELECT generate_series(1,10) AS base, base.nr * 2 A...<br />
^<br />
HINT: There is an entry for table "base", but it cannot be referenced from this part of the query.</pre><br />
<br />
Using <tt>LATERAL JOIN</tt>, it's now possible for the second subquery to reference a value from the first:<br />
<br />
SELECT base.nr,<br />
multiples.multiple<br />
FROM (SELECT generate_series(1,10) AS nr) base,<br />
LATERAL (<br />
SELECT multiples.multiple FROM<br />
( SELECT generate_series(1,10) AS b_nr, base.nr * 2 AS multiple ) multiples<br />
WHERE multiples.b_nr = base.nr<br />
) multiples;<br />
<br />
Note that function calls can now directly reference columns from preceding <tt>FROM</tt> items, even without the <tt>LATERAL</tt> keyword. Example:<br />
<br />
CREATE FUNCTION multiply(INT, INT)<br />
RETURNS INT<br />
LANGUAGE SQL<br />
AS<br />
$$<br />
SELECT $1 * $2;<br />
$$<br />
<br />
Query with function call in the <tt>FROM</tt> list:<br />
<br />
SELECT base.nr,<br />
multiple<br />
FROM (SELECT generate_series(1,10) AS nr) base,<br />
multiply(base.nr, 2) AS multiple<br />
<br />
In previous versions, this query would generate an error like this:<br />
<br />
ERROR: function expression in FROM cannot refer to other relations of same query level<br />
LINE 4: multiply(base.nr, 2) AS multiple<br />
<br />
See the articles linked below for some more realistic examples.<br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/sql-select.html Documentation: SELECT] ''(see section <tt>LATERAL</tt>)''<br />
* [http://www.depesz.com/2012/08/19/waiting-for-9-3-implement-sql-standard-lateral-subqueries/ Waiting for 9.3: Implement SQL standard lateral subqueries]<br />
* [http://www.postgresonline.com/journal/archives/284-PostgreSQL-9.3-Lateral-Part-1-Use-with-HStore.html PostgreSQL 9.3 Lateral Part 1: Use with HStore] <br />
* [http://www.postgresonline.com/journal/archives/285-PostgreSQL-9.3-Lateral-Part2-The-Lateral-Left-Join.html PostgreSQL 9.3 Lateral Part 2: The Lateral Left Join]<br />
<br />
== Parallel pg_dump for faster backups ==<br />
<br />
The new ''-j '''njobs''''' (''--jobs=''''njobs'''''') option enables pg_dump to dump '''njobs''' tables simultaneously, reducing the time it takes to dump a database. Example:<br />
<br />
pg_dump -U postgres -j4 -Fd -f /tmp/mydb-dump mydb<br />
<br />
This dumps the contents of database "mydb" to the directory "/tmp/mydb-dump" using four simultaneous connections.<br />
<br />
Caveats:<br />
* Parallel dumps can only be in directory format<br />
* Parellel dumps will place more load on the database, although total dump time should be shorter<br />
* pg_dump will open njobs + 1 connections to the database, so max_connections should be set appropriately<br />
* Requesting exclusive locks on database objects while running a parallel dump could cause the dump to fail<br />
* Parallel dumps from pre-9.2 servers need special attention<br />
<br />
An ad-hoc test of this feature on a 4.5GB database (which compresses to around 370MB as a dump) with different values of ''-j '' produced following timings:<br />
<br />
* (''no -j''): 1m3s<br />
* -j2: 0m28s<br />
* -j3: 0m24s<br />
* -j4: 0m24s<br />
* -j5: 0m25s<br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/app-pgdump.html pg_dump documentation]<br />
* [http://www.depesz.com/2013/03/26/2646/ Waiting for 9.3 – Add parallel pg_dump option]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-parallel-pg_dump/ Postgres 9.3 feature highlight: parallel pg_dump]<br />
<br />
== 'pg_isready' server monitoring tool ==<br />
<br />
pg_isready is a wrapper for PQping created as a standard client application. It accepts a libpq-style connection string and returns one of four exit statuses:<br />
<br />
* 0: server is accepting connections normally<br />
* 1: server is rejecting connections (for example during startup)<br />
* 2: server did not response to the connection attempt<br />
* 3: no connection attempt was made (e.g. due to invalid connection parameters)<br />
<br />
Example usage:<br />
<br />
barwick@localhost:~$ pg_isready<br />
/tmp:5432 - accepting connections<br />
barwick@localhost:~$ pg_isready --quiet && echo "OK"<br />
OK<br />
barwick@localhost:~$ pg_isready -p5431 -h localhost<br />
localhost:5431 - accepting connections<br />
barwick@localhost:~$ pg_isready -h example.com<br />
example.com:5432 - no response<br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/app-pg-isready.html Documentation]<br />
* [http://www.depesz.com/2013/01/26/waiting-for-9-3-pg_isready/ pg_isready]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-server-monitoring-with-pg_isready/ Server monitoring with pg_isready]<br />
<br />
== Switch to Posix shared memory and mmap() ==<br />
<br />
In 9.3, PostgreSQL has switched from using SysV shared memory to using Posix shared memory and mmap for memory management. This allows easier installation and configuration of PostgreSQL, and means that except in unusual cases, system parameters such as SHMMAX and SHMALL no longer need to be adjusted. We need users to rigorously test and ensure that no memory management issues have been introduced by the change. <br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/kernel-resources.html#SYSVIPC Documentation]<br />
<br />
== Trigger Features ==<br />
=== Event Triggers ===<br />
<br />
Triggers can now be defined on DDL events (CREATE, ALTER, DROP).<br />
<br />
'''Links'''<br />
* Documentation:<br />
** [http://www.postgresql.org/docs/9.3/interactive/sql-createeventtrigger.html CREATE EVENT TRIGGER]<br />
** [http://www.postgresql.org/docs/9.3/interactive/event-trigger-matrix.html Event Trigger Firing Matrix]<br />
** [http://www.postgresql.org/docs/9.3/interactive/plpgsql-trigger.html#PLPGSQL-EVENT-TRIGGER Triggers on events]<br />
<br />
* [http://www.depesz.com/2012/07/29/waiting-for-9-3-event-triggers/ Waiting for 9.3 – Event triggers]<br />
<br />
== VIEW Features ==<br />
=== Materialized Views ===<br />
<br />
Materialized views are a special kind of view which cache the view's output as a physical table, rather than executing the underlying query on every access. Conceptually they are similar to "CREATE TABLE AS", but store the view definition so it can be easily refreshed.<br />
<br />
Note that materialized views cannot be auto-refreshed; refreshes are not incremental; and the base table cannot be manipulated. They will however be automatically populated by pg_restore (more precisely, pg_dump includes a "REFRESH MATERIALIZED VIEW" statement).<br />
<br />
'''Contrived example'''<br />
<br />
Create and populate a table with some arbitrary data:<br />
<br />
CREATE TABLE matview_test_table (<br />
id SERIAL PRIMARY KEY,<br />
ts TIMESTAMPTZ NOT NULL<br />
)<br />
<br />
INSERT INTO matview_test_table VALUES (<br />
DEFAULT,<br />
((NOW() - '2 days'::INTERVAL) + (generate_series(1,1000) || ' seconds')::INTERVAL)::TIMESTAMPTZ<br />
)<br />
<br />
Create a materialized view which lists the 5 most recent entries:<br />
<br />
CREATE MATERIALIZED VIEW matview_test_view AS<br />
SELECT id, ts<br />
FROM matview_test_table<br />
ORDER BY id DESC <br />
LIMIT 5<br />
<br />
postgres=# SELECT * from matview_test_view ;<br />
id | ts <br />
------+-------------------------------<br />
1000 | 2013-05-06 12:02:10.974711+09<br />
999 | 2013-05-06 12:02:09.974711+09<br />
998 | 2013-05-06 12:02:08.974711+09<br />
997 | 2013-05-06 12:02:07.974711+09<br />
996 | 2013-05-06 12:02:06.974711+09<br />
(5 rows)<br />
<br />
Add more data to the table:<br />
<br />
INSERT INTO matview_test_table VALUES (<br />
DEFAULT,<br />
((NOW() - '1 days'::INTERVAL) + (generate_series(1,1000) || ' seconds')::INTERVAL)::TIMESTAMPTZ<br />
)<br />
<br />
View output does not change:<br />
<br />
postgres=# SELECT * from matview_test_view ;<br />
id | ts <br />
------+-------------------------------<br />
1000 | 2013-05-06 12:02:10.974711+09<br />
999 | 2013-05-06 12:02:09.974711+09<br />
998 | 2013-05-06 12:02:08.974711+09<br />
997 | 2013-05-06 12:02:07.974711+09<br />
996 | 2013-05-06 12:02:06.974711+09<br />
(5 rows)<br />
<br />
Refresh the view to display the latest table entries:<br />
<br />
postgres=# REFRESH MATERIALIZED VIEW matview_test_view ;<br />
REFRESH MATERIALIZED VIEW<br />
postgres=# SELECT * from matview_test_view ;<br />
id | ts <br />
------+-------------------------------<br />
2001 | 2013-05-07 12:03:10.696626+09<br />
2000 | 2013-05-07 12:03:09.696626+09<br />
1999 | 2013-05-07 12:03:08.696626+09<br />
1998 | 2013-05-07 12:03:07.696626+09<br />
1997 | 2013-05-07 12:03:06.696626+09<br />
(5 rows)<br />
<br />
The links below contain more detailed information and examples.<br />
<br />
'''Links'''<br />
* Documentation:<br />
** [http://www.postgresql.org/docs/9.3/static/rules-materializedviews.html Overview]<br />
** [http://www.postgresql.org/docs/9.3/static/sql-creatematerializedview.html CREATE command]<br />
* [http://www.depesz.com/2013/03/04/waiting-for-9-3-add-a-materialized-view-relations/ Waiting for 9.3 – Add a materialized view relations]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-materialized-views/ Postgres 9.3 feature highlight: Materialized views]<br />
<br />
=== Recursive View Syntax ===<br />
<br />
The CREATE RECURSIVE VIEW syntax provides a shorthand way of formulating a recursive common table expression (CTE) as a view.<br />
<br />
Taking the example from the [http://www.postgresql.org/docs/current/static/queries-with.html#QUERIES-WITH-SELECT CTE documentation]:<br />
<br />
WITH RECURSIVE t(n) AS (<br />
VALUES (1)<br />
UNION ALL<br />
SELECT n+1 FROM t WHERE n < 100<br />
)<br />
SELECT * FROM t;<br />
<br />
This can be created as a recursive view as follows:<br />
<br />
CREATE RECURSIVE VIEW t(n) AS<br />
VALUES (1)<br />
UNION ALL<br />
SELECT n+1 FROM t WHERE n < 100;<br />
<br />
'''Links'''<br />
* [http://www.postgresql.org/docs/9.3/static/sql-createview.html Documentation]<br />
* [http://www.depesz.com/2013/03/04/waiting-for-9-3-add-create-recursive-view-syntax/ Waiting for 9.3 – Add CREATE RECURSIVE VIEW syntax]<br />
<br />
=== Updatable Views ===<br />
<br />
Simple views can now be updated in the same way as regular tables. The view can only reference one table (or another updatable view) and must not contain more complex operators, join types etc. <br />
<br />
If the view has a WHERE condition, UPDATEs and DELETEs on the underlying table will be restricted to those rows it defines. However UPDATEs may change a row so that it is no longer visible in the view, and an INSERT command can potentiall insert rows which do not satisfy the WHERE condition.<br />
<br />
More complex views can be made updatable as before using INSTEAD OF triggers or INSTEAD rules.<br />
<br />
Simple example using the following table and view:<br />
<code> <br />
CREATE TABLE postgres_versions (<br />
version VARCHAR(3) PRIMARY KEY,<br />
nickname TEXT NOT NULL<br />
);<br />
<br />
INSERT INTO postgres_versions VALUES<br />
('8.0', 'Excitable Element'),<br />
('8.1', 'Fishy Foreign Key'),<br />
('8.2', 'Grumpy Grant'),<br />
('8.3', 'Hysterical Hstore'),<br />
('8.4', 'Insane Index'),<br />
('9.0', 'Jumpy Join'),<br />
('9.1', 'Killer Key'),<br />
('9.2', 'Laconical Lexer'),<br />
('9.3', 'Morose Module');<br />
<br />
CREATE VIEW postgres_versions_9 AS<br />
SELECT version, nickname<br />
FROM postgres_versions<br />
WHERE version LIKE '9.%';<br />
</code><br />
<br />
<code> <br />
postgres=# SELECT * from postgres_versions_9;<br />
version | nickname <br />
---------+-----------------<br />
9.0 | Jumpy Join<br />
9.1 | Killer Key<br />
9.2 | Laconical Lexer<br />
9.3 | Morose Module<br />
(4 rows)<br />
<br />
postgres=# UPDATE postgres_versions_9 SET nickname='Maniac Master' WHERE version='9.3';<br />
UPDATE 1<br />
postgres=# SELECT * from postgres_versions_9;<br />
version | nickname <br />
---------+-----------------<br />
9.0 | Jumpy Join<br />
9.1 | Killer Key<br />
9.2 | Laconical Lexer<br />
9.3 | Maniac Master<br />
(4 rows)<br />
</code><br />
<br />
'''Links'''<br />
* [http://www.postgresql.org/docs/9.3/static/sql-createview.html#SQL-CREATEVIEW-UPDATABLE-VIEWS Documentation]<br />
* [http://www.depesz.com/2012/12/11/waiting-for-9-3-support-automatically-updatable-views/ Waiting for 9.3 – Support automatically-updatable views]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-auto-updatable-views/ Postgres 9.3 feature highlight: auto-updatable views]<br />
<br />
== Writeable Foreign Tables ==<br />
<br />
"Foreign Data Wrappers" (FDW) were introduced in PostgreSQL 9.1, providing a way of accessing external data sources from within PostgreSQL using SQL. The original implementation was read-only, but 9.3 will enable write access as well, provided the individual FDW drivers have been updated to support this. At the time of writing, only the Redis and PostgreSQL drivers have write support (''need to verify this'').<br />
<br />
See [[#postgres_fdw|below]] for more information on the PostgreSQL driver and a simple example.<br />
<br />
'''Links'''<br />
<br />
* [http://www.postgresql.org/docs/9.3/static/sql-createserver.html CREATE SERVER]<br />
* [http://www.postgresql.org/docs/9.3/static/sql-createforeigndatawrapper.html CREATE FOREIGN DATA WRAPPER]<br />
* [http://www.postgresql.org/docs/9.3/static/fdwhandler.html Documentation: Writing A Foreign Data Wrapper]<br />
* [http://michael.otacoo.com/postgresql-2/postgres-9-3-feature-highlight-writable-foreign-tables/ Postgres 9.3 feature highlight: writable foreign tables]<br />
* [http://www.depesz.com/2013/03/17/waiting-for-9-3-support-writable-foreign-tables/ Waiting for 9.3 – Support writable foreign tables] <br />
<br />
=== postgres_fdw ===<br />
<br />
A new contrib module, postgres_fdw, provides the eponymous foreign data wrapper for read/write access to remote PostgreSQL servers (or to another database on the local server).<br />
<br />
A simple usage example (connecting to a different database on the same server for ease of testing).<br />
<br />
1. Build the postgres_fdw contrib module<br />
<br />
cd contrib/postgres_fdw<br />
make install<br />
<br />
2. Install the module as an extension<br />
<br />
postgres=# CREATE EXTENSION postgres_fdw;<br />
CREATE EXTENSION<br />
<br />
3. Create a test "remote" database<br />
<br />
postgres=# CREATE DATABASE fdw_test;<br />
CREATE DATABASE<br />
postgres=# \c fdw_test<br />
You are now connected to database "fdw_test" as user "barwick".<br />
fdw_test=# CREATE TABLE world (greeting TEXT);<br />
CREATE TABLE<br />
<br />
4. Create the server, user and table mapping so that the local PostgreSQL server knows about the remote database:<br />
<br />
postgres=# CREATE SERVER postgres_fdw_test FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host 'localhost', dbname 'fdw_test');<br />
CREATE SERVER<br />
postgres=# CREATE USER MAPPING FOR PUBLIC SERVER postgres_fdw_test OPTIONS (password <nowiki>''</nowiki>);<br />
CREATE USER MAPPING<br />
postgres=# CREATE FOREIGN TABLE other_world (greeting TEXT) SERVER postgres_fdw_test OPTIONS (table_name 'world');<br />
CREATE FOREIGN TABLE<br />
postgres=# \det<br />
List of foreign tables<br />
Schema | Table | Server <br />
--------+-------------+-------------------<br />
public | other_world | postgres_fdw_test<br />
(1 row)<br />
<br />
5. Manipulate the remote table as if it were a local one:<br />
<br />
postgres=# INSERT INTO other_world VALUES('Take me to your leader');<br />
INSERT 0 1 <br />
postgres=# \c fdw_test<br />
You are now connected to database "fdw_test" as user "barwick".<br />
fdw_test=# SELECT * FROM world;<br />
hello <br />
------------------------<br />
Take me to your leader<br />
(1 row)<br />
<br />
Here's another example, where we link to the "account" and "branches" tables on a remote pgbench database:<br />
<br />
create extension postgres_fdw;<br />
create user mapping for current_user server remotesrv options ( user 'postgres', password 'password' );<br />
create server remotesrv foreign data wrapper postgres_fdw options ( host '192.168.1.5', port '5433', dbname 'bench');<br />
create foreign table remoteacct (aid int, bid int, abalance int, filler char(84)) <br />
server remotesrv options ( table_name 'pgbench_accounts' );<br />
create foreign table remotebranch ( bid int, bbalance int, filler char(88) ) <br />
server remotesrv options ( table_name 'pgbench_branches');<br />
<br />
Having set this up, we can query the remote server:<br />
<br />
explain select * from remotebranch join remoteacct using ( bid ) where bid = 5;<br />
QUERY PLAN<br />
----------------------------------------------------------------------------<br />
Nested Loop (cost=200.00..225.40 rows=1 width=712)<br />
-> Foreign Scan on remotebranch (cost=100.00..112.66 rows=1 width=364)<br />
-> Foreign Scan on remoteacct (cost=100.00..112.73 rows=1 width=352)<br />
<br />
Notice a couple of things: first, JOIN push-down to the remote server isn't implemented yet (wait for 9.4!). Second, we're not getting real estimates for the remote tables. This is fixable, but telling Postgres to query the remote DB for EXPLAIN information:<br />
<br />
alter foreign table remotebranch options (add use_remote_estimate 'true');<br />
alter foreign table remoteacct options (add use_remote_estimate 'true');<br />
bench=# explain select * from remotebranch join remoteacct using ( bid ) where bid = 5;<br />
QUERY PLAN<br />
------------------------------------------------------------------------------<br />
Nested Loop (cost=200.42..7648.07 rows=99400 width=712)<br />
-> Foreign Scan on remotebranch (cost=100.00..101.14 rows=1 width=364)<br />
-> Foreign Scan on remoteacct (cost=100.42..6552.93 rows=99400 width=97)<br />
<br />
'''Links'''<br />
* [http://www.postgresql.org/docs/9.3/static/postgres-fdw.html Documentation]<br />
<br />
== Replication Improvements ==<br />
<br />
PostgreSQL's built-in binary replication has been improved in four ways: streaming-only remastering, fast failover, and architecture-independent streaming, and pg_basebackup conf setup.<br />
<br />
=== Streaming-Only Remastering ===<br />
<br />
"Remastering" is the process whereby a replica in a set of replicas becomes the new master for all of the other replicas. For example:<br />
<br />
# Master M1 is replicating to replicas R1, R2 and R3.<br />
# Master M1 needs to be taken down for a hardware upgrade.<br />
# The DBA promotes R1 to be the master. <br />
# R2 and R3 are reconfigured & restarted, and now replicate from R1<br />
<br />
That's remastering in a nutshell. It's even more useful in combination with cascading replication (introduced in 9.2).<br />
<br />
In prior versions of PostgreSQL, remastering required using WAL file archiving. Cascading replicas could not switch masters using streaming alone; they would have to be re-cloned. That restriction has now been lifted, allowing remastering from just the stream. This makes it much easier to set up large replication clusters; administrators no longer have to set up an online WAL archive if they don't need one for disaster recovery.<br />
<br />
Incidentally, this also makes it possible to set up "cycles" where replication is going in a circle. Whether that's a feature or a bug depends on your perspective.<br />
<br />
Links:<br />
* [http://www.databasesoup.com/2013/01/cascading-replication-and-cycles.html Cascading Replication and Cycles]<br />
<br />
=== Fast Failover ===<br />
<br />
Allows replicas to be promoted in less than a second, permitting 99.999% uptime. More details TBD.<br />
<br />
=== Architecture-Independent Streaming ===<br />
<br />
Allows streaming of base backups (using pg_basebackup) and log archiving (using pg_receivexlog) between different OSes and hardware architectures. (Note that you still need the same architecture to restore the backups, but this is useful for example with centralized backup servers)<br />
<br />
=== pg_basebackup conf setup ===<br />
<br />
If you use the -R switch, pg_basebackup will create a simple (streaming-only) recovery.conf file in the newly cloned data directory. This means that you can immediately start the new database server without doing additional editing.<br />
<br />
=Backward compatibility=<br />
<br />
These changes may incur regressions in your applications.<br />
<br />
== CREATE TABLE output ==<br />
<br />
CREATE TABLE will no longer output messages about implicit index and sequence creation unless the log level is set to DEBUG1.<br />
<br />
== Server settings ==<br />
<br />
* Parameter 'commit_delay' is restricted to superusers only<br />
* Parameter 'replication_timeout' has been renamed to 'wal_sender_timeout'<br />
* Parameter 'unix_socket_directory' has been replaced 'unix_socket_directories'<br />
* In-memory sorts to use their full memory allocation; if work_mem was set on the basis of the pre-9.3 behavior, its value may need to be reviewed.<br />
<br />
== WAL filenames may end in FF ==<br />
<br />
WAL files will now be written in a continuous stream, rather than skipping the last 16MB segment every 4GB, meaning WAL filenames may end in FF. WAL backup or restore scripts may need to be adapted.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Todo&diff=19328Todo2013-04-04T23:13:45Z<p>Schmiddy: Add link for 'Allow processing of multiple -f (file) options'</p>
<hr />
<div><div style="margin: 1ex 1em; float: right;"><br />
__TOC__<br />
</div><br />
<br />
This list contains '''known PostgreSQL bugs and feature requests''' and we hope it is complete. If you would like to work on an item, please read the [[Developer FAQ]] first. There is also a [[Development_information|development information page]].<br />
<br />
* {{TodoPending}} - marks ordinary, incomplete items<br />
* {{TodoEasy}} - marks items that are easier to implement<br />
* {{TodoDone}} - marks changes that are done, and will appear in the PostgreSQL 9.3 release.<br />
<br />
For help on editing this list, please see [[Talk:Todo]]. <b>Please do not add items here without discussion on the mailing list.</b><br />
<br />
<b>For Developers:</b> Unfortunately this list does not contain all the information necessary for someone to start coding a feature. Some of these items might have become unnecessary since they were added --- others might be desirable but the implementation might be unclear. When selecting items listed below, be prepared to first discuss the value of the feature. Do not assume that you can select one, code it and then expect it to be committed. Always discuss design on Hackers list before starting to code. The flow should be:<br />
<br />
Desirability -> Design -> Implement -> Test -> Review -> Commit<br />
<br />
<div style="padding: 1ex 4em;"><br />
== Administration ==<br />
<br />
{{TodoItem<br />
|Allow administrators to cancel multi-statement idle transactions<br />
|This allows locks to be released, but it is complex to report the cancellation back to the client.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01340.php <nowiki>Cancelling idle in transaction state</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00441.php <nowiki>Re: Cancelling idle in transaction state</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Check for unreferenced table files created by transactions that were in-progress when the server terminated abruptly<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00096.php <nowiki>Removing unreferenced files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Set proper permissions on non-system schemas during db creation<br />
|Currently all schemas are owned by the super-user because they are copied from the template1 database. However, since all objects are inherited from the template database, it is not clear that setting schemas to the db owner is correct.}}<br />
<br />
{{TodoItem<br />
|Allow log_min_messages to be specified on a per-module basis<br />
|This would allow administrators to see more detailed information from specific sections of the backend, e.g. checkpoints, autovacuum, etc. Another idea is to allow separate configuration files for each module, or allow arbitrary SET commands to be passed to them. See also [[Logging Brainstorm]].}}<br />
<br />
{{TodoItem<br />
|Simplify creation of partitioned tables<br />
|This would allow creation of partitioned tables without requiring creation of triggers or rules for INSERT/UPDATE/DELETE, and constraints for rapid partition selection. Options could include range and hash partition selection. See also [[Table partitioning]]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow custom variables to appear in pg_settings()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00850.php <nowiki>Re: count(*) performance improvement ideas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have custom variables be transaction-safe<br />
* {{MessageLink|4B577E9F.8000505@dunslane.net|Custom GUCs still a bit broken}}<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the SQL-standard mechanism whereby REVOKE ROLE revokes only the privilege granted by the invoking role, and not those granted by other roles<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00010.php <nowiki>Re: Grantor name gets lost when grantor role dropped</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent query cancel packets from being replayed by an attacker, especially when using SSL<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00345.php <nowiki>Replay attack of query cancel</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a way to query the log collector subprocess to determine the name of the currently active log file<br />
* [http://archives.postgresql.org/pgsql-general/2008-11/msg00418.php <nowiki>Current log files when rotating?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow simpler reporting of the unix domain socket directory and allow easier configuration of its default location<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg01555.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-10/msg01482.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow custom daemons to be automatically stopped/started along with the postmaster<br />
|This allows easier administration of daemons like user job schedulers or replication-related daemons.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01701.php <nowiki>Re: scheduler in core</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve logging of prepared transactions recovered during startup<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00092.php <nowiki>&quot;recovering prepared transaction&quot; after server restart message</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Consider using POSIX shared memory to avoid System V shared memory kernel limits<br />
* [http://archives.postgresql.org/message-id/4DFA2673.3010009@enterprisedb.com <nowiki>POSIX shared memory patch status</nowiki>]<br />
}}<br />
<br />
=== Configuration files ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Change pg_ident.conf parsing to be the same as pg_hba.conf<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg02204.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf file values to be changed via an SQL API, perhaps using SET GLOBAL<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00764.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-10/msg01509.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-11/msg00002.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider normalizing fractions in postgresql.conf, perhaps using '%'<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00550.php <nowiki>Fractions in GUC variables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow Kerberos to disable stripping of realms so we can check the username@realm against multiple realms<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00009.php <nowiki>krb_match_realm patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve LDAP authentication configuration options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01745.php <nowiki>Proposed Patch - LDAPS support for servers on port 636 w/o TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add external tool to auto-tune some postgresql.conf parameters<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00000.php <nowiki>Re: Overhauling GUCS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00033.php <nowiki>Simple postgresql.conf wizard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add 'hostgss' pg_hba.conf option to allow GSS link-level encryption<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01454.php <nowiki>Re: Plans for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Process pg_hba.conf keywords as case-insensitive<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00432.php <nowiki>More robust pg_hba.conf parsing/error logging</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Create utility to compute accurate random_page_cost value<br />
* http://archives.postgresql.org/pgsql-performance/2011-04/msg00162.php<br />
* http://archives.postgresql.org/pgsql-performance/2011-04/msg00362.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow configuration files to be independently validated<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01831.php<br />
* http://archives.postgresql.org/message-id/12666.1310774573@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf settings to be accepted by backends even if some settings are invalid for those backends<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00330.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00375.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow all backends to receive postgresql.conf setting changes at the same time<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00330.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00375.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow synchronous_standby_names to be disabled after communication failure with all synchronous standby servers exceeds some timeout<br />
|This also requires successful execution of a synchronous notification command.<br />
* http://archives.postgresql.org/pgsql-hackers/2012-07/msg00409.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Tablespaces ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow a database in tablespace t1 with tables created in tablespace t2 to be used as a template for a new database created with default tablespace t2<br />
|Currently all objects in the default database tablespace must have default tablespace specifications. This is because new databases are created by copying directories. If you mix default tablespace tables and tablespace-specified tables in the same directory, creating a new database from such a mixed directory would create a new database with tables that had incorrect explicit tablespaces. To fix this would require modifying pg_class in the newly copied database, which we don't currently do.}}<br />
<br />
{{TodoItem<br />
|Allow reporting of which objects are in which tablespaces<br />
|This item is difficult because a tablespace can contain objects from multiple databases. There is a server-side function that returns the databases which use a specific tablespace, so this requires a tool that will call that function and connect to each database to find the objects in each database for that tablespace.}}<br />
<br />
{{TodoItem<br />
|Allow WAL replay of CREATE TABLESPACE to work when the directory structure on the recovery computer is different from the original}}<br />
<br />
{{TodoItem<br />
|Allow per-tablespace quotas}}<br />
<br />
{{TodoItem<br />
|Allow tablespaces on RAM-based partitions for unlogged tables<br />
* http://archives.postgresql.org/pgsql-advocacy/2011-05/msg00033.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow toast tables to be moved to a different tablespace<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-05/msg00980.php]<br />
* {{messageLink|CAFEQCbH756DyyAPQ1ykh3+b+kE1-EhWRww1WO_x5v38C-uLnUg@mail.gmail.com|patch : Allow toast tables to be moved to a different tablespace}} (issues remain)<br />
* [http://archives.postgresql.org/message-id/CAFEQCbEq07OopgE5xFYv2Q3eMq45hRSJkjCBO+kvpJq9NEVhow@mail.gmail.com Allow toast tables to be moved to a different tablespace]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Statistics Collector ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow statistics last vacuum/analyze execution times to be displayed without requiring track_counts to be enabled<br />
* [http://archives.postgresql.org/pgsql-docs/2007-04/msg00028.php <nowiki>row-level stats and last analyze time</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Clear table counters on TRUNCATE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00169.php <nowiki>Small TRUNCATE glitch</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SSL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SSL authentication/encryption over unix domain sockets<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00924.php <nowiki>Re: Spoofing as the postmaster</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL key file permission checks to be optionally disabled when sharing SSL keys with other applications<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php <nowiki>BUG #3809: SSL &quot;unsafe&quot; private key permissions bug</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL CRL files to be re-read during configuration file reload, rather than requiring a server restart<br />
|Unlike SSL CRT files, CRL (Certificate Revocation List) files are updated frequently<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00832.php <nowiki>Automatic CRL reload</nowiki>]<br />
Alternatively or additionally supporting OCSP (online certificate security protocol) would provide real-time revocation discovery without reloading<br />
}}<br />
<br />
{{TodoItem<br />
| Allow automatic selection of SSL client certificates from a certificate store<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00406.php <nowiki>Allow multiple certificates or keys in the postgresql.crt/.key files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Send the full certificate server chain to the client<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-12/msg00145.php BUG #5245: Full Server Certificate Chain Not Sent to client]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Point-In-Time Recovery (PITR) ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Create dump tool for write-ahead logs for use in determining transaction id for point-in-time recovery<br />
|This is useful for checking PITR recovery.}}<br />
<br />
{{TodoItem<br />
|Allow archive_mode to be changed without server restart?<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01655.php <nowiki>Enabling archive_mode without restart</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider avoiding WAL switching via archive_timeout if there has been no database activity<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01469.php <nowiki>archive_timeout behavior for no activity</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00395.php <nowiki>Re: archive_timeout behavior for no activity</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow base backup from standby to continue when the standby is promoted.<br />
* [http://archives.postgresql.org/pgsql-hackers/2012-10/msg00239.php <nowiki>Re: Promoting a standby during base backup</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Standby server mode ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
| Allow pg_xlogfile_name() to be used in recovery mode<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1001190135vd9f62f1sa7868abc1ea61d12@mail.gmail.com <nowiki>Streaming replication and pg_xlogfile_name()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Prevent variables inherited from the server environment from begin used for making streaming replication connections.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01011.php <nowiki>Re: Parameter name standby_mode</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Change walsender so that it applies per-role settings<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00642.php<br />
}}<br />
<br />
{{TodoItem<br />
| Restructure configuration parameters for standby mode<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01820.php<br />
}}<br />
<br />
{{TodoItemf<br />
| Allow time-delayed application of logs on the standby<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00992.php<br />
}}<br />
<br />
{{TodoItem<br />
| Add -X parameter to pg_basebackup to specify a different directory for px_xlog, like initdb<br />
}}<br />
<br />
{{TodoItem<br />
| Add a new "eager" synchronous mode that starts out synchronous but reverts to asynchronous after a failure timeout period<br />
|This would require some type of command to be executed to alert administrators of this change.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-12/msg01224.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Data Types ==<br />
<br />
{{TodoItem<br />
|Fix data types where equality comparison is not intuitive, e.g. box<br />
* http://archives.postgresql.org/pgsql-hackers/2011-10/msg01643.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for public SYNONYMs<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00519.php <nowiki>Proposal for SYNONYMS</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php<br />
* http://archives.postgresql.org/pgsql-general/2010-12/msg00139.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for SQL-standard GENERATED/IDENTITY columns<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg00543.php <nowiki>Re: Three weeks left until feature freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00038.php <nowiki>GENERATED ... AS IDENTITY, Was: Re: Feature Freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00344.php <nowiki>Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00076.php <nowiki>Re: [HACKERS] Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00604.php <nowiki>IDENTITY/GENERATED patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider placing all sequences in a single table, or create a system view<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2012-02/msg00258.php Removing special case OID generation]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a special data type for regular expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg01067.php <nowiki>Why is there a tsquery data type?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce BIT data type overhead using short varlena headers<br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00273.php <nowiki>storage size of &quot;bit&quot; data type..</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow renaming and deleting enumerated values from an existing enumerated data type<br />
}}<br />
<br />
{{TodoItem<br />
|Support scoped IPv6 addresses in the inet type<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00111.php <nowiki>strange problem with ip6</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Considering improving performance of computing CHAR() value lengths<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00900.php <nowiki>char() overhead on read-only workloads not so insignifcant as the docs claim it is...</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01787.php <nowiki>Re: [PATCH] backend: compare word-at-a-time in bcTruelen</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add overlaps geometric operators that ignore point overlaps<br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00861.php<br />
}}<br />
<br />
{{TodoItem<br />
|Remove or improve rounding in geometric comparison operators<br />
* http://archives.postgresql.org/message-id/9804.1346187849@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
| Add IMMUTABLE column attribute<br />
* http://archives.postgresql.org/pgsql-hackers/2011-11/msg00623.php<br />
}}<br />
<br />
=== Domains ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow functions defined as casts to domains to be called during casting<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-05/msg00072.php <nowiki>bug? non working casts for domain</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg01681.php <nowiki>TODO: Fix CREATE CAST on DOMAINs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow values to be cast to domain types<br />
* [http://archives.postgresql.org/pgsql-hackers/2003-06/msg01206.php <nowiki>Domain casting still doesn't work right</nowiki>] <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00289.php <nowiki>domain casting?</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00812.php<br />
}}<br />
<br />
{{TodoItem<br />
|Make domains work better with polymorphic functions<br />
* [http://archives.postgresql.org/message-id/4887.1228700773@sss.pgh.pa.us Polymorphic types vs. domains]<br />
* [http://archives.postgresql.org/message-id/15535.1238774571@sss.pgh.pa.us some difficulties with fixing it]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Dates and Times ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow infinite intervals just like infinite timestamps<br />
* http://archives.postgresql.org/pgsql-hackers/2011-11/msg00076.php<br />
}}<br />
<br />
{{TodoItem<br />
|Determine how to represent date/time field extraction on infinite timestamps<br />
* [http://archives.postgresql.org/message-id/CA+mi_8bda-Fnev9iXeUbnqhVaCWzbYhHkWoxPQfBca9eDPpRMw@mail.gmail.com extract(epoch from infinity) is not 0]<br />
* [http://archives.postgresql.org/message-id/CADAkt-icuESH16uLOCXbR-dKpcvwtUJE4JWXnkdAjAAwP6j12g@mail.gmail.com converting between infinity timestamp and float8]<br />
}}<br />
<br />
<br />
{{TodoItem<br />
|Allow TIMESTAMP WITH TIME ZONE to store the original timezone information, either zone name or offset from UTC<br />
|If the TIMESTAMP value is stored with a time zone name, interval computations should adjust based on the time zone rules. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-10/msg00705.php <nowiki>timestamp with time zone a la sql99</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have timestamp subtraction not call justify_hours()?<br />
* [http://archives.postgresql.org/pgsql-sql/2006-10/msg00059.php <nowiki>timestamp subtraction (was Re: formatting intervals with to_char)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve TIMESTAMP WITH TIME ZONE subtraction to be DST-aware<br />
|Currently subtracting one date from another that crosses a daylight savings time adjustment can return '1 day 1 hour', but adding that back to the first date returns a time one hour in the future. This is caused by the adjustment of '25 hours' to '1 day 1 hour', and '1 day' is the same time the next day, even if daylight savings adjustments are involved.}}<br />
<br />
{{TodoItem<br />
|Fix interval display to support values exceeding 2^31 hours}}<br />
<br />
{{TodoItem<br />
|Add overflow checking to timestamp and interval arithmetic}}<br />
<br />
{{TodoItem<br />
|Add function to allow the creation of timestamps using parameters<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00232.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Arrays ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add support for arrays of domains<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00114.php <nowiki>Re: updated WIP: arrays of composites</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single-byte header storage for array elements}}<br />
<br />
{{TodoItem<br />
|Add function to detect if an array is empty<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00475.php <nowiki>Re: array_length()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of empty arrays<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01033.php <nowiki>So what's an &quot;empty&quot; array anyway?</nowiki>]<br />
* http://archives.postgresql.org/pgsql-general/2012-07/msg00633.php<br />
* [http://www.postgresql.org/message-id/1182.1363387349@sss.pgh.pa.us <nowiki>Allow declaration of an empty array?</nowiki>]<br />
* [http://www.postgresql.org/message-id/CADxJZo0keVhSRzUnot2Y6g46tsP7f-eV28iEmBd3AtLjU-YTMA@mail.gmail.com Exorcise "zero-dimensional" arrays]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULLs in arrays<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-11/msg00009.php <nowiki>BUG #4509: array_cat's null behaviour is inconsistent</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01040.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Binary Data ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve vacuum of large objects, like contrib/vacuumlo?}}<br />
<br />
{{TodoItem<br />
|Auto-delete large objects when referencing row is deleted<br />
|contrib/lo offers this functionality.}}<br />
<br />
{{TodoItem<br />
|Allow read/write into TOAST values like large objects<br />
|Writing might require the TOAST column to be stored EXTERNAL.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg00049.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add API for 64-bit large object access<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00781.php <nowiki>64-bit API for large objects</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01790.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== MONEY Data Type ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add locale-aware MONEY type, and support multiple currencies<br />
* [http://archives.postgresql.org/pgsql-general/2005-08/msg01432.php <nowiki>A real currency type</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01181.php <nowiki>Money type todos?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|MONEY dumps in a locale-specific format making it difficult to restore to a system with a different locale}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Text Search ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dictionaries to change the token that is passed on to later dictionaries<br />
* [http://archives.postgresql.org/pgsql-patches/2007-11/msg00081.php <nowiki>a tsearch2 (8.2.4) dictionary that only filters out stopwords</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a function-based API for '@@' searches<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00511.php <nowiki>Simplifying Text Search</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve text search error messages<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00966.php <nowiki>Poorly designed tsearch NOTICEs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01146.php <nowiki>Re: Poorly designed tsearch NOTICEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing error to warning for strings larger than one megabyte<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00190.php <nowiki>BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00062.php <nowiki>Re: [BUGS] BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|tsearch and tsdicts regression tests fail in Turkish locale on glibc<br />
* [http://archives.postgresql.org/message-id/49749645.5070801@gmx.net tsearch with Turkish locale]<br />
}}<br />
<br />
{{TodoItem<br />
|tsquery negator operator treated as part of lexeme<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-06/msg00346.php BUG #4887: inclusion operator (@>) on tsqeries behaves not conforming to documentation]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of dash and plus signs in email address user names, and perhaps improve URL parsing<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00772.php<br />
* [http://archives.postgresql.org/message-id/E1Ri8il-0008Ct-9p@wrigleys.postgresql.org tsearch does not recognize all valid emails]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve default parser, to more easily allow adding new tokens<br />
* http://archives.postgresql.org/message-id/23485.1297727826@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Add additional support functions<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg00319.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== XML ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow XML arrays to be cast to other data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00981.php <nowiki>proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00231.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00471.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add XML Schema validation and xmlvalidate functions (SQL:2008)}}<br />
<br />
{{TodoItem<br />
|Add xmlvalidatedtd variant to support validating against a DTD?}}<br />
<br />
{{TodoItem<br />
|Relax-NG validation; libxml2 supports this already}}<br />
<br />
{{TodoItem<br />
|Allow reliable XML operation non-UTF8 server encodings (xpath(), in particular, is known to not work)<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-01/msg00135.php <nowiki>BUG #4622: xpath only work in utf-8 server encoding</nowiki>] <br />
* http://archives.postgresql.org/message-id/4110.1238973350@sss.pgh.pa.us}}<br />
<br />
{{TodoItem<br />
|Add functions from SQL:2006: XMLDOCUMENT, XMLCAST, XMLTEXT}}<br />
<br />
{{TodoItem<br />
|Add XMLNAMESPACES support in XMLELEMENT and elsewhere}}<br />
<br />
{{TodoItem<br />
|Move XSLT from contrib/xml2 to a more reasonable location<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00539.php<br />
}}<br />
<br />
{{TodoItem<br />
|Report errors returned by the XSLT library<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00562.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve the XSLT parameter passing API<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00416.php<br />
}}<br />
<br />
{{TodoItem<br />
|XML Canonical: Convert XML documents to canonical form to compare them. libxml2 has support for this.}}<br />
<br />
{{TodoItem<br />
|Add pretty-printed XML output option<br />
|Parse a document and serialize it back in some indented form. libxml2 might support this.}}<br />
<br />
{{TodoItem<br />
|Add XMLQUERY (from the SQL/XML standard)}}<br />
<br />
{{TodoItem<br />
|Allow XML sthredding<br />
|In some cases shredding could be better option (if there is no need to keep XML docs entirely, e.g. if we have already developed tools that understand only relational data. This would be a separate module that implements annotated schema decomposition technique, similar to DB2 and SQL Server functionality.}}<br />
<br />
{{TodoItem<br />
|Fix Nested or repeated xpath() that apparently mess up namespaces [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00097.php] [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00144.php] [http://archives.postgresql.org/pgsql-general/2008-03/msg00295.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/message-id/004f01c90e91$138e9d10$3aabd730$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItem<br />
|XPath: Adding the <x> at the root causes problems [http://archives.postgresql.org/pgsql-bugs/2008-05/msg00184.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/pgsql-general/2008-07/msg00613.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table needs to be implemented/implementable to get rid of contrib/xml2 [http://archives.postgresql.org/pgsql-general/2008-05/msg00823.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table is pretty broken anyway [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02424.php]}}<br />
<br />
{{TodoItem<br />
|better handling of XPath data types [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00616.php] [http://archives.postgresql.org/message-id/004a01c90e90$4b986d90$e2c948b0$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItem<br />
|Improve handling of PIs and DTDs in xmlconcat() [http://archives.postgresql.org/message-id/200904211211.n3LCB09p008988@wwwmaster.postgresql.org]}}<br />
<br />
{{TodoItem<br />
|Restructure XML and /contrib/xml2 functionality<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02314.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00017.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Functions ==<br />
<br />
{{TodoItem<br />
|Allow INET subnet comparisons using non-constants to be indexed}}<br />
<br />
{{TodoItem<br />
|Add an INET overlaps operator, for use by exclusion constraints <br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00845.php<br />
}}<br />
<br />
{{TodoItem<br />
|Enforce typmod for function inputs, function results and parameters for spi_prepare'd statements called from PLs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01403.php <nowiki>Re: BUG #2917: spi_prepare doesn't accept typename aliases</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01160.php <nowiki>RFC for adding typmods to functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix IS OF so it matches the ISO specification, and add documentation<br />
* [http://archives.postgresql.org/pgsql-patches/2003-08/msg00060.php <nowiki>Re: [HACKERS] IS OF</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00060.php <nowiki>ToDo: add documentation for operator IS OF</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement Boyer-Moore searching in LIKE queries<br />
* {{messageLink|27645.1220635769@sss.pgh.pa.us|TODO item: Implement Boyer-Moore searching (First time hacker)}}<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent malicious functions from being executed with the permissions of unsuspecting users<br />
|Index functions are safe, so VACUUM and ANALYZE are safe too. Triggers, CHECK and DEFAULT expressions, and rules are still vulnerable. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00268.php <nowiki>Some notes about the index-functions security vulnerability</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce memory usage of aggregates in set returning functions<br />
* [http://archives.postgresql.org/pgsql-performance/2008-01/msg00031.php <nowiki>Re: Performance of aggregates over set-returning functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/ltree operator<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-11/msg00044.php <nowiki>BUG #3720: wrong results at using ltree</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/btree_gist's implementation of inet indexing<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-10/msg00099.php <nowiki>BUG #5705: btree_gist: Index on inet changes query result</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|<nowiki>Fix inconsistent precedence of =, &gt;, and &lt; compared to &lt;&gt;, &gt;=, and &lt;=</nowiki><br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00145.php <nowiki>BUG #3822: Nonstandard precedence for comparison operators</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix regular expression bug when using complex back-references<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00000.php <nowiki>BUG #3645: regular expression back references seem broken</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have /contrib/dblink reuse unnamed connections<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00895.php <nowiki>dblink un-named connection doesn't get re-used</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve formatting of pg_get_viewdef() output<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01648.php <nowiki>pg_get_viewdef formattiing</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01885.php <nowiki>Re: pretty print viewdefs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-12/msg00906.php reprise: pretty print viewdefs]<br />
}}<br />
<br />
{{TodoItem<br />
|Add function to dump pg_depend information cleanly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00226.php <nowiki>Elementary dependency look-up</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add function to allow easier transaction id comparisons<br />
* http://archives.postgresql.org/pgsql-hackers/2011-11/msg00786.php<br />
}}<br />
<br />
=== Character Formatting ===<br />
<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Allow to_date() and to_timestamp() to accept localized month names}}<br />
<br />
{{TodoItem<br />
|Add missing parameter handling in to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg00948.php <nowiki>Re: to_char and i18n</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Throw an error from to_char() instead of printing a string of "#" when a number doesn't fit in the desired output format.<br />
* discussed in [http://archives.postgresql.org/message-id/37ed240d0907290836w42187222n18664dfcbcb445b1@mail.gmail.com "to_char, support for EEEE format"]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow to_char() on interval values to accumulate the highest unit requested<br />
|2= Some special format flag would be required to request such accumulation. Such functionality could also be added to EXTRACT. Prevent accumulation that crosses the month/day boundary because of the uneven number of days in a month.<br />
* to_char(INTERVAL '1 hour 5 minutes', 'MI') =&gt; 65<br />
* to_char(INTERVAL '43 hours 20 minutes', 'MI' ) =&gt; 2600<br />
* to_char(INTERVAL '43 hours 20 minutes', 'WK:DD:HR:MI') =&gt; 0:1:19:20<br />
* to_char(INTERVAL '3 years 5 months','MM') =&gt; 41<br />
}}<br />
<br />
{{TodoItem<br />
|Fix to_number() handling for values not matching the format string<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01447.php <nowiki>Re: numeric_to_number() function skipping some digits</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Multi-Language Support ==<br />
<br />
{{TodoItem<br />
|Add NCHAR (as distinguished from ordinary varchar),}}<br />
<br />
{{TodoItem<br />
|Add a cares-about-collation column to pg_proc, so that unresolved-collation errors can be thrown at parse time<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-03/msg01520.php <nowiki>Open issues for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Integrate collations with text search configurations<br />
* [http://archives.postgresql.org/message-id/28887.1303579034@sss.pgh.pa.us <nowiki>Some TODO items for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Integrate collations with to_char() and related functions<br />
* [http://archives.postgresql.org/message-id/28887.1303579034@sss.pgh.pa.us <nowiki>Some TODO items for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support collation-sensitive equality and hashing functions<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-06/msg00472.php <nowiki> contrib/citext versus collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a LOCALE option to CREATE DATABASE, as a shorthand<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00119.php <nowiki> Re: 8.4 open items list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support multiple simultaneous character sets, per SQL:2008}}<br />
<br />
{{TodoItem<br />
|Improve UTF8 combined character handling?}}<br />
<br />
{{TodoItem<br />
|Add octet_length_server() and octet_length_client()}}<br />
<br />
{{TodoItem<br />
|Make octet_length_client() the same as octet_length()?}}<br />
<br />
{{TodoItem<br />
|Fix problems with wrong runtime encoding conversion for NLS message files}}<br />
<br />
{{TodoItem<br />
|Add URL to more complete multi-byte regression tests<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-07/msg00272.php <nowiki>Multi-byte and client side character encoding tests for copy command..</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix contrib/fuzzystrmatch to work with multibyte encodings<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00047.php <nowiki> soundex function returns UTF-16 characters</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00138.php <nowiki> dmetaphone woes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Change memory allocation for multi-byte functions so memory is allocated inside conversion functions<br />
|Currently we preallocate memory based on worst-case usage.}}<br />
<br />
{{TodoItem<br />
|Add ability to use case-insensitive regular expressions on multi-byte characters<br />
|Currently it works for UTF-8, but not other multi-byte encodings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00433.php <nowiki>Regexps vs. locale</nowiki>]<br />
* {{MessageLink|20091201210024.B1393753FB7@cvs.postgresql.org|A partial solution for UTF-8}}<br />
}}<br />
<br />
{{TodoItem<br />
|Improve encoding of connection startup messages sent to the client<br />
|Currently some authentication error messages are sent in the server encoding<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00801.php <nowiki>encoding of PostgreSQL messages</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-01/msg00005.php <nowiki>Re: encoding of PostgreSQL messages</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|More sensible support for Unicode combining characters, normal forms<br />
* http://archives.postgresql.org/message-id/200904141532.44618.peter_e@gmx.net<br />
}}<br />
<br />
== Views and Rules ==<br />
{{TodoItemDone<br />
|Automatically create rules on views so they are updateable, per SQL:2008<br />
|We can only auto-create rules for simple views. For more complex cases users will still have to write rules manually.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00586.php <nowiki>Proposal for updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-08/msg00255.php <nowiki>Updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01746.php <nowiki>Re: [COMMITTERS] pgsql: Automatic view update rules Bernd Helmle</nowiki>]<br />
* http://wiki.postgresql.org/wiki/Updatable_views<br />
* http://archives.postgresql.org/pgsql-hackers/2012-07/msg00035.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-08/msg00303.php<br />
}}<br />
{{TodoItem<br />
|Add the functionality of the WITH CHECK OPTION clause to CREATE VIEW<br />
}}<br />
{{TodoItem<br />
|Allow VIEW/RULE recompilation when the underlying tables change<br />
|This is both difficult and controversial.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01723.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01724.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change2"]<br />
* [http://archives.postgresql.org/message-id/CACk%3DU9NFSzWrEba8G5dZ%3DTZLy3_hx3QXGyCcKVWT%3D4iA1FjMuA@mail.gmail.com VIEW still referring to old name of field]<br />
}}<br />
{{TodoItem<br />
|Make it possible to use RETURNING together with conditional DO INSTEAD rules, such as for partitioning setups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00577.php <nowiki>RETURNING and DO INSTEAD ... Intentional or not?</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add materialized views<br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to modify views via ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00691.php <nowiki>Re: idea: storing view source in system catalogs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01410.php <nowiki>modifying views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00300.php <nowiki>Re: patch: Add columns via CREATE OR REPLACE VIEW</nowiki>]<br />
}}<br />
<br />
== SQL Commands ==<br />
<br />
{{TodoItem<br />
|Add CORRESPONDING BY to UNION/INTERSECT/EXCEPT<br />
* [http://dissipatedheat.com/2011/11/10/how-not-to-write-a-patch-for-postgresql/ How not to write this patch.]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve type determination of unknown (NULL or quoted literal) result columns for UNION/INTERSECT/EXCEPT<br />
* [http://archives.postgresql.org/message-id/9799.1302719551@sss.pgh.pa.us <nowiki>UNION construct type cast gives poor error message</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ROLLUP, CUBE, GROUPING SETS options to GROUP BY<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00838.php <nowiki>WIP: grouping sets support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00466.php <nowiki>Implementation of GROUPING SETS (T431: Extended grouping capabilities)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow prepared transactions with temporary tables created and dropped in the same transaction, and when an ON COMMIT DELETE ROWS temporary table is accessed<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00047.php <nowiki>Re: &quot;could not open relation 1663/16384/16584: No such file or directory&quot; in a specific combination of transactions with temp tables</nowiki>]<br />
* [http://archives.postgresql.org/message-id/492543D5.9050904@enterprisedb.com A suggestion on how to implement this]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a GUC variable to warn about non-standard SQL usage in queries}}<br />
<br />
{{TodoItem<br />
|Add SQL-standard MERGE/REPLACE/UPSERT command<br />
|MERGE is typically used to merge two tables. REPLACE or UPSERT command does UPDATE, or on failure, INSERT. See [[SQL MERGE]] for notes on the implementation details.<br />
}}<br />
<br />
{{TodoItem<br />
|Add NOVICE output level for helpful messages<br />
|For example, have it warn about unjoined tables. This could also control automatic sequence/index creation messages.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow NOTIFY in rules involving conditionals}}<br />
<br />
{{TodoItem<br />
|Allow EXPLAIN to identify tables that were skipped because of constraint_exclusion<br />
}}<br />
<br />
{{TodoItem<br />
|Simplify dropping roles that have objects in several databases}}<br />
<br />
{{TodoItem<br />
|Allow the count returned by SELECT, etc to be represented as an int64 to allow a higher range of values}}<br />
<br />
{{TodoItem<br />
|Add support for WITH RECURSIVE ... CYCLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00291.php <nowiki>WITH RECURSIVE ... CYCLE in vanilla SQL: issues with arrays of rows</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Add DEFAULT .. AS OWNER so permission checks are done as the table owner<br />
|This would be useful for SERIAL nextval() calls and CHECK constraints.}}<br />
<br />
{{TodoItem<br />
|Allow DISTINCT to work in multiple-argument aggregate calls}}<br />
<br />
{{TodoItem<br />
|Add comments on system tables/columns using the information in catalogs.sgml<br />
|Ideally the information would be pulled from the SGML file automatically.}}<br />
<br />
{{TodoItem<br />
|Prevent the specification of conflicting transaction read/write options<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00684.php <nowiki>Re: SET TRANSACTION and SQL Standard</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Support LATERAL subqueries<br />
|Lateral subqueries can reference columns of tables defined outside the subquery at the same level, i.e. ''laterally''.<br />
For example, a LATERAL subquery in a FROM clause could reference tables defined in the same FROM clause.<br />
Currently only the columns of tables defined ''above'' subqueries are recognized.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00292.php <nowiki>LATERAL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00991.php <nowiki>Re: LATERAL</nowiki>]<br />
* [http://archives.postgresql.org/message-id/4F5AA202.9020906@gmail.com lateral function as a subquery - WIP patch]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Prevent temporary tables created with ON COMMIT DELETE ROWS from repeatedly truncating the table on every commit if the table is already empty<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00842.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-03/msg00392.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-04/msg00046.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow DELETE and UPDATE to be used with LIMIT and ORDER BY<br />
* http://archives.postgresql.org/pgadmin-hackers/2010-04/msg00078.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01997.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00021.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow PREPARE of cursors}}<br />
<br />
{{TodoItem<br />
|Have DISCARD PLANS discard plans cached by functions<br />
|DISCARD all should do the same.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00431.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid multiple-evaluation of BETWEEN and IN arguments containing volatile expressions<br />
* http://archives.postgresql.org/message-id/4D95B605.2020709@enterprisedb.com<br />
}}<br />
<br />
{{TodoItem<br />
|Fix nested CASE-WHEN constructs<br />
* http://archives.postgresql.org/message-id/4DDCEEB8.50602@enterprisedb.com<br />
}}<br />
<br />
=== CREATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow CREATE TABLE AS to determine column lengths for complex expressions like SELECT col1 || col2}}<br />
<br />
{{TodoItem<br />
|Have WITH CONSTRAINTS also create constraint indexes<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00149.php <nowiki>Re: CREATE TABLE LIKE INCLUDING INDEXES support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move NOT NULL constraint information to pg_constraint<br />
|Currently NOT NULL constraints are stored in pg_attribute without any designation of their origins, e.g. primary keys. One manifest problem is that dropping a PRIMARY KEY constraint does not remove the NOT NULL constraint designation. Another issue is that we should probably force NOT NULL to be propagated from parent tables to children, just as CHECK constraints are. (But then does dropping PRIMARY KEY affect children?)<br />
* http://archives.postgresql.org/message-id/19768.1238680878@sss.pgh.pa.us<br />
* http://archives.postgresql.org/message-id/200909181005.n8IA5Ris061239@wwwmaster.postgresql.org<br />
* http://archives.postgresql.org/pgsql-hackers/2011-07/msg01223.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-07/msg00358.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent concurrent CREATE TABLE from sometimes returning a cryptic error message<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00169.php <nowiki>BUG #3692: Conflicting create table statements throw unexpected error</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add CREATE SCHEMA ... LIKE that copies a schema}}<br />
<br />
{{TodoItem<br />
|Fix CREATE OR REPLACE FUNCTION to not leave objects depending on the function in inconsistent state<br />
* [http://archives.postgresql.org/pgsql-general/2008-08/msg00985.php indexes on functions and create or replace function]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow temporary tables to exist as empty by default in all sessions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00006.php <nowiki>what is difference between LOCAL and GLOBAL TEMP TABLES in PostgreSQL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg01329.php <nowiki>idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-05/msg00016.php <nowiki>Re: idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01098.php <nowiki>global temporary tables</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2012-04/msg01148.php Temporary tables under hot standby]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow the creation of "distinct" types<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01647.php <nowiki>Distinct types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider analyzing temporary tables when they are first used in a query<br />
|Autovacuum cannot analyze or vacuum temporary tables.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00416.php <nowiki>autovacuum and temp tables support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow an unlogged table to be changed to logged<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00315.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00437.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00323.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg00237.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== UPDATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow UPDATE tab SET ROW (col, ...) = (SELECT...)</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg01308.php <nowiki>Re: [PATCHES] extension for sql update</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00865.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00315.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00237.php <nowiki>Re: UPDATE using sub selects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Research self-referential UPDATEs that see inconsistent row versions in read-committed mode<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00507.php <nowiki>Concurrently updating an updatable view</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00016.php <nowiki>Re: Do we need a TODO? (was Re: Concurrently updating anupdatable view)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve performance of EvalPlanQual mechanism that rechecks already-updated rows<br />
|This is related to the previous item, which questions whether it even has the right semantics<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-09/msg00045.php <nowiki>BUG #4401: concurrent updates to a table blocks one update indefinitely</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-07/msg00302.php <nowiki>BUG #4945: Parallel update(s) gone wild</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ALTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have ALTER TABLE RENAME of a SERIAL column rename the sequence<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
* [http://archives.postgresql.org/message-id/CADLWmXUV4LbLhMZL8rYMhCy72aZZLB5BSARPQVgoX0BrxA0FFg@mail.gmail.com renaming implicit sequences]<br />
}}<br />
<br />
{{TodoItem<br />
|Have ALTER SEQUENCE RENAME rename the sequence name stored in the sequence table<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-09/msg00092.php <nowiki>BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00007.php <nowiki>Re: BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ALTER DOMAIN to modify the underlying data type}}<br />
<br />
{{TodoItem<br />
|Allow ALTER TABLESPACE to move the tablespace to different directories}}<br />
<br />
{{TodoItem<br />
|Allow moving system tables to other tablespaces, where possible<br />
|Currently non-global system tables must be in the default database tablespace. Global system tables can never be moved.}}<br />
<br />
{{TodoItem<br />
|Have ALTER INDEX update the name of a constraint using that index}}<br />
<br />
{{TodoItem<br />
|Allow column display reordering by recording a display, storage, and permanent id for every column?<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00782.php <nowiki>Re: column ordering, was Re: [PATCHES] Enums patch v2</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01029.php <nowiki>Column reordering in pg_dump</nowiki>]<br />
* http://archives.postgresql.org/message-id/1324412114-sup-9608@alvh.no-ip.org<br />
}}<br />
<br />
{{TodoItem<br />
|Allow deactivating (and reactivating) indexes via ALTER TABLE<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg01191.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add ALTER OPERATOR ... RENAME<br />
|needs to consider effects of changing operator precedence<br />
* [http://archives.postgresql.org/message-id/1322948781.26266.9.camel@vanquo.pezone.net Missing rename support]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add ALTER TABLE ... RENAME RULE<br />
* [http://archives.postgresql.org/message-id/1322948781.26266.9.camel@vanquo.pezone.net Missing rename support]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== CLUSTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Automatically maintain clustering on a table<br />
|This might require some background daemon to maintain clustering during periods of low usage. It might also require tables to be only partially filled for easier reorganization. Another idea would be to create a merged heap/index data file so an index lookup would automatically access the heap data too. A third idea would be to store heap rows in hashed groups, perhaps using a user-supplied hash function.<br />
* [http://archives.postgresql.org/pgsql-performance/2004-08/msg00350.php <nowiki>Equivalent praxis to CLUSTERED INDEX?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00155.php <nowiki>Re: Grouped Index Tuples</nowiki>]<br />
* http://community.enterprisedb.com/git/<br />
* [http://archives.postgresql.org/pgsql-performance/2009-10/msg00346.php <nowiki>Re: maintain_cluster_order_v5.patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Allow CLUSTER to be used on a partial index<br />
* http://www.postgresql.org/message-id/CAMkU%3D1zYwoHHsqJ8wfK3GdG_t_a6t4RK-GFDSKymQ0EGP%3DtypA@mail.gmail.com<br />
}} <br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== COPY ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report error lines and continue<br />
|This requires the use of a savepoint before each COPY line is processed, with ROLLBACK on COPY failure. <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00572.php <nowiki>Re: VLDB Features</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY FROM to create index entries in bulk<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00811.php <nowiki>Batch update of indexes on data loading</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY in CSV mode to control whether a quoted zero-length string is treated as NULL<br />
|Currently this is always treated as a zero-length string, which generates an error when loading into an integer column <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00905.php <nowiki>Re: [PATCHES] allow CSV quote in NULL</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve COPY performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00954.php <nowiki>Re: 8.3 / 8.2.6 restore comparison</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01882.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report errors sooner<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01169.php <nowiki>Timely reporting of COPY errors</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to handle other number formats<br />
|E.g. the German notation. Best would be something like WITH DECIMAL ','.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow a stalled COPY to exit if the backend is terminated<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00067.php <nowiki>Re: possible bug not in open items</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GRANT/REVOKE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SERIAL sequences to inherit permissions from the base table?}}<br />
<br />
{{TodoItem<br />
|Allow dropping of a role that has connection rights<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00736.php <nowiki>DROP ROLE dependency tracking ...</nowiki>]<br />
}}<br />
{{TodoEndSubsection}}<br />
<br />
=== DECLARE CURSOR ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent DROP TABLE from dropping a table referenced by its own open cursor?}}<br />
<br />
{{TodoItem<br />
|Provide some guarantees about the behavior of cursors that invoke volatile functions<br />
* [http://archives.postgresql.org/message-id/20997.1244563664@sss.pgh.pa.us Re: Cursor with hold emits the same row more than once across commits in 8.3.7]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== INSERT ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow INSERT/UPDATE of the system-generated oid value for a row}}<br />
<br />
{{TodoItemDone<br />
|In rules, allow VALUES() to contain a mixture of 'old' and 'new' references}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SHOW/SET ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM ANALYZE, and CLUSTER}}<br />
<br />
{{TodoItem<br />
|Rationalize the discrepancy between settings that use values in bytes and SHOW that returns the object count<br />
* [http://archives.postgresql.org/pgsql-docs/2008-07/msg00007.php <nowiki>Re: [ADMIN] shared_buffers and shmmax</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ANALYZE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE issue NOTICE messages when the estimated and actual row counts differ by a specified percentage}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE report rows as floating-point numbers<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01363.php <nowiki>explain analyze rows=%.0f</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00108.php <nowiki>Re: explain analyze rows=%.0f</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve how ANALYZE computes in-doubt tuples<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00771.php <nowiki>VACUUM/ANALYZE counting of in-doubt tuples</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Window Functions ===<br />
See {{messageLink|357.1230492361@sss.pgh.pa.us|TODO items for window functions}}.<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Support creation of user-defined window functions<br />
|We have the ability to create new window functions written in C. Is it<br />
worth the effort to create an API that would let them be written in PL/pgsql, etc?}}<br />
<br />
{{TodoItem<br />
|Implement full support for window framing clauses<br />
|In addition to done clauses described in the [http://developer.postgresql.org/pgdocs/postgres/sql-expressions.html#SYNTAX-WINDOW-FUNCTIONS latest doc], these clauses are not implemented yet.<br />
* RANGE BETWEEN ... PRECEDING/FOLLOWING<br />
* EXCLUDE<br />
}}<br />
<br />
{{TodoItem<br />
|Investigate tuplestore performance issues<br />
|The tuplestore_in_memory() thing is just a band-aid, we ought to try to solve it properly. tuplestore_advance seems like a weak spot as well.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00152.php <nowiki>tuplestore potential performance problem</nowiki>]<br />
}}<br />
<br />
{{TodoItem|Do we really need so much duplicated code between Agg and WindowAgg?}}<br />
<br />
{{TodoItem<br />
|Teach planner to evaluate multiple windows in the optimal order<br />
|Currently windows are always evaluated in the query-specified order.<br />
* http://archives.postgresql.org/message-id/3CDAD71E9D70417290FCF66F0178D1E1@amd64<br />
}}<br />
<br />
{{TodoItem<br />
|Implement DISTINCT clause in window aggregates<br />
|Some proprietary RDBMSs have implemented it already, so it helps with porting from those.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Integrity Constraints ==<br />
=== Keys ===<br />
<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve deferrable unique constraints for cases with many conflicts<br />
|The current implementation fires a trigger for each potentially conflicting row. This might not scale well for an update that changes many key values at once.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Referential Integrity ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add MATCH PARTIAL referential integrity}}<br />
<br />
{{TodoItem<br />
|Change foreign key constraint for array -&gt; element to mean element in array?<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01814.php <nowiki>foreign keys for array/period contains relationships</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when cascading referential triggers make changes on cascaded tables, seeing the tables in an intermediate state<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00174.php <nowiki>Re: [PATCHES] Work-in-progress referential action trigger timing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Are ri_KeysEqual checks in the RI enforcement triggers still necessary?<br />
* [http://archives.postgresql.org/pgsql-performance/2005-10/msg00458.php <nowiki>Re: Effects of cascading references in foreign keys</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Optimize referential integrity checks involving null values<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00744.php <nowiki>Can't ri_KeysEqual() consider two nulls as equal?</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Check Constraints ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Run check constraints only when affected columns are changed<br />
* http://archives.postgresql.org/message-id/1326055327.15293.13.camel@vanquo.pezone.net<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Server-Side Languages ==<br />
<br />
{{TodoItem<br />
|Add support for polymorphic arguments and return types to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add support for OUT and INOUT parameters to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add more fine-grained specification of functions taking arbitrary data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement stored procedures<br />
|This might involve the control of transaction state and the return of multiple result sets<br />
* [http://archives.postgresql.org/pgsql-general/2008-10/msg00454.php <nowiki>PL/pgSQL stored procedure returning multiple result sets (SELECTs)?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01375.php <nowiki>Proposal: real procedures again (8.4)</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00542.php<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-04/msg01149.php <nowiki>Gathering specs and discussion on feature (post 9.1)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow holdable cursors in SPI}}<br />
<br />
{{TodoItemEasy<br />
|Add SPI_gettypmod() to return a field's typemod from a TupleDesc<br />
* http://archives.postgresql.org/pgsql-hackers/2005-11/msg00250.php<br />
}}<br />
<br />
=== SQL-Language Functions ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Rethink query plan caching and timing of parse analysis within SQL-language functions<br />
|They should work more like plpgsql functions do ...<br />
* [http://archives.postgresql.org/pgsql-bugs/2011-05/msg00078.php <nowiki>Re: BUG #6019: invalid cached plan on inherited table</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/pgSQL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow handling of %TYPE arrays, e.g. tab.col%TYPE[]}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow listing of record column names, and access to record columns via variables, e.g. columns := r.(*), tval2 := r.(colname)</nowiki><br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00458.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00302.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00031.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow row and record variables to be set to NULL constants, and allow NULL tests on such variables<br />
|Because a row is not scalar, do not allow assignment from NULL-valued scalars.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00070.php <nowiki>NULL and plpgsql rows</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider keeping separate cached copies when search_path changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01009.php <nowiki>pl/pgsql Plan Invalidation and search_path</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULL row values vs. NULL rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg01758.php <nowiki>Null row vs. row of nulls in plpgsql</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg01973.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve PERFORM handling of WITH queries or document limitation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00309.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Perl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow regex operations in plperl using UTF8 characters in non-UTF8 encoded databases}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Python ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Develop a trusted variant of PL/Python.}}<br />
<br />
{{TodoItem<br />
|Create a new restricted execution class that will allow passing function arguments in as locals. Passing them as globals means functions cannot be called recursively.<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-02/msg01468.php <nowiki>Re: pl/python do not delete function arguments</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a DB-API compliant interface on top of the SPI interface<br />
* http://petereisentraut.blogspot.com/2011/11/plpydbapi-db-api-for-plpython.html<br />
}}<br />
<br />
{{TodoItem<br />
|For functions returning a setof record with a composite type, cache the I/O functions for the composite type<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02007.php<br />
}}<br />
<br />
{{TodoItem<br />
|Fix loss of information during conversion of numeric type to Python float}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Tcl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add table function support}}<br />
<br />
{{TodoItem<br />
|Check encoding validity of values passed back to Postgres in function returns, trigger tuple changes, and SPI calls.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Clients ==<br />
<br />
{{TodoItem<br />
|Add a function like pg_get_indexdef() that report more detailed index information<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00166.php <nowiki>BUG #3829: Wrong index reporting from pgAdmin III (v1.8.0 rev 6766-6767)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Split out pg_resetxlog output into pre- and post-sections<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg02040.php<br />
}}<br />
<br />
=== pg_ctl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve pg_ctl's detection of running postmasters<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg00000.php<br />
* http://archives.postgresql.org/pgsql-committers/2011-06/msg00001.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add additional shutdown modes, and change the default?<br />
* http://archives.postgresql.org/pgsql-hackers/2012-04/msg01283.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== psql ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have psql \ds show all sequences and their settings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00916.php <nowiki>Re: TODO item: Have psql show current values for a sequence</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00401.php <nowiki>Quick patch: Display sequence owner</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move psql backslash database information into the backend, use mnemonic commands?<br />
|This would allow non-psql clients to pull the same information out of the database as psql. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-01/msg00191.php <nowiki>Re: psql \d option list overloaded</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make psql's \d commands more consistent in their handling of schemas<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00014.php <nowiki>Re: psql and schemas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make psql's \d commands distinguish default privileges from no privileges<br />
|ACL displays were visibly different for the two cases before we "improved" them by using array_to_string.<br />
* [http://archives.postgresql.org/pgsql-bugs/2011-05/msg00082.php <nowiki>BUG #6021: There is no difference between default and empty access privileges with \dp</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consistently display privilege information for all objects in psql}}<br />
<br />
{{TodoItemEasy<br />
|\s without arguments (display history) fails with libedit, doesn't use pager either<br />
* [http://archives.postgresql.org/pgsql-bugs/2011-06/msg00114.php <nowiki> psql \s not working - OS X</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a \set variable to control whether \s displays line numbers<br />
|Another option is to add \# which lists line numbers, and allows command execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00255.php <nowiki>Re: psql possible TODO</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Include the symbolic SQLSTATE name in verbose error reports<br />
* [http://archives.postgresql.org/pgsql-general/2007-09/msg00438.php <nowiki>Re: Checking is TSearch2 query is valid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add prompt escape to display the client and server versions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00310.php <nowiki>WIP patch for TODO Item: Add prompt escape to display the client and server versions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to wrap column values at whitespace boundaries, rather than chopping them at a fixed width.<br />
|Currently, &quot;wrapped&quot; format chops values into fixed widths. Perhaps the word wrapping could use the same algorithm documented in the W3C specification. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00404.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
* http://www.w3.org/TR/CSS21/tables.html#auto-table-layout}}<br />
<br />
{{TodoItem<br />
|Support the ReST table output format<br />
|Details about the ReST format: http://docutils.sourceforge.net/rst.html#reference-documentation<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01007.php <nowiki>Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00518.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00609.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to print advice for people familiar with other databases<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01845.php <nowiki>MySQL-ism help patch for psql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ability to edit views with \ev<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00023.php <nowiki>Adding \ev view editor?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix FETCH_COUNT to handle SELECT ... INTO and WITH queries<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01565.php<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00192.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent psql from sending remaining single-line multi-statement queries after reconnecting<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00159.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01283.php<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Add \i option to bring in the specified file as a quoted literal<br />
|This would be useful for creating functions and other areas. Details still need to be worked out.<br />
* http://archives.postgresql.org/pgsql-bugs/2011-02/msg00016.php<br />
* http://archives.postgresql.org/pgsql-bugs/2011-02/msg00020.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having psql -c read .psqlrc, for consistency<br />
|psql -f already reads .psqlrc<br />
}}<br />
<br />
{{TodoItem<br />
|Allow processing of multiple -f (file) options<br />
* http://www.postgresql.org/message-id/AANLkTikFpzrTRl6392GhatQdwlCWQTXFdSMxh5CP51iv@mail.gmail.com<br />
}}<br />
<br />
{{TodoItem<br />
|Improve line drawing characters<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00386.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider improving the continuation prompt<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01772.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve speed of tab completion by using LIKE<br />
* http://www.postgresql.org/message-id/20121012060345.GA29214@toroid.org<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== pg_dump / pg_restore ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|<nowiki>Add full object name to the tag field. eg. for operators we need '=(integer, integer)', instead of just '='.</nowiki>}}<br />
<br />
{{TodoItemEasy<br />
|Modify pg_dump to create skeleton views for reload (which are then updated via CREATE OR REPLACE VIEW) when views have circular dependencies. This should eliminate the need for the CREATE RULE "_RETURN" hack currently used to address this issue. Thread and additional information here:<br />
* [http://www.postgresql.org/message-id/25554.1360895028@sss.pgh.pa.us Description of change]<br />
|}}<br />
<br />
{{TodoItem<br />
|Add pg_dumpall custom format dumps?<br />
* [http://archives.postgresql.org/pgsql-general/2010-05/msg00509.php pg_dumpall custom format]<br />
|}}<br />
<br />
{{TodoItem<br />
|Avoid using platform-dependent locale names in pg_dumpall output<br />
|Using native locale names puts roadblocks in the way of porting a dump to another platform. One possible solution is to get<br />
CREATE DATABASE to accept some agreed-on set of locale names and fix them up to meet the platform's requirements.<br />
* http://archives.postgresql.org/message-id/21396.1241716688@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Allow selection of individual object(s) of all types, not just tables}}<br />
<br />
{{TodoItem<br />
|In a selective dump, allow dumping of an object and all its dependencies}}<br />
<br />
{{TodoItem<br />
|Add options like pg_restore -l and -L to pg_dump}}<br />
<br />
{{TodoItemDone<br />
|Add support for multiple pg_restore -t options, like pg_dump}}<br />
<br />
{{TodoItem<br />
|Stop dumping CASCADE on DROP TYPE commands in clean mode}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump --clean to drop roles that own objects or have privileges<br />
|tgl says: if this is about pg_dumpall, it's done as of 8.4. If it's really about pg_dump, what does it mean? pg_dump has no business dropping roles.}}<br />
<br />
{{TodoItem<br />
|Allow pg_restore to load different parts of the COPY data for a single table simultaneously}}<br />
<br />
{{TodoItem<br />
|Remove support for dumping from pre-7.3 servers<br />
|In 7.3 and later, we can get accurate dependency information from the server. pg_dump still contains a lot of crufty code<br />
to try to deal with the lack of dependency info in older servers, but the usefulness of maintaining that code grows small.}}<br />
<br />
{{TodoItem<br />
|Refactor handling of database attributes between pg_dump and pg_dumpall<br />
|Currently only pg_dumpall emits database attributes, such as ALTER DATABASE SET commands and database-level GRANTs.<br />
Many people wish that pg_dump would do that. One proposal is to let pg_dump issue such commands if the -C switch was used,<br />
but it's unclear whether that will satisfy the demand.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01031.php <nowiki>ALTER DATABASE vs pg_dump</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-05/msg00010.php summary of the issues]<br />
}}<br />
<br />
{{TodoItem<br />
|Change pg_dump so that a comment on the dumped database is applied to the loaded database, even if the database has a different name.<br />
|This will require new backend syntax, perhaps COMMENT ON CURRENT DATABASE. This is related to the previous item.}}<br />
<br />
{{TodoItem<br />
|Allow parallel restore of tar dumps<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-02/msg01154.php <nowiki>Re: parallel restore</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Preserve sparse storage of large objects over dump/restore<br />
* [http://archives.postgresql.org/message-id/18789.1349750451@sss.pgh.pa.us <nowiki>TODO item: teach pg_dump about sparsely-stored large objects</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ecpg ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Docs<br />
|Document differences between ecpg and the SQL standard and information about the Informix-compatibility module.}}<br />
<br />
{{TodoItem<br />
|Solve cardinality &gt; 1 for input descriptors / variables?}}<br />
<br />
{{TodoItem<br />
|Add a semantic check level, e.g. check if a table really exists}}<br />
<br />
{{TodoItem<br />
|fix handling of DB attributes that are arrays}}<br />
<br />
{{TodoItem<br />
|Fix nested C comments}}<br />
<br />
{{TodoItemEasy<br />
|sqlwarn[6] should be 'W' if the PRECISION or SCALE value specified}}<br />
<br />
{{TodoItem<br />
|Make SET CONNECTION thread-aware, non-standard?}}<br />
<br />
{{TodoItem<br />
|Allow multidimensional arrays}}<br />
<br />
{{TodoItem<br />
|Implement COPY FROM STDIN}} <br />
<br />
{{TodoItem<br />
|Provide a way to specify size of a bytea parameter<br />
* [http://archives.postgresql.org/message-id/200906192131.n5JLVoMo044178@wwwmaster.postgresql.org <nowiki>BUG #4866: ECPG and BYTEA</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Fix small memory leaks in ecpg<br />
|Memory leaks in a short running application like ecpg are not really a problem, but make debugging more complicated}} <br />
<br />
{{TodoItem<br />
|Allow reuse of cursor name variables<br />
* [http://archives.postgresql.org/message-id/20100329113435.GA3430@feivel.credativ.lan <nowiki>Problems with variable cursorname in ecpg</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== libpq ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent PQfnumber() from lowercasing unquoted column names<br />
|PQfnumber() should never have been doing lowercasing, but historically it has so we need a way to prevent it}}<br />
<br />
{{TodoItem<br />
|Consider disallowing multiple queries in PQexec() as an additional barrier to SQL injection attacks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00184.php <nowiki>Re: InitPostgres and flatfiles question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add PQexecf() that allows complex parameter substitution<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01803.php <nowiki>Last minute mini-proposal (I know, know) for PQexecf()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add SQLSTATE and severity to errors generated within libpq itself<br />
* [http://archives.postgresql.org/pgsql-interfaces/2007-11/msg00015.php <nowiki>v8.1: Error severity on libpq PGconn*</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01425.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for interface/ipaddress binding to libpq<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01811.php <nowiki>SR/libpq - outbound interface/ipaddress binding</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== HTTP===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow access to the database via HTTP<br />
|See [[HTTP_API]]}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Triggers ==<br />
<br />
{{TodoItem<br />
|Improve storage of deferred trigger queue<br />
|Right now all deferred trigger information is stored in backend memory. This could exhaust memory for very large trigger queues. This item involves dumping large queues into files, or doing some kind of join to process all the triggers, some bulk operation, or a bitmap. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00876.php <nowiki>Re: BUG #4204: COPY to table with FK has memory leak</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00464.php <nowiki>Scaling up deferred unique checks and the after trigger queue</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2011-08/msg00023.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow triggers to be disabled in only the current session.<br />
|This is currently possible by starting a multi-statement transaction, modifying the system tables, performing the desired SQL, restoring the system tables, and committing the transaction. ALTER TABLE ... TRIGGER requires a table lock so it is not ideal for this usage.}}<br />
<br />
{{TodoItem<br />
|With disabled triggers, allow pg_dump to use ALTER TABLE ADD FOREIGN KEY<br />
|If the dump is known to be valid, allow foreign keys to be added without revalidating the data.}}<br />
<br />
{{TodoItem<br />
|Allow statement-level triggers to access modified rows}}<br />
<br />
{{TodoItem<br />
|When statement-level triggers are defined on a parent table, have them fire only on the parent table, and fire child table triggers only where appropriate<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01883.php <nowiki>Statement-level triggers and inheritance</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add event triggers<br />
}}<br />
<br />
{{TodoItem<br />
|Tighten trigger permission checks<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00564.php <nowiki>Security leak with trigger functions?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow BEFORE INSERT triggers on views<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg01466.php <nowiki>Re: Why can't I put a BEFORE EACH ROW trigger on a view?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add database and transaction-level triggers<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00451.php <nowiki>Proposal for db level triggers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00620.php <nowiki>triggers on prepare, commit, rollback... ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce locking requirements for creating a trigger<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00635.php <nowiki>Re: Change lock requirements for adding a trigger</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid requirement for "AFTER" trigger functions to return a value<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02384.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow creation of inline triggers<br />
* http://archives.postgresql.org/pgsql-hackers/2012-02/msg00708.php<br />
}}<br />
<br />
== Inheritance ==<br />
<br />
{{TodoItem<br />
|Allow inherited tables to inherit indexes, UNIQUE constraints, and primary/foreign keys<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-05/msg00285.php <nowiki>Partitioning/inherited tables vs FKs</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00039.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00305.php<br />
}}<br />
<br />
{{TodoItem<br />
|Honor UNIQUE INDEX on base column in INSERTs/UPDATEs on inherited table, e.g. INSERT INTO inherit_table (unique_index_col) VALUES (dup) should fail<br />
|The main difficulty with this item is the problem of creating an index that can span multiple tables.}}<br />
<br />
{{TodoItem<br />
|Determine whether ALTER TABLE / SET SCHEMA should work on inheritance hierarchies (and thus support ONLY). If yes, implement it.}}<br />
<br />
{{TodoItem<br />
|ALTER TABLE variants sometimes support recursion and sometimes not, but this is poorly/not documented, and the ONLY marker would then be silently ignored. Clarify the documentation, and reject ONLY if it is not supported.}}<br />
<br />
== Indexes ==<br />
<br />
{{TodoItem<br />
|Prevent index uniqueness checks when UPDATE does not modify the column<br />
|Uniqueness (index) checks are done when updating a column even if the column is not modified by the UPDATE.<br />
However, HOT already short-circuits this in common cases, so more work might not be helpful.<br />
* http://www.postgresql.org/message-id/CA+TgmoZOyaTanfEvNUdiHBCuu9Zh0JVP1e_UTVbx6Rvj9vFC9Q@mail.gmail.com<br />
}}<br />
<br />
{{TodoItem<br />
|Allow the creation of on-disk bitmap indexes which can be quickly combined with other bitmap indexes<br />
|Such indexes could be more compact if there are only a few distinct values. Such indexes can also be compressed. Keeping such indexes updated can be costly.<br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00512.php <nowiki>Re: Bitmap index AM</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01107.php <nowiki>Bitmap index thoughts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00265.php <nowiki>Stream bitmaps</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01214.php <nowiki>Re: Bitmapscan changes - Requesting further feedback</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00013.php <nowiki>Updated bitmap index patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00741.php <nowiki>Reviewing new index types (was Re: [PATCHES] Updated bitmap indexpatch)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01023.php <nowiki>Bitmap Indexes: request for feedback</nowiki>]<br />
* http://archives.postgresql.org/message-id/800923.27831.qm@web29010.mail.ird.yahoo.com <br />
}}<br />
<br />
{{TodoItem<br />
|Allow accurate statistics to be collected on indexes with more than one column or expression indexes, perhaps using per-index statistics<br />
* [http://archives.postgresql.org/pgsql-performance/2006-10/msg00222.php <nowiki>Re: Simple join optimized badly?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01131.php <nowiki>Stats for multi-column indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00741.php <nowiki>Cross-column statistics revisited</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01431.php <nowiki>Multi-Dimensional Histograms</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00913.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02179.php <br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00459.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02054.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01731.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00894.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-09/msg00679.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having a larger statistics target for indexed columns and expression indexes. <br />
}}<br />
<br />
{{TodoItem<br />
|Consider smaller indexes that record a range of values per heap page, rather than having one index entry for every heap row<br />
|This is useful if the heap is clustered by the indexed values. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00341.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01264.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00465.php <nowiki>Grouped Index Tuples / Clustered Indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-03/msg00163.php <nowiki>Bitmapscan changes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00014.php <nowiki>Re: GIT patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00487.php <nowiki>Re: Index Tuple Compression Approach?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01589.php <nowiki>Re: Index AM change proposals, redux</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add REINDEX CONCURRENTLY, like CREATE INDEX CONCURRENTLY<br />
|This is difficult because you must upgrade to an exclusive table lock to replace the existing index file. CREATE INDEX CONCURRENTLY does not have this complication. This would allow index compaction without downtime. <br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00289.php <nowiki>Re: When/if to Reindex</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2012-09/msg00911.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-10/msg00128.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multiple indexes to be created concurrently, ideally via a single heap scan<br />
|pg_restore allows parallel index builds, but it is done via subprocesses, and there is no SQL interface for this.<br />
Cluster could definitely benefit from this.<br />
* http://archives.postgresql.org/pgsql-performance/2011-04/msg00093.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider sorting entries before inserting into btree index<br />
* [http://archives.postgresql.org/pgsql-general/2008-01/msg01010.php <nowiki>Re: ATTN: Clodaldo was Performance problem. Could it be related to 8.3-beta4?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow creation of an index that can do comparisons to test if a value is between two column values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00757.php <nowiki>Proposal: temporal extension &quot;period&quot; data type</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using "effective_io_concurrency" for index scans<br />
|Currently only bitmap scans use this, which might be fine because most multi-row index scans use bitmap scans.<br />
* [http://www.postgresql.org/message-id/CAGTBQpZzf70n0PYJ%3DVQLd+jb3wJGo%3D2TXmY+SkJD6G_vjC5QNg@mail.gmail.com Prefetch index pages for B-Tree index scans]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem with btree page splits during checkpoints<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00052.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-09/msg00184.php<br />
}}<br />
<br />
{{TodoItem<br />
|[http://archives.postgresql.org/pgsql-hackers/2012-05/msg00669.php Support amgettuple() in GIN (useful for exclusion constraints)]<br />
}}<br />
<br />
{{TodoItem<br />
| Allow "loose" or "skip" scans on btree indexes in which the first column has low cardinality<br />
* http://archives.postgresql.org/pgsql-performance/2012-08/msg00159.php<br />
}}<br />
<br />
{{TodoItem<br />
| Make the planner's "special index operator" mechanism extensible<br />
* http://www.postgresql.org/message-id/27270.1364700924@sss.pgh.pa.us<br />
}}<br />
<br />
<br />
=== GIST ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add more GIST index support for geometric data types}}<br />
<br />
{{TodoItem<br />
|Allow GIST indexes to create certain complex index types, like digital trees (see Aoki)}}<br />
<br />
{{TodoItem<br />
|Fix performance issues in contrib/seg and contrib/cube GiST support<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904161633160.4053@aragorn.flymine.org GiST index performance]<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904221704470.22330@aragorn.flymine.org draft patch]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-05/msg00069.php <nowiki>Re: GiST index performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-06/msg00068.php <nowiki>GiST index performance</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|[http://archives.postgresql.org/message-id/4DC8D284-05CF-4E3D-9670-AC9A32C37A36@justatheory.com GiST index support for arrays]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Hash ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add UNIQUE capability to hash indexes}}<br />
<br />
{{TodoItem<br />
|Add hash WAL logging for crash recovery<br />
* http://archives.postgresql.org/pgsql-performance/2011-09/msg00196.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multi-column hash indexes}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Sorting ==<br />
<br />
{{TodoItem<br />
|Consider whether duplicate keys should be sorted by block/offset<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00558.php <nowiki>Remove hacks for old bad qsort() implementations?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider being smarter about memory and external files used during sorts<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01101.php <nowiki>Sorting Improvements for 8.4</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00045.php <nowiki>Re: Sorting Improvements for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider detoasting keys before sorting}}<br />
<br />
{{TodoItem<br />
|Allow sorts to use more available memory<br />
* http://archives.postgresql.org/pgsql-hackers/2007-11/msg01026.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01123.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg01957.php<br />
}}<br />
<br />
== Fsync ==<br />
<br />
{{TodoItem<br />
|Determine optimal fdatasync/fsync, O_SYNC/O_DSYNC options and whether fsync does anything<br />
|Ideally this requires a separate test program like /contrib/pg_test_fsync that can be run at initdb time or optionally later.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider sorting writes during checkpoint<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php <nowiki>Sorted writes in checkpoint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00050.php <nowiki>Re: Sorting writes during checkpoint</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg02012.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00278.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-01/msg00493.php<br />
}}<br />
<br />
== Cache Usage ==<br />
<br />
{{TodoItem<br />
|Provide a way to calculate an &quot;estimated COUNT(*)&quot;<br />
|Perhaps by using the optimizer's cardinality estimates or random sampling.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-11/msg00943.php <nowiki>Re: Improving count(*)</nowiki>]<br />
* http://wiki.postgresql.org/wiki/Slow_Counting<br />
}}<br />
<br />
{{TodoItem<br />
|Consider automatic caching of statements at various levels:<br />
* Parsed query tree<br />
* Query execute plan<br />
* Query results <br />
<br />
:<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00823.php <nowiki>Cached Query Plans (was: global prepared statements)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing internal areas (NUM_CLOG_BUFFERS) when shared buffers is increased<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-10/msg01419.php <nowiki>Re: slru.c race condition (was Re: TRAP: FailedAssertion(&quot;!((itemid)-&gt;lp_flags &amp; 0x01)&quot;,)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00030.php <nowiki>clog_buffers to 64 in 8.3?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00024.php <nowiki>CLOG Patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the amount of memory used by PrivateRefCount<br />
|<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00797.php <nowiki>PrivateRefCount (for 8.3)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00752.php <nowiki>Re: PrivateRefCount (for 8.3)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing higher priority queries to have referenced buffer cache pages stay in memory longer<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00562.php <nowiki>Re: How to keep a table in memory?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve cache lookup speed for sessions accessing many relations<br />
* http://archives.postgresql.org/pgsql-hackers/2012-11/msg00356.php<br />
}}<br />
<br />
== Vacuum ==<br />
<br />
{{TodoItem<br />
|Auto-fill the free space map by scanning the buffer cache or by checking pages written by the background writer<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-02/msg01125.php <nowiki>Dead Space Map</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00011.php <nowiki>Re: Automatic free space map filling</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow concurrent inserts to use recently created pages rather than creating new ones<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg00853.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having single-page pruning update the visibility map<br />
* <nowiki>https://commitfest.postgresql.org/action/patch_view?id=75</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02344.php <nowiki>Re: visibility maps and heap_prune</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve tracking of total relation tuple counts now that vacuum doesn't always scan the whole heap<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00531.php Partial vacuum versus pg_class.reltuples]<br />
}}<br />
<br />
{{TodoItem<br />
|Bias FSM towards returning free space near the beginning of the heap file, in hopes that empty pages at the end can be truncated by VACUUM<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01124.php <nowiki>FSM search modes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a more compact data representation for dead tuple locations within VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00143.php <nowiki>Re: Have vacuum emit a warning when it runs out of maintenance_work_mem</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more information in order to improve user-side estimates of dead space bloat in relations<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg01039.php <nowiki>Re: Bloated Table</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve locking behaviour of vacuum during trailing page truncation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00319.php<br />
* http://archives.postgresql.org/message-id/4D8DF88E.7080205@Yahoo.com<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce the number of table scans performed by vacuum<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg01119.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg00605.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-07/msg00624.php<br />
}}<br />
<br />
{{TodoItem<br />
|Vacuum Gin indexes in physically order rather than logical order<br />
* http://archives.postgresql.org/pgsql-hackers/2012-04/msg00443.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid creation of the free space map for small tables<br />
* http://archives.postgresql.org/pgsql-hackers/2011-11/msg01751.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-08/msg00552.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-08/msg00615.php<br />
}}<br />
<br />
=== Auto-vacuum ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Issue log message to suggest VACUUM FULL if a table is nearly empty?}}<br />
<br />
{{TodoItem<br />
|Prevent long-lived temporary tables from causing frozen-xid advancement starvation<br />
|The problem is that autovacuum cannot vacuum them to set frozen xids; only the session that created them can do that. <br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01645.php <nowiki>Re: AutoVacuum Behaviour Question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent autovacuum from running if an old transaction is still running from the last vacuum<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00899.php <nowiki>Re: Autovacuum and OldestXmin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have autoanalyze of parent tables occur when child tables are modified<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00137.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-10/msg00271.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow visibility map all-visible bits to be set even when an auto-ANALYZE is running<br />
* http://archives.postgresql.org/pgsql-hackers/2012-01/msg00356.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow parallel cores to be used by vacuumdb<br />
* [http://archives.postgresql.org/message-id/4F10A728.7090403@agliodbs.com vacuumdb -j]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve autovacuum tuning<br />
* http://www.postgresql.org/message-id/5078AD6B.8060802@agliodbs.com<br />
* http://www.postgresql.org/message-id/20130124215715.GE4528@alvh.no-ip.org<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Locking ==<br />
<br />
{{TodoItem<br />
|Fix priority ordering of read and write light-weight locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00893.php <nowiki>lwlocks and starvation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00905.php <nowiki>Re: lwlocks and starvation</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when multiple subtransactions of the same outer transaction hold different types of locks, and one subtransaction aborts<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg01011.php <nowiki>FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00001.php <nowiki>Re: FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00435.php <nowiki>Re: [PATCHES] [pgsql-patches] Phantom Command IDs, updated patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00773.php <nowiki>Re: savepoints and upgrading locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow UPDATEs on only non-referential integrity columns not to conflict with referential integrity locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00073.php <nowiki>Referential Integrity and SHARE locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add idle_in_transaction_timeout GUC so locks are not held for long periods of time}}<br />
<br />
{{TodoItem<br />
|Improve deadlock detection when a page cleaning lock conflicts with a shared buffer that is pinned<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-01/msg00138.php <nowiki>BUG #3883: Autovacuum deadlock with truncate?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-committers/2008-01/msg00365.php <nowiki>Re: pgsql: Add checks to TRUNCATE, CLUSTER, and REINDEX to prevent</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Detect deadlocks involving LockBufferForCleanup()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow finer control over who is cancelled in a deadlock<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01727.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Consider a lock timeout parameter<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00485.php <nowiki>SELECT ... FOR UPDATE [WAIT integer | NOWAIT] for 8.5</nowiki>]<br />
}}<br />
<br />
== Startup Time Improvements ==<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for backend creation<br />
|This would prevent the overhead associated with process creation. Most operating systems have trivial process creation time compared to database startup overhead, but a few operating systems (Win32, Solaris) might benefit from threading. Also explore the idea of a single session using multiple threads to execute a statement faster.}}<br />
<br />
{{TodoItem<br />
|Allow backends to change their database without restart<br />
|This allows for faster server startup.<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00843.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00336.php<br />
}}<br />
<br />
== Write-Ahead Log ==<br />
<br />
{{TodoItem<br />
|Eliminate need to write full pages to WAL before page modification<br />
|Currently, to protect against partial disk page writes, we write full page images to WAL before they are modified so we can correct any partial page writes during recovery. These pages can also be eliminated from point-in-time archive files. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-06/msg00655.php <nowiki>Re: Index Scans become Seq Scans after VACUUM ANALYSE</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg01191.php<br />
* [http://archives.postgresql.org/message-id/20120105061916.GB21048@fetter.org WIP double writes]<br />
* [http://archives.postgresql.org/message-id/4EFC449F02000025000441CD@gw.wicourts.gov double writes]<br />
* [http://archives.postgresql.org/message-id/20120110214344.GB21106@fetter.org Double-write with Fast Checksums]<br />
* [http://archives.postgresql.org/message-id/1962493974.656458.1327703514780.JavaMail.root@zimbra-prod-mbox-4.vmware.com double writes using "double-write buffer" approach]<br />
* http://archives.postgresql.org/pgsql-hackers/2012-10/msg01463.php<br />
}}<br />
<br />
{{TodoItem<br />
|When full page writes are off, write CRC to WAL and check file system blocks on recovery<br />
|If CRC check fails during recovery, remember the page in case a later CRC for that page properly matches. The difficulty is that hint bits are not WAL logged, meaning a valid page might not match the earlier CRC.}}<br />
<br />
{{TodoItem<br />
|Write full pages during file system write and not when the page is modified in the buffer cache<br />
|This allows most full page writes to happen in the background writer. It might cause problems for applying WAL on recovery into a partially-written page, but later the full page will be replaced from WAL.<br />
* [http://archives.postgresql.org/message-id/CAGvK12UST-tPhyLrSLuSpwFxZbAO79yYrhV2xaLmS2MkUxNUVQ@mail.gmail.com Page Checksums + Double Writes]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce WAL traffic so only modified values are written rather than entire rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01589.php <nowiki>Reduction in WAL for UPDATEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow WAL information to recover corrupted pg_controldata<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00025.php <nowiki>Re: [HACKERS] pg_resetxlog -r flag</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a way to reduce rotational delay when repeatedly writing last WAL page<br />
|Currently fsync of WAL requires the disk platter to perform a full rotation to fsync again. One idea is to write the WAL to different offsets that might reduce the rotational delay. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-11/msg00483.php <nowiki>500 tpsQL + WAL log implementation</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Speed WAL recovery by allowing more than one page to be prefetched<br />
|This should be done utilizing the same infrastructure used for prefetching in general to avoid introducing complex error-prone code in WAL replay. <br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00683.php <nowiki>Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00497.php <nowiki>Re: [GENERAL] Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg01279.php <nowiki>Read-ahead and parallelism in redo recovery</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve WAL concurrency by increasing lock granularity<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00556.php <nowiki>Reworking WAL locking</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Be more aggressive about creating WAL files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01325.php <nowiki>Re: PANIC caused by open_sync on Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-07/msg01075.php <nowiki>PreallocXlogFiles</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-04/msg00556.php <nowiki>WAL/PITR additional items</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have resource managers report the duration of their status changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01468.php <nowiki>Recovery of Multi-stage WAL actions</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Move pgfoundry's xlogdump to /contrib and have it rely more closely on the WAL backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00035.php <nowiki>xlogdump</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Close deleted WAL files held open in *nix by long-lived read-only backends<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01754.php <nowiki>Deleted WAL files held open by backends in Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00060.php <nowiki>Re: Deleted WAL files held open by backends in Linux</nowiki>]<br />
}}<br />
<br />
== Optimizer / Executor ==<br />
<br />
{{TodoItem<br />
|Improve selectivity functions for geometric operators}}<br />
<br />
{{TodoItem<br />
|Consider increasing the default values of from_collapse_limit, join_collapse_limit, and/or geqo_threshold<br />
* [http://archives.postgresql.org/message-id/4136ffa0905210551u22eeb31bn5655dbe7c9a3aed5@mail.gmail.com from_collapse_limit vs. geqo_threshold]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to display optimizer analysis using OPTIMIZER_DEBUG<br />
* http://archives.postgresql.org/pgsql-hackers/2012-08/msg00597.php<br />
}}<br />
<br />
{{TodoItem<br />
|Log statements where the optimizer row estimates were dramatically different from the number of rows actually found?}}<br />
<br />
{{TodoItem<br />
|Consider compressed annealing to search for query plans<br />
|This might replace GEQO.<br />
* http://archives.postgresql.org/message-id/15658.1241278636%40sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Improve use of expression indexes for ORDER BY <br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01553.php <nowiki>Resjunk sort columns, Heikki's index-only quals patch, and bug #5000</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Modify the planner to better estimate caching effects<br />
* http://archives.postgresql.org/pgsql-performance/2010-11/msg00117.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow shared buffer cache contents to affect index cost computations<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg01140.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow the CTE (Common Table Expression) optimization fence to be optionally disabled<br />
* http://archives.postgresql.org/pgsql-hackers/2012-09/msg00700.php<br />
* http://archives.postgresql.org/pgsql-performance/2012-11/msg00161.php<br />
}}<br />
<br />
=== Hashing ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Consider using a hash for joining to a large IN (VALUES ...) list<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00450.php <nowiki>Planning large IN lists</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single batch hash joins to preserve outer pathkeys<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00806.php Re: Potential Join Performance Issue]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|"lazy" hash tables - look up only the tuples that are actually requested<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid building the same hash table more than once during the same query<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid hashing for distinct and then re-hashing for hash join<br />
* [http://archives.postgresql.org/message-id/4136ffa0902191346g62081081v8607f0b92c206f0a@mail.gmail.com Re: Fixing Grittner's planner issues]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Background Writer ==<br />
<br />
{{TodoItem<br />
|Consider having the background writer update the transaction status hint bits before writing out the page<br />
|Implementing this requires the background writer to have access to system catalogs and the transaction status log.}}<br />
<br />
{{TodoItem<br />
|Consider adding buffers the background writer finds reusable to the free list <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
* [http://archives.postgresql.org/message-id/CA+U5nMKtvyDcV4zTr7bq7t6cA2nBfLxCJ8tQgVBnc5ddRPO+Bg@mail.gmail.com our buffer replacement strategy is kind of lame]<br />
}}<br />
<br />
{{TodoItem<br />
|Automatically tune bgwriter_delay based on activity rather then using a fixed interval<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
* [http://archives.postgresql.org/message-id/CA+U5nMKtvyDcV4zTr7bq7t6cA2nBfLxCJ8tQgVBnc5ddRPO+Bg@mail.gmail.com our buffer replacement strategy is kind of lame]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider whether increasing BM_MAX_USAGE_COUNT improves performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg01007.php <nowiki>Bgwriter LRU cleaning: we've been going at this all wrong</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Test to see if calling PreallocXlogFiles() from the background writer will help with WAL segment creation latency<br />
* [http://archives.postgresql.org/pgsql-patches/2007-06/msg00340.php <nowiki>Re: Load Distributed Checkpoints, final patch</nowiki>]<br />
}}<br />
<br />
== Concurrent Use of Resources ==<br />
<br />
{{TodoItem<br />
|Do async I/O for faster random read-ahead of data<br />
|Async I/O allows multiple I/O requests to be sent to the disk with results coming back asynchronously.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00820.php <nowiki>Asynchronous I/O Support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-09/msg00255.php <nowiki>Re: random_page_costs - are defaults of 4.0 realistic for SCSI RAID 1</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00027.php <nowiki>There's random access and then there's random access</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-01/msg00170.php <nowiki>Bitmap index scan preread using posix_fadvise (Was: There's random access and then there's random access)</nowiki>]<br />
The above patch is already applied as of 8.4, but it still remains to figure out how to handle plain indexscans effectively.<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-01/msg00806.php Problems with the patch submitted for posix_fadvise in index scans]<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better I/O utilization<br />
|This would allow a single query to make use of multiple I/O channels simultaneously. One idea is to create a background reader that can pre-fetch sequential and index scan pages needed by other backends. This could be expanded to allow concurrent reads from multiple devices in a partitioned table.<br />
* http://archives.postgresql.org/pgsql-performance/2011-02/msg00123.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-10/msg01139.php<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better CPU utilization<br />
|This would allow several CPUs to be used for a single query, such as for sorting or query execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00945.php <nowiki>Multi CPU Queries - Feedback and/or suggestions wanted!</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|SMP scalability improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00439.php <nowiki>Straightforward changes for increased SMP scalability</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00206.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
== TOAST ==<br />
<br />
{{TodoItem<br />
|Allow user configuration of TOAST thresholds<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00213.php <nowiki>Re: Proposed adjustments in MaxTupleSize and toastthresholds</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00082.php <nowiki>pg_lzcompress strategy parameters</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce unnecessary cases of deTOASTing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00895.php <nowiki>Re: [PATCHES] Eliminate more detoast copies for packed varlenas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce costs of repeat de-TOASTing of values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01096.php <nowiki>WIP patch: reducing overhead for repeat de-TOASTing</nowiki>]<br />
}}<br />
<br />
== Monitoring ==<br />
{{TodoItem<br />
|Expand pg_stat_activity for easier integration with monitoring tools<br />
|* http://archives.postgresql.org/message-id/4DFA13A5.2060200@2ndQuadrant.com<br />
}}<br />
<br />
{{TodoItem<br />
|Add column to pg_stat_activity that shows the progress of long-running commands like CREATE INDEX and VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00203.php <nowiki>EXPLAIN progress info</nowiki>]<br />
* The CLUSTER/VACUUM FULL implementation would also be useful to track this way<br />
}}<br />
<br />
{{TodoItem<br />
|Have pg_stat_activity display query strings in the correct client encoding<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00131.php <nowiki>pg_stats queries versus per-database encodings</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Expose pg_controldata via an SQL interface<br />
|Helpful for monitoring replicated databases<br />
* http://archives.postgresql.org/message-id/4B901D73.8030003@agliodbs.com<br />
* [http://archives.postgresql.org/message-id/4B959D7A.6010907@joeconway.com initial patch]<br />
}}<br />
<br />
{{TodoItem<br />
| Add entry creation timestamp column to pg_stat_replication<br />
* http://archives.postgresql.org/pgsql-hackers/2011-08/msg00694.php<br />
}}<br />
<br />
{{TodoItem<br />
| Allow reporting of stalls due to wal_buffer wrap-around<br />
* http://archives.postgresql.org/pgsql-hackers/2012-02/msg00826.php<br />
}}<br />
<br />
{{TodoItem<br />
| Restructure pg_stat_database columns tup_returned and tup_fetched to return meaningful values<br />
* http://www.postgresql.org/message-id/20121012060345.GA29214@toroid.org<br />
}}<br />
<br />
== Miscellaneous Performance ==<br />
<br />
{{TodoItem<br />
|Use mmap() rather than SYSV for shared buffers?<br />
|This would remove the requirement for SYSV SHM but would introduce portability issues. Anonymous mmap (or mmap to /dev/zero) is required to prevent I/O overhead. We could also consider mmap() for writing WAL.<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00750.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00756.php<br />
}}<br />
<br />
{{TodoItem<br />
|Rather than consider mmap()-ing in 8k pages, consider mmap()'ing entire files into a backend?<br />
|Doing I/O to large tables would consume a lot of address space or require frequent mapping/unmapping. Extending the file also causes mapping problems that might require mapping only individual pages, leading to thousands of mappings. Another problem is that there is no way to _prevent_ I/O to disk from the dirty shared buffers so changes could hit disk before WAL is written.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01239.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider ways of storing rows more compactly on disk:<br />
* Reduce the row header size?<br />
* Consider reducing on-disk varlena length from four bytes to two because a heap row cannot be more than 64k in length}}<br />
<br />
{{TodoItem<br />
|Consider transaction start/end performance improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00948.php <nowiki>Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow configuration of backend priorities via the operating system<br />
|Though backend priorities make priority inversion during lock waits possible, research shows that this is not a huge problem.<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg00493.php <nowiki>Priorities for users or queries?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing the minimum allowed number of shared buffers<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00157.php <nowiki>Re: [PATCH] Don't bail with legitimate -N/-B options</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider if CommandCounterIncrement() can avoid its AcceptInvalidationMessages() call<br />
* [http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php <nowiki>pgsql: Avoid incrementing the CommandCounter when</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider Cartesian joins when both relations are needed to form an indexscan qualification for a third relation<br />
* [http://archives.postgresql.org/pgsql-performance/2007-12/msg00090.php <nowiki>Re: TB-sized databases</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider not storing a NULL bitmap on disk if all the NULLs are trailing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00624.php <nowiki>Proposal for Null Bitmap Optimization(for Trailing NULLs)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-12/msg00109.php <nowiki>Re: [HACKERS] Proposal for Null Bitmap Optimization(for TrailingNULLs)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Sort large UPDATE/DELETEs so it is done in heap order<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01119.php <nowiki>Possible future performance improvement: sort updates/deletes by ctid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the I/O caused by updating tuple hint bits<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00847.php <nowiki>Hint Bits and Write I/O</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00199.php <nowiki>Re: [HACKERS] Hint Bits and Write I/O</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00695.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00792.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg01063.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01408.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01453.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid the requirement of freezing pages that are infrequently modified <br />
|If all rows on a page are visible, it is possible to set a bit in the visibility map (once the visibility map is 100% reliable) and not need to freeze the page, avoiding a page rewrite<br />
* http://archives.postgresql.org/message-id/4BF701CF.2090205@agliodbs.com<br />
* http://archives.postgresql.org/pgsql-hackers/2010-06/msg00082.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid reading in b-tree pages when replaying vacuum records in hot standby mode<br />
* [http://archives.postgresql.org/message-id/1272571938.4161.14739.camel@ebony <nowiki>Hot Standby tuning for btree_xlog_vacuum()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure truncation logic to be more resistant to failure<br />
|This also involves not writing dirty buffers for a truncated or dropped relation<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01032.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider adding logic to increase large tables by more than 8k<br />
|This would reduce file system fragmentation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00337.php<br />
}}<br />
<br />
== Miscellaneous Other ==<br />
<br />
{{TodoItem<br />
|Deal with encoding issues for filenames in the server filesystem<br />
* {{MessageLink|20090413184335.39BE.52131E4D@oss.ntt.co.jp|a proposed patch here}}<br />
* {{MessageLink|8484.1244655656@sss.pgh.pa.us|some issues about it here}}<br />
* {{MessageLink|20100107103740.97A5.52131E4D@oss.ntt.co.jp|Windows-specific patch here}}<br />
}}<br />
<br />
{{TodoItem<br />
|Deal with encoding issues in the output of localeconv()<br />
* [http://archives.postgresql.org/message-id/40c6d9160904210658y590377cfw6dbbecb53d2b8be0@mail.gmail.com bug report]<br />
* [http://archives.postgresql.org/message-id/49EF8DA0.90008@tpf.co.jp draft patch]<br />
* [http://archives.postgresql.org/message-id/21710.1243620986@sss.pgh.pa.us review of patch]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide schema name and other fields available from SQL GET DIAGNOSTICS in error reports<br />
* [http://archives.postgresql.org/message-id/dcc563d10810211907n3c59a920ia9eb7cd2a6d5ea58@mail.gmail.com <nowiki>How to get schema name which violates fk constraint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg00846.php <nowiki>patch - Report the schema along table name in a referential failure error message</nowiki>]<br />
* {{MessageLink|3191.1263306359@sss.pgh.pa.us|Re: NOT NULL violation and error-message}}<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00213.php <nowiki>the case for machine-readable error fields</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Provide [http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html#LIBPQ-CONNECT-FALLBACK-APPLICATION-NAME fallback_application_name] in contrib/pgbench, oid2name, and dblink.<br />
* {{MessageLink|w2g9837222c1004070216u3bc46b3ahbddfdffdbfb46212@mail.gmail.com|fallback_application_name and pgbench}}<br />
}}<br />
<br />
{{TodoItem<br />
|Add 64-bit support to /contrib/pgbench<br />
* http://archives.postgresql.org/pgsql-hackers/2010-07/msg00153.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00705.php<br />
}}<br />
<br />
== Source Code ==<br />
<br />
{{TodoItemEasy<br />
|Remove warnings created by -Wcast-align}}<br />
<br />
{{TodoItem<br />
|Move platform-specific ps status display info from ps_status.c to ports}}<br />
<br />
{{TodoItemDone<br />
|Add optional CRC checksum to heap and index pages<br />
|One difficulty is how to prevent hint bit changes from affecting the computed CRC checksum.<br />
* http://archives.postgresql.org/message-id/19934.1226601952%40sss.pgh.pa.us<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00002.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01028.php <nowiki>double-buffering page writes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00524.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01101.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00011.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00249.php<br />
* http://archives.postgresql.org/message-id/20111221215913.GA4536@fetter.org<br />
* http://archives.postgresql.org/message-id/CA+U5nMJzQyxcObkpNAf1SYTX-gO_Mom3O9JXHnGpxRo1kXJ7ww@mail.gmail.com<br />
* http://archives.postgresql.org/pgsql-hackers/2012-01/msg00128.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-01/msg00113.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-02/msg00172.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-03/msg00001.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-03/msg00188.php<br />
* http://www.postgresql.org/message-id/1352422901.31259.28.camel@sussancws0025<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a faster CRC32 algorithm<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01112.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow cross-compiling by generating the zic database on the target system}}<br />
<br />
{{TodoItem<br />
|Improve NLS maintenance of libpgport messages linked onto applications}}<br />
<br />
{{TodoItem<br />
|Use UTF8 encoding for NLS messages so all server encodings can read them properly}}<br />
<br />
{{TodoItem<br />
|Allow creation of universal binaries for Darwin<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00884.php <nowiki>Getting to universal binaries for Darwin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider GnuTLS if OpenSSL license becomes a problem<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00892.php<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00040.php <nowiki>[PATCH] Add support for GnuTLS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01213.php <nowiki>TODO: GNU TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider making NAMEDATALEN more configurable in future releases}}<br />
<br />
{{TodoItem<br />
|Research use of signals and sleep wake ups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00003.php <nowiki>Restartable signals 'n all that</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow C++ code to more easily access backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00302.php <nowiki>Mostly Harmless: Welcoming our C++ friends</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider simplifying how memory context resets handle child contexts<br />
* [http://archives.postgresql.org/pgsql-patches/2007-08/msg00067.php <nowiki>Re: Memory leak in nodeAgg</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Create three versions of libpgport to simplify client code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00154.php <nowiki>8.4 TODO item: make src/port support libpq and ecpg directly</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve detection of shared memory segments being used by others by checking the SysV shared memory field 'nattch'<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00656.php <nowiki>postgresql in FreeBSD jails: proposal</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00673.php <nowiki>Re: postgresql in FreeBSD jails: proposal</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the non-threaded Avahi service discovery protocol<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00939.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00097.php <nowiki>Re: Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg01211.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00001.php <nowiki>Re: [HACKERS] Avahi support for Postgresql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce data row alignment requirements on some 64-bit systems<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00369.php <nowiki>[WIP] Reduce alignment requirements on 64-bit systems.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure TOAST internal storage format for greater flexibility<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00049.php <nowiki>Re: PG_PAGE_LAYOUT_VERSION 5 - time for change</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Add regression tests for pg_dump/restore<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01967.php <nowiki>"make install-check-pg_dump" target in src/regress]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Research different memory allocation methods for lists<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01467.php <br />
}}<br />
<br />
{{TodoItem<br />
| Consider removing the attribute options cache<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00039.php<br />
}}<br />
<br />
{{TodoItem<br />
| Restructure /contrib section<br />
* http://archives.postgresql.org/pgsql-hackers/2011-06/msg00705.php<br />
}}<br />
<br />
{{TodoItem<br />
| Consider adding explicit huge page support<br />
* http://archives.postgresql.org/pgsql-hackers/2012-07/msg00123.php<br />
}}<br />
<br />
=== /contrib/pg_upgrade ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Handle large object comments<br />
|This is difficult to do because the large object doesn't exist when --schema-only is loaded.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using pg_depend for checking object usage in version.c<br />
}}<br />
<br />
{{TodoItem<br />
|If reindex is necessary, allow it to be done in parallel with pg_dump custom format<br />
}}<br />
<br />
{{TodoItem<br />
|Migrate pg_statistic by dumping it out as a flat file, so analyze is not necessary<br />
|pg_class.oid is not preserved so schema.tablename must be used.<br />
* [http://archives.postgresql.org/message-id/CAAZKuFaWdLkK8eozSAooZBets9y_mfo2HS6urPAKXEPbd-JLCA@mail.gmail.com pg_upgrade and statistics]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve testing, perhaps using the buildfarm<br />
|The buildfarm has access to multiple versions of PostgreSQL.<br />
}}<br />
<br />
{{TodoItem<br />
|Create machine-readable output of pg_controldata<br />
|This would avoid parsing its output. The problem is we need pg_controldata output from both the old and new clusters so we would need to support both formats.<br />
}}<br />
<br />
{{TodoItem<br />
|Find cleaner way to start/stop dedicated servers for upgrades<br />
* http://archives.postgresql.org/pgsql-hackers/2012-08/msg00275.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a way to run pg_upgrade on standby servers<br />
* http://archives.postgresql.org/pgsql-hackers/2012-07/msg00453.php<br />
* http://archives.postgresql.org/pgsql-hackers/2012-09/msg00056.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Windows ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Remove configure.in check for link failure when cause is found}}<br />
<br />
{{TodoItem<br />
|Remove readdir() errno patch when runtime/mingwex/dirent.c rev 1.4 is released}}<br />
<br />
{{TodoItem<br />
|Allow psql to use readline once non-US code pages work with backslashes}}<br />
<br />
{{TodoItem<br />
|Fix problem with shared memory on the Win32 Terminal Server}}<br />
<br />
{{TodoItem<br />
|Improve signal handling<br />
* [http://archives.postgresql.org/pgsql-patches/2005-06/msg00027.php <nowiki>Simplify Win32 Signaling code</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Convert MSVC build system to remove most batch files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00961.php <nowiki>MSVC build system</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support pgxs when using MSVC}}<br />
<br />
{{TodoItem<br />
|Fix MSVC NLS support, like for to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00485.php <nowiki>NLS on MSVC strikes back!</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php <nowiki>Fix for 8.3 MSVC locale (Was [HACKERS] NLS on MSVC strikes back!)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a correct rint() substitute on Windows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00808.php <nowiki>Minor bug in src/port/rint.c</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix global namespace issues when using multiple terminal server sessions<br />
* [http://archives.postgresql.org/message-id/48F3BFCC.8030107@dunslane.net problems with Windows global namespace]}}<br />
<br />
{{TodoItem<br />
|Change from the current autoconf/gmake build system to cmake<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01869.php <nowiki>About CMake (was Re: [COMMITTERS] pgsql: Append major version number and for libraries soname major)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve consistency of path separator usage<br />
* http://archives.postgresql.org/message-id/49C0BDC5.4010002@hagander.net<br />
}}<br />
<br />
{{TodoItem<br />
|Fix cross-compiling on Windows<br />
* http://archives.postgresql.org/pgsql-bugs/2010-10/msg00110.php<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce file statistics overhead on directory reads<br />
* http://www.postgresql.org/message-id/1338325561.82125.YahooMailNeo@web39304.mail.mud.yahoo.com<br />
}}<br />
<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Wire Protocol Changes ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dynamic character set handling}}<br />
<br />
{{TodoItem<br />
|Let the client indicate character encoding of database names, user names, and passwords<br />
* http://www.postgresql.org/message-id/16160.1360540050@sss.pgh.pa.us}}<br />
<br />
{{TodoItem<br />
|Add decoded type, length, precision}}<br />
<br />
{{TodoItem<br />
|Mark result columns as known-not-null when possible<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-11/msg01029.php <nowiki>Adding nullable indicator to Describe</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more control over planner treatment of statements being prepared}}<br />
<br />
{{TodoItem<br />
|Use compression<br />
|If SSL is used, hopefully avoid the overhead of key negotiation and encryption<br />
* http://archives.postgresql.org/pgsql-hackers/2012-06/msg00793.php<br />
}}<br />
<br />
{{TodoItem<br />
|Update clients to use data types, typmod, schema.table.column names of result sets using new statement protocol}}<br />
<br />
{{TodoItem<br />
|Set protocol for wire format negotiation<br />
* [http://archives.postgresql.org/message-id/CACMqXCKkGrGXxQhjHCKCe0B8hn6sTt-1sdgHZOSGQMxrusOsQA@mail.gmail.com GUC_REPORT for protocol tunables]<br />
}}<br />
<br />
{{TodoItem<br />
|Make sure upgrading to a 4.1 protocol version will actually work smoothly<br />
* [http://archives.postgresql.org/message-id/28307.1318255008@sss.pgh.pa.us Re: libpq, PQdescribePrepared -> PQftype, PQfmod, no PQnullable]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Documentation ==<br />
<br />
{{TodoItemEasy <br />
| Add contrib functions to the index<br />
* Add the functions and GUCs in the contrib modules to [http://www.postgresql.org/docs/current/static/sql-createindex.html the documentation index]: [http://archives.postgresql.org/message-id/50A2E173.6030404@2ndQuadrant.com per list discussion]<br />
}}<br />
<br />
{{TodoItem<br />
|Convert single quotes to apostrophes in the PDF documentation<br />
* [http://archives.postgresql.org/pgsql-docs/2007-12/msg00059.php <nowiki>SGML docs and pdf single-quotes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a manpage for postgresql.conf<br />
* {{messageLink|20080819194311.GH4428@alvh.no-ip.org|A smaller default postgresql.conf}}<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Change the manpage-generating toolchain to use the new XML-based docbook2x tools<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing documentation format from SGML to XML<br />
* [http://archives.postgresql.org/pgsql-docs/2006-12/msg00152.php <nowiki>Re: Authoring Tools WAS: Switching to XML</nowiki>]<br />
* http://archives.postgresql.org/pgsql-docs/2011-04/msg00020.php<br />
* http://wiki.postgresql.org/wiki/Switching_PostgreSQL_documentation_from_SGML_to_XML<br />
}}<br />
<br />
{{TodoItem<br />
|Document support for N<nowiki>' '</nowiki> national character string literals, if it matches the SQL standard<br />
* http://archives.postgresql.org/message-id/1275895438.1849.1.camel@fsopti579.F-Secure.com<br />
}}<br />
<br />
{{TodoItem<br />
|Add diagrams to the documentation<br />
* http://archives.postgresql.org/pgsql-docs/2010-07/msg00001.php<br />
}}<br />
<br />
== Exotic Features ==<br />
<br />
{{TodoItem<br />
|Add pre-parsing phase that converts non-ISO syntax to supported syntax<br />
|This could allow SQL written for other databases to run without modification.}}<br />
<br />
{{TodoItem<br />
|Allow plug-in modules to emulate features from other databases}}<br />
<br />
{{TodoItem<br />
|Add features of Oracle-style packages<br />
|A package would be a schema with session-local variables, public/private functions, and initialization functions. It is also possible to implement these capabilities in any schema and not use a separate &quot;packages&quot; syntax at all.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00384.php <nowiki>proposal for PL packages for 8.3.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing control of upper/lower case folding of unquoted identifiers<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-04/msg00818.php <nowiki>Bringing PostgreSQL torwards the standard regarding case folding</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg01527.php <nowiki>Re: [SQL] Case Preservation disregarding case sensitivity?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00849.php <nowiki>TODO Item: Consider allowing control of upper/lower case folding of unquoted, identifiers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add autonomous transactions<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00893.php <nowiki>autonomous transactions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Give query progress indication<br />
* [[Query progress indication]]<br />
}}<br />
<br />
{{TodoItem<br />
|Rethink our type system<br />
* [[Rethinking datatypes]]<br />
}}<br />
<br />
== Features We Do ''Not'' Want ==<br />
<br />
The following features have been discussed ad nauseum on the PostgreSQL mailing lists and the consensus has been that the project is not interested in them. As such, if you are going to bring them up as potential features, you will want to be familiar with all of the arguments against these features which have been previously made over the years. If you decide to work on such features anyway, you should be aware that you face a higher-than-normal barrier to get the Project to accept them.<br />
<br />
{{TodoItem<br />
|All backends running as threads in a single process (not wanted)<br />
|This eliminates the process protection we get from the current setup. Thread creation is usually the same overhead as process creation on modern systems, so it seems unwise to use a pure threaded model, and MySQL and DB2 have demonstrated that threads introduce as many issues as they solve. Threading specific operations such as I/O, seq scans, and connection management has been discussed and will probably be implemented to enable specific performance features. Moving to a threaded engine would also require halting all other work on PostgreSQL for one to two years.}}<br />
<br />
{{TodoItem<br />
|"Oracle-style" optimizer hints (not wanted)<br />
|Optimizer hints, as implemented in Oracle and other RDBMSes, are used to work around problems in the optimizer and introduce upgrade and maintenance issues. We would rather have such problems reported and fixed. We have discussed a more sophisticated system of per-class cost adjustment instead, but a specification remains to be developed. See [[OptimizerHintsDiscussion|Optimizer Hints Discussion]] for further information.}}<br />
<br />
{{TodoItem<br />
|Embedded server (not wanted)<br />
|While PostgreSQL clients runs fine in limited-resource environments, the server requires multiple processes and a stable pool of resources to run reliably and efficiently. Stripping down the PostgreSQL server to run in the same process address space as the client application would add too much complexity and failure cases. Besides, there are several very mature embedded SQL databases already available.}}<br />
<br />
{{TodoItem<br />
|Obfuscated function source code (not wanted)<br />
|Obfuscating function source code has minimal protective benefits because anyone with super-user access can find a way to view the code. At the same time, it would greatly complicate backups and other administrative tasks. To prevent non-super-users from viewing function source code, remove SELECT permission on pg_proc.<br />
* [http://archives.postgresql.org/pgsql-general/2008-09/msg00668.php <nowiki>Obfuscated stored procedures (was Re: Oracle and Postgresql)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Indeterminate behavior for the GROUP BY clause (not wanted)<br />
|At least one other database product allows specification of a subset of the result columns which GROUP BY would need to be able to provide predictable results; the server is free to return any value from the group. This is not viewed as a desirable feature. PostgreSQL 9.1 allows result columns that are not referenced by GROUP BY if a primary key for the same table is referenced in GROUP BY.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg00297.php <nowiki>Re: SQL compatibility reminder: MySQL vs PostgreSQL</nowiki>]<br />
}}<br />
<br />
</div><br />
<br />
[[Category:Todo]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Streaming_Replication&diff=18798Streaming Replication2013-01-05T17:44:55Z<p>Schmiddy: SPAM undo revision 18716 by User:Davidbooker2012</p>
<hr />
<div>'''Streaming Replication''' (SR) provides the capability to continuously ship and<br />
apply the [http://www.postgresql.org/docs/current/static/wal.html WAL XLOG] records to some number of standby servers in order to keep them current.<br />
<br />
This feature was added to PostgreSQL 9.0. The discussion below is a developer oriented one that contains some out of data information. Users of this feature should use the documentation for the feature or a setup tutorial instead:<br />
<br />
* [[Binary Replication Tutorial]] provides an introduction to using this replication feature.<br />
* [http://www.postgresql.org/docs/9.1/static/warm-standby.html 9.1 Replication Documentation]<br />
* [http://www.postgresql.org/docs/9.0/static/warm-standby.html 9.0 Replication Documentation]<br />
<br />
= Project =<br />
SR was developed for inclusion in PostgreSQL 9.0 by NTT OSS Center. The lead developer is [mailto:masao.fujii@gmail.com Masao Fujii]. [http://www.pgcon.org/2008/schedule/events/76.en.html Synchronous Log Shipping Replication Presentation] introduces the early design of the feature.<br />
<br />
= Usage =<br />
== Users Overview ==<br />
* '''Log-shipping'''<br />
** XLOG records generated in the primary are periodically shipped to the standby via the network.<br />
** In the existing warm standby, only records in a filled file are shipped, what's referred to as file-based log-shipping. In SR, XLOG records in partially-filled XLOG file are shipped too, implementing record-based log-shipping. This means the window for data loss in SR is usually smaller than in warm standby, unless the warm standby was also configured for record-based shipping (which is complicated to setup).<br />
** The content of XLOG files written to the standby are exactly the same as those on the primary. XLOG files shipped can be used for a normal recovery and PITR.<br />
* '''Multiple standbys'''<br />
** More than one standby can establish a connection to the primary for SR. XLOG records are concurrently shipped to all these standbys. The delay/death of a standby does not harm log-shipping to other standbys.<br />
** The maximum number of standbys can be specified as a GUC variable.<br />
* '''Continuous recovery'''<br />
** The standby continuously replays XLOG records shipped without using pg_standby.<br />
** XLOG records shipped are replayed as soon as possible without waiting until XLOG file has been filled. The combination of [[Hot Standby]] and SR would make the latest data inserted into the primary visible in the standby almost immediately.<br />
** The standby periodically removes old XLOG files which are no longer needed for recovery, to prevent excessive disk usage.<br />
* '''Setup'''<br />
** The start of log-shipping does not interfere with any query processing on the primary.<br />
** The standby can be started in various conditions.<br />
*** If there are XLOG files in archive directory and restore_command is supplied, at first those files are replayed. Then the standby requests XLOG records following the last applied one to the primary. This prevents XLOG files already present in the standby from being shipped again. Similarly, XLOG files in pg_xlog are also replayed before starting log-shipping.<br />
*** If there is no XLOG files on the standby, the standby requests XLOG records following the starting XLOG location of recovery (the redo starting location).<br />
* '''Connection settings and authentication'''<br />
** A user can configure the same settings as a normal connection to a connection for SR (e.g., keepalive, pg_hba.conf).<br />
* '''Activation'''<br />
** The standby can keep waiting for activation as long as a user likes. This prevents the standby from being automatically brought up by failure of recovery or network outage.<br />
* '''Progress report'''<br />
** The primary and standby report the progress of log-shipping in PS display.<br />
* '''Graceful shutdown'''<br />
** When smart/fast shutdown is requested, the primary waits to exit until XLOG records have been sent to the standby, up to the shutdown checkpoint record.<br />
<br />
== Restrictions ==<br />
* '''Synchronous log-shipping'''<br />
** By default, SR supports operates in asynchronous manner, so the commit command might return a "success" to a client before the corresponding XLOG records are shipped to the standby. To enable synchronous replication, see [http://www.postgresql.org/docs/current/static/warm-standby.html#SYNCHRONOUS-REPLICATION Synchronous Replication]<br />
* '''Replication beyond timeline'''<br />
** A user has to get a fresh backup whenever making the old standby catch up.<br />
* '''Clustering'''<br />
** Postgres doesn't provide any clustering feature.<br />
<br />
== How to Use ==<br />
* '''1.''' Install postgres in the primary and standby server as usual. This requires only ''configure'', ''make'' and ''make install''.<br />
* '''2.''' Create the initial database cluster in the primary server as usual, using ''initdb''.<br />
* '''3.''' Set up connections and authentication so that the standby server can successfully connect to the ''replication'' pseudo-database on the primary.<br />
$ $EDITOR postgresql.conf<br />
<br />
listen_addresses = '192.168.0.10'<br />
<br />
$ $EDITOR pg_hba.conf<br />
<br />
# The standby server must have superuser access privileges.<br />
host replication postgres 192.168.0.20/22 trust<br />
* '''4.''' Set up the streaming replication related parameters on the primary server.<br />
$ $EDITOR postgresql.conf<br />
<br />
# To enable read-only queries on a standby server, wal_level must be set to<br />
# "hot_standby". But you can choose "archive" if you never connect to the<br />
# server in standby mode.<br />
wal_level = hot_standby<br />
<br />
# Set the maximum number of concurrent connections from the standby servers.<br />
max_wal_senders = 5<br />
<br />
# To prevent the primary server from removing the WAL segments required for<br />
# the standby server before shipping them, set the minimum number of segments<br />
# retained in the pg_xlog directory. At least wal_keep_segments should be<br />
# larger than the number of segments generated between the beginning of<br />
# online-backup and the startup of streaming replication. If you enable WAL<br />
# archiving to an archive directory accessible from the standby, this may<br />
# not be necessary.<br />
wal_keep_segments = 32<br />
<br />
# Enable WAL archiving on the primary to an archive directory accessible from<br />
# the standby. If wal_keep_segments is a high enough number to retain the WAL<br />
# segments required for the standby server, this is not necessary.<br />
archive_mode = on<br />
archive_command = 'cp %p /path_to/archive/%f'<br />
* '''5.''' Start postgres on the primary server.<br />
* '''6.''' Make a base backup by copying the primary server's data directory to the standby server.<br />
$ psql -c "SELECT pg_start_backup('label', true)"<br />
$ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid<br />
$ psql -c "SELECT pg_stop_backup()"<br />
* '''7.''' Set up replication-related parameters, connections and authentication in the standby server like the primary, so that the standby might work as a primary after failover.<br />
* '''8.''' Enable read-only queries on the standby server. But if wal_level is ''archive'' on the primary, leave hot_standby unchanged (i.e., off).<br />
$ $EDITOR postgresql.conf<br />
<br />
hot_standby = on<br />
* '''9.''' Create a recovery command file in the standby server; the following parameters are required for streaming replication.<br />
$ $EDITOR recovery.conf<br />
# Note that recovery.conf must be in $PGDATA directory.<br />
<br />
# Specifies whether to start the server as a standby. In streaming replication,<br />
# this parameter must to be set to on.<br />
standby_mode = 'on'<br />
<br />
# Specifies a connection string which is used for the standby server to connect<br />
# with the primary.<br />
primary_conninfo = 'host=192.168.0.10 port=5432 user=postgres'<br />
<br />
# Specifies a trigger file whose presence should cause streaming replication to<br />
# end (i.e., failover).<br />
trigger_file = '/path_to/trigger'<br />
<br />
# Specifies a command to load archive segments from the WAL archive. If<br />
# wal_keep_segments is a high enough number to retain the WAL segments<br />
# required for the standby server, this may not be necessary. But<br />
# a large workload can cause segments to be recycled before the standby<br />
# is fully synchronized, requiring you to start again from a new base backup.<br />
restore_command = 'cp /path_to/archive/%f "%p"'<br />
* '''10.''' Start postgres in the standby server. It will start streaming replication.<br />
* '''11.''' You can calculate the replication lag by comparing the current WAL write location on the primary with the last WAL location received/replayed by the standby. They can be retrieved using ''pg_current_xlog_location'' on the primary and the ''pg_last_xlog_receive_location''/''pg_last_xlog_replay_location'' on the standby, respectively.<br />
$ psql -c "SELECT pg_current_xlog_location()" -h192.168.0.10 (primary host)<br />
pg_current_xlog_location <br />
--------------------------<br />
0/2000000<br />
(1 row)<br />
<br />
$ psql -c "select pg_last_xlog_receive_location()" -h192.168.0.20 (standby host)<br />
pg_last_xlog_receive_location <br />
-------------------------------<br />
0/2000000<br />
(1 row)<br />
<br />
$ psql -c "select pg_last_xlog_replay_location()" -h192.168.0.20 (standby host)<br />
pg_last_xlog_replay_location <br />
------------------------------<br />
0/2000000<br />
(1 row)<br />
* '''12.''' You can also check the progress of streaming replication by using ''ps'' command.<br />
# The displayed LSNs indicate the byte position that the standby server has<br />
# written up to in the xlogs.<br />
[primary] $ ps -ef | grep sender<br />
postgres 6879 6831 0 10:31 ? 00:00:00 postgres: wal sender process postgres 127.0.0.1(44663) streaming 0/2000000<br />
<br />
[standby] $ ps -ef | grep receiver<br />
postgres 6878 6872 1 10:31 ? 00:00:01 postgres: wal receiver process streaming 0/2000000<br />
* How to do failover<br />
** Create the trigger file in the standby after the primary fails.<br />
* How to stop the primary or the standby server<br />
** Shut down it as usual (''pg_ctl stop'').<br />
* How to restart streaming replication after failover<br />
** Repeat the operations from '''6th'''; making a fresh backup, some configurations and starting the original primary as the standby. The primary server doesn't need to be stopped during these operations.<br />
* How to restart streaming replication after the standby fails<br />
** Restart postgres in the standby server after eliminating the cause of failure.<br />
* How to disconnect the standby from the primary<br />
** Create the trigger file in the standby while the primary is running. Then the standby would be brought up.<br />
* How to re-synchronize the stand-alone standby after isolation<br />
** Shut down the standby as usual. And repeat the operations from '''6th'''.<br />
* If you have more than one slave, promoting one will break the other(s). Update their recovery.conf settings to point to the new master, set recovery_target_timeline to 'latest', scp/rsync the pg_xlog directory, and restart the slave.<br />
<br />
= Todo =<br />
== v9.0 ==<br />
<br />
Moved to [[PostgreSQL_9.0_Open_Items]]<br />
<br />
=== Committed ===<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01455.php Retrying from archive and some refactoring around Read/FetchRecord().] - [http://archives.postgresql.org/pgsql-committers/2010-01/msg00395.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg02601.php SR wrongly treats the WAL-boundary.] - [http://archives.postgresql.org/pgsql-committers/2010-01/msg00396.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01715.php Adjust SR for some later changes about wal-skipping.] - [http://archives.postgresql.org/pgsql-committers/2010-01/msg00399.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00024.php VACUUM FULL unexpectedly writes an XLOG UNLOGGED record.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00038.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01754.php Add a message type header.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00037.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01536.php Documentation: Add a new "Replication" chapter.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00115.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00350.php Failed assertion during recovery of partial WAL file.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00124.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00712.php A PANIC error might occur in the standby because of a partially-filled archived WAL file.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00137.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00330.php Improve the standby messages.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00140.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01672.php pq_getbyte_if_available() is not working because the win32 socket emulation layer simply wasn't designed to deal with non-blocking sockets.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00198.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01488.php Walsender might emit unfit messages.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00239.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01236.php Streaming replication on win32, still broken.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00270.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00992.php Create new section for recovery.conf.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00295.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01824.php Assertion failure in walreceiver.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00356.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01717.php Forbid a startup of walsender during recovery, and emit a suitable message? Or allow walsender to be started also during recovery?] - [http://archives.postgresql.org/message-id/20100316090955.9A5107541D0@cvs.postgresql.org commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01003.php How do we clean down the archive without using pg_standby?] - [http://archives.postgresql.org/message-id/20100318091718.BC14D7541D0@cvs.postgresql.org commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01510.php File-based log shipping without pg_standby doesn't replay the WAL files in pg_xlog.] - [http://archives.postgresql.org/pgsql-committers/2010-03/msg00356.php commit]<br />
<br />
== v9.1 ==<br />
=== Synchronization capability ===<br />
* Introduce the replication mode which can control how long transaction commit waits for replication before the commit command returns a "success" to a client. The valid modes are ''async'', ''recv'' and ''fsync''.<br />
** ''async'' doesn't make transaction commit wait for replication, i.e., asynchronous replication.<br />
** ''recv'' or ''fsync'' makes transaction commit wait for XLOG to be received or fsynced by the standby, respectively.<br />
** (''apply'' makes transaction commit wait for XLOG to be replayed by the standby. This mode will be supported in v9.2 or later)<br />
** The replication mode is specified in recovery.conf of the standby as well as other parameters for replication.<br />
*** The startup process reads the replication mode from recovery.conf and shares it to walreceiver via new shared-memory variable.<br />
*** Walreceiver also shares it to walsender by using the replication handshake message (existing protocol needs to be extended).<br />
** Based on the replication mode, walreceiver sends the reply meaning that replication is done up to the specified location to the primary.<br />
*** In async, walreceiver doesn't need to send any reply other than end-of-replication message.<br />
*** In recv or fsync, walreceiver sends the reply just after receiving or flushing XLOG, respectively.<br />
*** New message type for the reply needs to be defined. The reply is sent as CopyData message.<br />
** Walreceiver writes all the outstanding XLOG to disk before shutting down.<br />
** Walsender receives the reply from the standby, updates the location of the last record replicated, and announces completion of replication.<br />
*** New shared-memory variable to keep that location is required.<br />
** When processing the commit command, backend waits for XLOG to be replicated to only the standbys which are in the recv or fsync replication mode.<br />
*** Also smart shutdown waits for XLOG of shutdown checkpoint to be replicated.<br />
* Required optimization<br />
** Walsender should send outstanding XLOG without waiting wal_sender_delay.<br />
*** When processing the commit command, backend signals walsender to send outstanding XLOG immediately.<br />
** Backend should exit the wait loop as soon as the reply arrives at the primary.<br />
*** When receiving the reply, walsender signals backends to get up from the sleep and determine whether to exit the wait loop by checking the location of the last XLOG replicated.<br />
*** Only backends waiting for XLOG to be replicated up to the location contained in the reply are sent the signal.<br />
** Walsender waits for the signal from backends and the reply from the standby at the same time, by using select/poll.<br />
** Walsender reads XLOG from not only disk but also shared memory (wal buffers).<br />
** Walreceiver should flush XLOG file only when XLOG file is switched or the related page is flushed.<br />
*** When startup process or bgwriter flushes the buffer page, it checks whether the related XLOG has already been flushed via shared memory (location of the last XLOG flushed).<br />
*** It flushes the buffer page, if XLOG file has already been flushed.<br />
*** It signals walreceiver to flush XLOG file immediately and waits for the flush to complete, if XLOG file has not been flushed yet.<br />
** While the standby is catching up with the primary, those servers should ignore the replication mode and perform asynchronous replication.<br />
*** After those servers have almost gotten into synchronization, they perform replication based on the specified replication mode.<br />
*** New replication states like 'catching-up', 'sync', etc need to be defined, and the state machine for them is required on both servers.<br />
*** Current replication state can be monitored on both servers via SQL.<br />
* Required timeout<br />
** Add new parameter replication_timeout which is the maximum time to wait until XLOG is replicated to the standby.<br />
** Add new parameter (replication_timeout_action) to specify the reaction to replication_timeout.<br />
<br />
== Future release ==<br />
* '''Synchronization capability'''<br />
** Introduce the synchronization mode which can control how long transaction commit waits for replication before the commit command returns a "success" to a client. The valid modes are ''async'', ''recv'', ''fsync'' and ''apply''.<br />
*** ''async'' doesn't make transaction commit wait for replication, i.e., asynchronous replication.<br />
*** ''recv'', ''fsync'' and ''apply'' makes transaction commit wait for XLOG records to be received, fsynced and applied on the standby, respectively.<br />
** Change walsender to be able to read XLOG from not only the disk but also shared memory.<br />
** Add new parameter replication_timeout which is the maximum time to wait until XLOG records are replicated to the standby.<br />
** Add new parameter (replication_timeout_action) to specify the reaction to replication_timeout.<br />
* '''Monitoring'''<br />
** Provide the capability to check the progress and gap of streaming replication via one query. A collaboration of HS and SR is necessary to provide that capability on the standby side.<br />
** Provide the capability to check if the specified repliation is in progress via a query. Also more detailed status information might be necessary, e.g, the standby is catching up now, has already gotten into sync, and so on.<br />
** Change the stats collector to collect the statistics information about replication, e.g., average delay of replication time.<br />
** Develop the tool to calculate the latest XLOG position from XLOG files. This is necessary to check the gap of replication after the server fails.<br />
** Also develop the tool to extract the user-readable contents from XLOG files. This is necessary to see the contents of the gap, and manually restore them.<br />
* '''Easy to Use'''<br />
** Introduce the parameters like:<br />
*** replication_halt_timeout - replication will halt if no data has been sent for this much time.<br />
*** replication_halt_segments - replication will halt if number of WAL files in pg_xlog exceeds this threshold.<br />
*** These parameters allow us to avoid disk overflow.<br />
** Add new feature which transfers also base backup via the direct connection between the primary and the standby.<br />
** Add new hooks like walsender_hook and walreceiver_hook to cooperate with the add-on program for compression like pglesslog.<br />
** Provide a graceful termination of replication via a query on the primary. On the standby, a trigger file mechanism already provides that capability.<br />
** Support replication beyond timeline. The timeline history files need to be shipped from the primary to the standby.<br />
* '''Robustness'''<br />
** Support keepalive in libpq. This is useful for a client and the standby to detect a failure of the primary immediately.<br />
** [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01536.php New privilege for replication.]<br />
*** Currently superuser privilege is required when the standby connects to the primary. But there is the complaint that we should add new privilege for replication and use it instead of superuser because current approach is not good for security.<br />
* '''Miscellaneous'''<br />
** Standalone walreceiver tool, which connects to the primary, continuously receives and writes XLOG records, independently from postgres server.<br />
** Cascade streaming replication. Allow walsender to send XLOG to another standby during recovery.<br />
** WAL archiving during recovery.<br />
<br />
[[Category:Replication]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Synchronous_replication&diff=18797Synchronous replication2013-01-05T17:37:03Z<p>Schmiddy: remove doubled title</p>
<hr />
<div>Synchronous replication is available starting in PostgreSQL 9.1 by enabling the [http://developer.postgresql.org/pgdocs/postgres/runtime-config-wal.html#GUC-SYNCHRONOUS-STANDBY-NAMES synchronous_standby_names] parameter. It includes user-controlled durability specified on the master using the [http://developer.postgresql.org/pgdocs/postgres/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT synchronous_commit] parameter. The design also provides high throughput by allowing concurrent processes to handle the WAL stream. <br />
<br />
== Design Notes ==<br />
See also [[Synchronous Replication 9/2010 Proposal]], though those notes pertain to a patch different than what has been committed.<br />
<br />
[[Category:Replication]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Synchronous_replication&diff=18796Synchronous replication2013-01-05T17:36:18Z<p>Schmiddy: Remove the notes describing Simon's 9/2010 patch, and put a link to the wiki page where they have been moved.</p>
<hr />
<div>= Synchronous Replication =<br />
Synchronous replication is available starting in PostgreSQL 9.1 by enabling the [http://developer.postgresql.org/pgdocs/postgres/runtime-config-wal.html#GUC-SYNCHRONOUS-STANDBY-NAMES synchronous_standby_names] parameter. It includes user-controlled durability specified on the master using the [http://developer.postgresql.org/pgdocs/postgres/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT synchronous_commit] parameter. The design also provides high throughput by allowing concurrent processes to handle the WAL stream. <br />
<br />
== Design Notes ==<br />
See also [[Synchronous Replication 9/2010 Proposal]], though those notes pertain to a patch different than what has been committed.<br />
<br />
[[Category:Replication]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Synchronous_Replication_9/2010_Proposal&diff=18795Synchronous Replication 9/2010 Proposal2013-01-05T17:21:21Z<p>Schmiddy: Import from Synchronous Replication</p>
<hr />
<div>= PAGE STATUS =<br />
This page serves as documentation for a Synchronous Replication [http://archives.postgresql.org/message-id/1284486530.1952.3976.camel@ebony patch] posted in September 2010 by Simon Riggs. Note, this proposal is somewhat different than the version which ended up being committed: see [[Synchronous replication]] for more details.<br />
<br />
=WHAT'S DIFFERENT ABOUT THIS PATCH?=<br />
<br />
The implementation in 9.1 includes several innovations, beyond [http://wiki.postgresql.org/wiki/Streaming_Replication Fujii Masao's work] providing an earlier synchronous replication implementation for PostgreSQL 9.0:<br />
<br />
* Low complexity of code on Standby<br />
* User control: All decisions to wait take place on master, allowing fine-grained control of synchronous replication. Max replication level can also be set on the standby.<br />
* Low bandwidth: Very small response packet size with no increase in number of responses when system is under high load means very little additional bandwidth required<br />
* Performance: Standby processes work concurrently to give good overall throughput on standby and minimal latency in all modes. 4 performance options don't interfere with each other, so offer different levels of performance/durability alongside each other.<br />
<br />
These are major wins for PostgreSQL project over and above the basic sync rep feature.<br />
<br />
=SYNCHRONOUS REPLICATION OVERVIEW=<br />
<br />
Synchronous replication offers the guarantee that all changes made by a<br />
transaction have been transferred to remote standby nodes. This is an<br />
extension to the standard level of durability offered by a transaction<br />
commit.<br />
<br />
When synchronous replication is requested the transaction will wait<br />
after it commits until it receives confirmation that the transfer has<br />
been successful. Waiting for confirmation increases the user's certainty<br />
that the transfer has taken place but it also necessarily increases the<br />
response time for the requesting transaction. Synchronous replication<br />
usually requires carefully planned and placed standby servers to ensure<br />
applications perform acceptably. Waiting doesn't utilise system<br />
resources, but transaction locks continue to be held until the transfer<br />
is confirmed. As a result, incautious use of synchronous replication<br />
will lead to reduced performance for database applications.<br />
<br />
It may seem that there is a simple choice between durability and<br />
performance. However, there is often a close relationship between the<br />
importance of data and how busy the database needs to be, so this is<br />
seldom a simple choice. With this patch, PostgreSQL now provides a range<br />
of features designed to allow application architects to design a system<br />
that has both good overall performance and yet good durability of the<br />
most important data assets.<br />
<br />
PostgreSQL allows the application designer to specify the durability<br />
level required via replication. This can be specified for the system<br />
overall, though it can also be specified for individual transactions.<br />
This allows to selectively provide highest levels of protection for<br />
critical data. <br />
<br />
For example we, an application might consist of two types of work:<br />
* 10% of changes are changes to important customer details<br />
* 90% of changes are less important data that the business can more easily survive if it is lost, such as chat messages between users.<br />
<br />
With sync replication options specified at the application level (on the<br />
master) we can offer sync rep for the most important changes, without<br />
slowing down the bulk of the total workload. Application level options<br />
are an important and practical tool for allowing the benefits of<br />
synchronous replication for high performance applications.<br />
<br />
Without sync rep options specified at app level, we would have a choice<br />
of either slowing down 90% of the workload because 10% of it is<br />
important. Or giving up our durability goals because of performance. Or<br />
splitting those two functions onto separate database servers so that we<br />
can set options differently on each. None of those 3 options is truly<br />
attractive.<br />
<br />
PostgreSQL also allows the system administrator the ability to specify<br />
the service levels offered by standby servers. This allows multiple<br />
standby servers to work together in various roles within a server farm.<br />
<br />
''Note: the information about the parameters used here reflects and earlier version of this feature, and needs to be updated to reflect the form it was committed into 9.1 as''<br />
<br />
Control of this feature relies on just 3 parameters:<br />
On the master we can set<br />
<br />
* synchronous_replication<br />
* synchronous_replication_timeout<br />
<br />
On the standby we can set<br />
<br />
* synchronous_replication_service<br />
<br />
These are explained in more detail in the following sections.<br />
<br />
=USER'S OVERVIEW=<br />
<br />
Two new USERSET parameters on the master control this <br />
* synchronous_replication = async (default) | recv | fsync | apply<br />
* synchronous_replication_timeout = 0+ (0 means never timeout)<br />
(default timeout 10sec)<br />
<br />
synchronous_replication = async is the default and means that no<br />
synchronisaton is requested and so the commit will not wait. This is the<br />
fastest setting. The word async is short for "asynchronous" and you may<br />
see the term asynchronous replication discussed.<br />
<br />
Other settings refer to progressively higher levels of durability. The<br />
higher the level of durability requested, the longer the wait for that<br />
level of durability to be achieved.<br />
<br />
The precise meaning of the synchronous_replication settings is<br />
* async - commit does not wait for a standby before replying to user<br />
* recv - commit waits until standby has received WAL<br />
* fsync - commit waits until standby has received and fsynced WAL<br />
* apply - commit waits until standby has received, fsynced and applied<br />
This provides a simple, easily understood mechanism - and one that in<br />
its default form is very similar to other RDBMS (e.g. Oracle).<br />
<br />
Note that in apply mode it is possible that the changes could be<br />
accessible on the standby before the transaction that made the change<br />
has been notified that the change is complete. Minor issue.<br />
<br />
Network delays may occur and the standby may also crash. If no reply is<br />
received within the timeout we raise a NOTICE and then return successful<br />
commit (no other action is possible). Note that it is possible to<br />
request that we never timeout, so if no standby is available we wait for<br />
it one to appear.<br />
<br />
When user commits, if the master does not have a currently connected<br />
standby offering the required level of replication it will pick the next<br />
best available level of replication. It is up to the sysadmin to provide<br />
sufficient range of standby nodes to ensure at least one is available to<br />
meet the requested service levels.<br />
<br />
If multiple standbys exist, the first standby to reply that the desired<br />
level of durability has been achieved will release the waiting commit on<br />
the master. Other options are available also via a plugin.<br />
<br />
==ADMINISTRATOR'S OVERVIEW==<br />
<br />
On the standby we specify the highest type of replication service<br />
offered by this standby server. This information is passed to the master<br />
server when the standby connects for replication.<br />
<br />
This allows sysadmins to designate preferred standbys. It also allows<br />
sysadmins to completely refuse to offer a synchronous replication<br />
service, allowing a master to explicitly avoid synchronisation across<br />
low bandwidth or high latency links.<br />
<br />
An additional parameter can be set in recovery.conf on the standby<br />
<br />
* synchronous_replication_service = async (def) | recv | fsync | apply<br />
<br />
<br />
= IMPLEMENTATION =<br />
<br />
Some aspects can be changed without significantly altering basic<br />
proposal, for example master-specified standby registration wouldn't<br />
really alter this very much.<br />
<br />
== STANDBY ==<br />
<br />
Master-controlled sync rep means that all user wait logic is centred on<br />
the master. The details of sync rep requests on the master are not sent<br />
to the standby, so there is no additional master to standby traffic nor<br />
standby-side bookkeeping overheads. It also reduces complexity of<br />
standby code.<br />
<br />
On the standby side the WAL Writer now operates during recovery. This<br />
frees the WALReceiver to spend more time sending and receiving messages,<br />
thereby minimising latency for users choosing the "recv" option. We now<br />
have 3 processes handling WAL in an asynchronous pipeline: WAL Receiver<br />
reads WAL data from the libpq connection then writes it to the WAL file,<br />
the WAL Writer then fsyncs the WAL file and then the Startup process<br />
replays the WAL. These processes act independently, so WAL pointers<br />
(LSNs) are defined as WALReceiverLSN >= WALWriterLSN >= StartupLSN<br />
<br />
For each new message WALReceiver gets from master we issue a reply. Each<br />
reply sends the current state of the 3 LSNs, so the reply message size<br />
is only 28 bytes. Replies are sent half-duplex, i.e. we don't reply<br />
while a new message is arriving.<br />
<br />
Note that there is absolutely not one reply per transaction on the<br />
master. The standby knows nothing about what has been requested on the<br />
master - replies always refer to the latest standby state and<br />
effectively batch the responses.<br />
<br />
We act according to the requested synchronous_replication_service<br />
* async - no replies are sent<br />
* recv - replies are sent upon receipt only<br />
* fsync - replies are sent upon receipt and following fsync only<br />
* apply - replies are sent following receipt, fsync and apply.<br />
<br />
Replies are sent at the next available opportunity.<br />
<br />
In apply mode, when the WALReceiver is completely quiet this means we<br />
send 3 reply messages - one at recv, one at fsync and one at apply. When<br />
WALreceiver is busy the volume of messages does *not* increase since the<br />
reply can't be sent until the current incoming message has been<br />
received, after which we were going to reply anyway so it is not an<br />
additional message. This means we piggyback an "apply" response onto a<br />
later "recv" reply. As a result we get minimum response times in *all*<br />
modes and maximum throughput is not impaired at all.<br />
<br />
When each new messages arrives from master the WALreceiver will write<br />
the new data to the WAL file, wake the WALwriter and then reply. Each<br />
new message from master receives a reply. If no further WAL data has<br />
been received the WALreceiver waits on the latch. If the WALReceiver is<br />
woken by WALWriter or Startup then it will reply to master with a<br />
message, even if no new WAL has been received.<br />
<br />
So in both recv, fsync and apply cases a message as soon as possible to<br />
master, so in all cases the wait time is minimised.<br />
<br />
When WALwriter is woken it sees if there is outstanding WAL data and if<br />
so fsyncs it and wakes both WALreceiver and Startup. When no WAL remains<br />
it waits on the latch.<br />
<br />
Startup process will wake WALreceiver when it has got to the end of the<br />
latest chunk of WAL. If no further WAL is available then it waits on its<br />
latch.<br />
<br />
== MASTER ==<br />
<br />
When user backends request sync rep they wait in a queue ordered by<br />
requested LSN. A separate queue exists for each request mode.<br />
<br />
WALSender receives the 3 LSNs from the standby. It then wakes backends<br />
in sequence from each queue.<br />
<br />
We provide a single wakeup rule: first WALSender to reply with the<br />
requested XLogRecPtr will wake the backend. This guarantees that the WAL<br />
data for the commit is transferred as requested to at least one standby.<br />
That is sufficient for the use cases we have discussed.<br />
<br />
More complex wakeup rules would be possible via a plugin.<br />
<br />
Wait timeout would be set by individual backends with a timer, just as<br />
we do for statement_timeout.<br />
<br />
= CODE =<br />
<br />
Total code to implement this is low. Breaks down into 5 areas<br />
* Zoltan's libpq changes, included almost verbatim; fairly modular, so easy to replace with something we like better<br />
* A new module syncrep.c and syncrep.h handle the backend wait/wakeup<br />
* Light changes to allow streaming rep to make appropriate calls<br />
* Small amount of code to allow WALWriter to be active in recovery<br />
* Parameter code<br />
No docs yet.<br />
<br />
The patch works on top of latches, though does not rely upon them for<br />
its bulk performance characteristics. Latches only improve response time<br />
for very low transaction rates; latches provide no additional throughput<br />
for medium to high transaction rates.<br />
<br />
= PERFORMANCE ANALYSIS =<br />
<br />
Since we reply to each new chunk sent from master, "recv" mode has<br />
absolutely minimal latency, especially since WALreceiver no longer<br />
performs majority of fsyncs, as in 9.0 code. WALreceiver does not wait<br />
for fsync or apply actions to complete before we reply, so fsync and<br />
apply modes will always wait at least 2 standby->master messages which<br />
is appropriate because those actions will typically occur much later. <br />
<br />
This response mechanism offers highest responsive performance achievable<br />
in "recv" mode and very good throughput under load. Note that the<br />
different modes do not interfere with each other and can co-exist<br />
happily while providing highest performance.<br />
<br />
Starting WALWriter is helpful, no matter what the<br />
synchronous_replication_service specified.<br />
<br />
Can we optimise the sending of reply messages so that only chunks that<br />
contain a commit deserve a reply? We could, but then we'd need to do<br />
extra work on the master to do bookkeeping of that. It would need to be<br />
demonstrated that there is a performance issue big enough to be worth<br />
the overhead on master and extra code.<br />
<br />
Is there an optimisation from reducing the number of options the standby<br />
provides? The architecture on the standby side doesn't rely heavily on<br />
the service level specified, nor does it rely in any way on the actual<br />
sync rep mode specified on master. No further simplification is<br />
possible.<br />
<br />
<br />
= NOT YET IMPLEMENTED =<br />
<br />
* Timeout code & NOTICE<br />
* Code and test plugin <br />
* Loops in walsender, walwriter and receiver treat shutdown incorrectly<br />
<br />
I haven't yet looked at Fujii's code for this, not even sure where it<br />
is, though hope to do so in the future. Zoltan's libpq code is the only<br />
part of that patch used.<br />
<br />
So far I have spent 3.5 days on this and expect to complete tomorrow. I<br />
think that throws out the argument that this proposal is too complex to<br />
develop in this release. <br />
<br />
= OTHER ISSUES =<br />
<br />
* How should master behave when we shut it down?<br />
* How should standby behave when we shut it down?<br />
<br />
[[Category:Replication]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=VACUUM_FULL&diff=18695VACUUM FULL2012-12-11T18:25:55Z<p>Schmiddy: fix link to CREATE INDEX</p>
<hr />
<div>= VACUUM vs VACUUM FULL =<br />
<br />
The <code>[http://www.postgresql.org/docs/current/static/sql-vacuum.html VACUUM]</code> command and associated autovacuum process are PostgreSQL's way of controlling MVCC bloat. The <code>VACUUM</code> command has two main forms of interest - ordinary <code>VACUUM</code>, and <code>VACUUM FULL</code>. These two commands are actually quite different and should not be confused.<br />
<br />
<code>VACUUM</code> scans a table, marking tuples that are no longer needed as free space so that they can be overwritten by newly inserted or updated data. See [[Introduction to VACUUM, ANALYZE, EXPLAIN, and COUNT]] and the PostgreSQL documentation on [http://www.postgresql.org/docs/current/static/mvcc.html MVCC] for a detailed explanation of this. Note that you should rarely need to use the <code>VACUUM</code> command directly on a modern PostgreSQL database, as [http://www.postgresql.org/docs/current/static/routine-vacuuming.html#AUTOVACUUM autovacuum] should take care of it for you if properly set up.<br />
<br />
<code>VACUUM FULL</code>, unlike <code>VACUUM</code>, touches data that has not been deleted. On pre-9.0 versions of PostgreSQL, it moves data into spaces earlier in the file that have been freed. Once it has created a free space at the end of the file, it truncates the file so that the OS knows that space is free and may be reused for other things. Moving in-use data around this way can have adverse side-effects, including taking heavy weight locks, increased i/o, and adding index bloat. On older systems, there are better ways to free space if you need to, and better ways to optimize tables (see below) so you should essentially never use <code>VACUUM FULL</code> on a pre-9.x system. Even on 9.x and above, the system is designed with the goal that you should never be running <code>VACUUM FULL</code> regularly, and doing so can have costs like huge WAL archive output and high loads on any streaming replication servers. <br />
<br />
For clarity, <b>9.0 changes VACUUM FULL</b>. As covered in the [http://developer.postgresql.org/pgdocs/postgres/sql-vacuum.html documentation], the VACUUM FULL implementation has been changed to one that's similar to using CLUSTER in older versions. This gives a slightly different set of trade-offs from the older VACUUM FULL described here. While the potential to make the database slower via index bloating had been removed by this change, it's still something you may want to avoid doing, due to the locking and general performance overhead of a VACUUM FULL.<br />
<br />
== When to use <code>VACUUM FULL</code> and when not to ==<br />
<br />
Many people, either based on misguided advice on the 'net or on the assumption that it must be "better", periodically run <code>VACUUM FULL</code> on their tables. This is generally not recommended and in some cases can make your database slower, not faster.<br />
<br />
<code>VACUUM FULL</code> is only needed when you have a table that is mostly dead rows - ie, the vast majority of its contents have been deleted. It should not be used for table optimization or periodic maintenance, as it's generally counterproductive. In most cases the freed space will be promptly re-allocated, possibly increasing file-system-level fragmentation and requiring file system space allocations that're slower than just re-using existing free space within a table.<br />
<br />
When you run <code>VACUUM FULL</code> on a table, that table is locked for the duration of the operation, so nothing else can work with the table. <code>VACUUM FULL</code> is <i>much</i> slower than a normal <i>VACUUM</i>, so the table may be unavailable for a while. <br />
<br />
More importantly, on pre-9.0 systems, while <code>VACUUM FULL</code> compacts the table, it does not compact the indexes - and in fact may increase their size, thus slowing them down, causing more disk I/O when the indexes are used, and increasing the amount of memory they require. A <code>REINDEX</code> may be required after <code>VACUUM FULL</code> on PostgreSQL versions older than 9.0. See the main documentation's [http://www.postgresql.org/docs/current/static/routine-vacuuming.html#VACUUM-BASICS notes on VACUUM vs VACUUM FULL].<br />
<br />
== What to use instead ==<br />
<br />
If you shouldn't use regularly use <code>VACUUM FULL</code> (or use it at all on versions older than 9.0) ... what should you be using?<br />
<br />
=== [http://www.postgresql.org/docs/current/static/routine-vacuuming.html#AUTOVACUUM Autovacuum] ===<br />
<br />
If [http://www.postgresql.org/docs/current/static/routine-vacuuming.html#AUTOVACUUM autovacuum] is running frequently enough and aggressively enough, your tables should never grow ("bloat") due to unreclaimed dead rows, so you should never need to return "dead" space to the OS.<br />
<br />
Autovacuum continues to improve dramatically with every PostgreSQL version, and is a very good reason to make sure you are running the latest version. For example, with 8.4 the free space map is now managed automatically, removing a no-longer-necessary tuning parameter and eliminating a major source of table bloat.<br />
<br />
If autovacuum isn't doing enough to keep your tables and indexes bloat-free, tune it, don't supplement it with manual vacuuming and reindexing. You may need to increase your free space map settings (pre-8.4), tune autovacuum to run more frequently, and/or tell autovacuum to vacuum certain frequently-updated tables more aggressively than others.<br />
<br />
=== <code>[http://www.postgresql.org/docs/current/static/sql-vacuum.html VACUUM]</code> ===<br />
<br />
Unless you need to return space to the OS so that other tables or other parts of the system can use that space, or you are trying to repair a table that has bloated out of control due to insufficient autovacuum, you should use <code>VACUUM</code> instead of <code>VACUUM FULL</code>.<br />
<br />
If you need to manually <code>VACUUM</code> your tables at any time other than when running major admin or update tasks that rewrite large parts of your tables, you probably don't have autovacuum set up well enough.<br />
<br />
=== <code>[http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER]</code> ===<br />
<br />
If you're trying to "optimise" your tables by packing them down and removing table bloat that's accumulated due to (say) insufficiently aggressive autovacuuming, or you're trying to return dead space in a table to the operating system, it's fine to use VACUUM FULL in PostgreSQL 9.0 and above. <br />
<br />
Consider setting a FILLFACTOR of less than the default 100, so the rewritten table has some free space pre-alloacated within it for updates and new inserts; otherwise you'll just get file system allocations as soon as you do anything to the table.<br />
<br />
In older versions, it's preferable to use <code>[http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER]</code>. It runs a <i>lot</i> faster than pre-9.0 <code>VACUUM FULL</code> and will compact and optimise the indexes as well as the table its self. However, you will need enough free space to hold all the in-use data from the table while <code>CLUSTER</code> runs. As with post-9.0 VACUUM FULL, a non-default FILLFACTOR may be wise.<br />
<br />
=== <code>[http://www.postgresql.org/docs/current/static/sql-truncate.html TRUNCATE TABLE]</code> ===<br />
<br />
If you've been using <code>VACUUM FULL</code> to free space from a table that's periodically completely emptied using <code>DELETE FROM tablename;</code> (without a <code>WHERE</code> clause), you can use <code>[http://www.postgresql.org/docs/current/static/sql-truncate.html TRUNCATE TABLE]</code> to replace those two steps with one much, <i>much</i> faster one.<br />
<br />
Instead of:<br />
<br />
<pre><br />
DELETE FROM tablename;<br />
VACUUM FULL tablename;<br />
</pre><br />
<br />
write:<br />
<br />
<pre><br />
TRUNCATE TABLE tablename;<br />
</pre><br />
<br />
Please make sure to read the caveats in the notes on the <code>[http://www.postgresql.org/docs/current/static/sql-truncate.html TRUNCATE TABLE]</code> documentation. If <code>TRUNCATE TABLE</code> isn't suitable for your needs, you can use <code>DELETE</code> followed by <code>CLUSTER</code> instead.<br />
<br />
If using <code>DELETE</code> on a table that is the target of foreign key references, consider adding an index to the referencing columns. That will allow checks for foreign key enforcement to avoid a sequential scan on the referencing table, making <code>DELETE</code> from the referenced table vastly faster.<br />
<br />
=== <code>[http://www.postgresql.org/docs/current/static/sql-altertable.html ALTER TABLE .. SET DATA TYPE]</code> (relevant for 8.4 and below only)===<br />
<br />
''This section is obsolete for PostgreSQL 9.0 and above. Skip it unless you use a very old version.''<br />
<br />
The problem with <code>CLUSTER</code> is that it reorders the table following an index. If the table is not already approximately in that index' order, this will take a long time because it will have to do a scattered read of the table pages over and over as it looks for each tuple. A faster alternative is to request a full table rewrite without requiring a particular order. PostgreSQL versions prior to 9.0 do not offer any direct way to invoke this operation; however, you can use the following workaround. Choose any table column, and use <code>[http://www.postgresql.org/docs/current/static/sql-altertable.html ALTER TABLE]</code> to change its type <i>to the same type</i>. This is obviously going to cause no logical change to the table, but the server will have to rewrite the table, getting rid of dead tuples while at it.<br />
<br />
For example, assuming the <code>an_integer_column</code> is of type <code>INTEGER</code>:<br />
<br />
<pre><br />
ALTER TABLE your_table ALTER an_integer_column SET DATA TYPE integer;<br />
</pre><br />
<br />
This trick will not work in PostgreSQL versions 9.1 or later, as it detects that the change in data type is degenerate and so no rewrite is necessary.<br />
<br />
=== <code>[http://www.postgresql.org/docs/current/static/sql-selectinto.html SELECT ... INTO]</code> (relevant for 8.4 and below only) ===<br />
<br />
''This section is obsolete for PostgreSQL 9.0 and above. Skip it unless you use a very old version.''<br />
<br />
Sometimes it can be faster to use a <code>[http://www.postgresql.org/docs/current/static/sql-selectinto.html SELECT ... INTO]</code> command to copy data from a bloated table into a new table, then re-create the indexes and finally rename the tables to replace the old one with the new one. It's rarely worth doing this instead of using <code>CLUSTER</code>, though, as <code>CLUSTER</code> does almost the same thing automatically and can rebuild indexes in parallel. The main reason you may want to use <code>SELECT ... INTO</code> instead of <code>CLUSTER</code> is if you don't want to sort the table.<br />
<br />
<br />
== Recovering from index bloat caused by <code>VACUUM FULL</code> (relevant for 8.4 and below only) ==<br />
<br />
''This section is obsolete for PostgreSQL 9.0 and above. Skip it unless you use a very old version.''<br />
<br />
If you have indexes badly bloated by regular use of <code>VACUUM FULL</code>, your best bet is usually going to be to use <code>CLUSTER</code> to rewrite the table and rebuild the indexes.<br />
<br />
If you can't afford to have the table locked for that long, you can rebuild each index individually while queries continue to run on the table. PostgreSQL unfortunately doesn't have a <code>REINDEX CONCURRENTLY</code> command, but it can be simulated with appropriate use of [http://www.postgresql.org/docs/current/static/sql-createindex.html CREATE INDEX ... CONCURRENTLY], [http://www.postgresql.org/docs/current/static/sql-alterindex.html ALTER INDEX ... RENAME] and [http://www.postgresql.org/docs/current/static/sql-dropindex.html DROP INDEX] to create new indexes, swap the old and new ones by renaming, and drop the old indexes. Note that since you can't drop some indexes, such as primary keys, this may not be a possible cleanup technique for all of them.<br />
<br />
Originally by --[[User:Ringerc|Ringerc]] 03:48, 26 November 2009 (UTC)<br />
<br />
[[Category:FAQ]][[Category:Vacuuming]][[Category:Administration]][[Category:Performance]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Streaming_Replication&diff=18542Streaming Replication2012-11-09T21:06:16Z<p>Schmiddy: Make clear that synchronous replication is supported now</p>
<hr />
<div>'''Streaming Replication''' (SR) provides the capability to continuously ship and<br />
apply the [http://www.postgresql.org/docs/current/static/wal.html WAL XLOG] records to some number of standby servers in order to keep them current.<br />
<br />
This feature was added to PostgreSQL 9.0. The discussion below is a developer oriented one that contains some out of data information. Users of this feature should use the documentation for the feature or a setup tutorial instead:<br />
<br />
* [[Binary Replication Tutorial]] provides an introduction to using this replication feature.<br />
* [http://www.postgresql.org/docs/9.1/static/warm-standby.html 9.1 Replication Documentation]<br />
* [http://www.postgresql.org/docs/9.0/static/warm-standby.html 9.0 Replication Documentation]<br />
<br />
= Project =<br />
SR was developed for inclusion in PostgreSQL 9.0 by NTT OSS Center. The lead developer is [mailto:masao.fujii@gmail.com Masao Fujii]. [http://www.pgcon.org/2008/schedule/events/76.en.html Synchronous Log Shipping Replication Presentation] introduces the early design of the feature.<br />
<br />
= Usage =<br />
== Users Overview ==<br />
* '''Log-shipping'''<br />
** XLOG records generated in the primary are periodically shipped to the standby via the network.<br />
** In the existing warm standby, only records in a filled file are shipped, what's referred to as file-based log-shipping. In SR, XLOG records in partially-filled XLOG file are shipped too, implementing record-based log-shipping. This means the window for data loss in SR is usually smaller than in warm standby, unless the warm standby was also configured for record-based shipping (which is complicated to setup).<br />
** The content of XLOG files written to the standby are exactly the same as those on the primary. XLOG files shipped can be used for a normal recovery and PITR.<br />
* '''Multiple standbys'''<br />
** More than one standby can establish a connection to the primary for SR. XLOG records are concurrently shipped to all these standbys. The delay/death of a standby does not harm log-shipping to other standbys.<br />
** The maximum number of standbys can be specified as a GUC variable.<br />
* '''Continuous recovery'''<br />
** The standby continuously replays XLOG records shipped without using pg_standby.<br />
** XLOG records shipped are replayed as soon as possible without waiting until XLOG file has been filled. The combination of [[Hot Standby]] and SR would make the latest data inserted into the primary visible in the standby almost immediately.<br />
** The standby periodically removes old XLOG files which are no longer needed for recovery, to prevent excessive disk usage.<br />
* '''Setup'''<br />
** The start of log-shipping does not interfere with any query processing on the primary.<br />
** The standby can be started in various conditions.<br />
*** If there are XLOG files in archive directory and restore_command is supplied, at first those files are replayed. Then the standby requests XLOG records following the last applied one to the primary. This prevents XLOG files already present in the standby from being shipped again. Similarly, XLOG files in pg_xlog are also replayed before starting log-shipping.<br />
*** If there is no XLOG files on the standby, the standby requests XLOG records following the starting XLOG location of recovery (the redo starting location).<br />
* '''Connection settings and authentication'''<br />
** A user can configure the same settings as a normal connection to a connection for SR (e.g., keepalive, pg_hba.conf).<br />
* '''Activation'''<br />
** The standby can keep waiting for activation as long as a user likes. This prevents the standby from being automatically brought up by failure of recovery or network outage.<br />
* '''Progress report'''<br />
** The primary and standby report the progress of log-shipping in PS display.<br />
* '''Graceful shutdown'''<br />
** When smart/fast shutdown is requested, the primary waits to exit until XLOG records have been sent to the standby, up to the shutdown checkpoint record.<br />
<br />
== Restrictions ==<br />
* '''Synchronous log-shipping'''<br />
** By default, SR supports operates in asynchronous manner, so the commit command might return a "success" to a client before the corresponding XLOG records are shipped to the standby. To enable synchronous replication, see [http://www.postgresql.org/docs/current/static/warm-standby.html#SYNCHRONOUS-REPLICATION Synchronous Replication]<br />
* '''Replication beyond timeline'''<br />
** A user has to get a fresh backup whenever making the old standby catch up.<br />
* '''Clustering'''<br />
** Postgres doesn't provide any clustering feature.<br />
<br />
== How to Use ==<br />
* '''1.''' Install postgres in the primary and standby server as usual. This requires only ''configure'', ''make'' and ''make install''.<br />
* '''2.''' Create the initial database cluster in the primary server as usual, using ''initdb''.<br />
* '''3.''' Set up connections and authentication so that the standby server can successfully connect to the ''replication'' pseudo-database on the primary.<br />
$ $EDITOR postgresql.conf<br />
<br />
listen_addresses = '192.168.0.10'<br />
<br />
$ $EDITOR pg_hba.conf<br />
<br />
# The standby server must have superuser access privileges.<br />
host replication postgres 192.168.0.20/22 trust<br />
* '''4.''' Set up the streaming replication related parameters on the primary server.<br />
$ $EDITOR postgresql.conf<br />
<br />
# To enable read-only queries on a standby server, wal_level must be set to<br />
# "hot_standby". But you can choose "archive" if you never connect to the<br />
# server in standby mode.<br />
wal_level = hot_standby<br />
<br />
# Set the maximum number of concurrent connections from the standby servers.<br />
max_wal_senders = 5<br />
<br />
# To prevent the primary server from removing the WAL segments required for<br />
# the standby server before shipping them, set the minimum number of segments<br />
# retained in the pg_xlog directory. At least wal_keep_segments should be<br />
# larger than the number of segments generated between the beginning of<br />
# online-backup and the startup of streaming replication. If you enable WAL<br />
# archiving to an archive directory accessible from the standby, this may<br />
# not be necessary.<br />
wal_keep_segments = 32<br />
<br />
# Enable WAL archiving on the primary to an archive directory accessible from<br />
# the standby. If wal_keep_segments is a high enough number to retain the WAL<br />
# segments required for the standby server, this is not necessary.<br />
archive_mode = on<br />
archive_command = 'cp %p /path_to/archive/%f'<br />
* '''5.''' Start postgres on the primary server.<br />
* '''6.''' Make a base backup by copying the primary server's data directory to the standby server.<br />
$ psql -c "SELECT pg_start_backup('label', true)"<br />
$ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid<br />
$ psql -c "SELECT pg_stop_backup()"<br />
* '''7.''' Set up replication-related parameters, connections and authentication in the standby server like the primary, so that the standby might work as a primary after failover.<br />
* '''8.''' Enable read-only queries on the standby server. But if wal_level is ''archive'' on the primary, leave hot_standby unchanged (i.e., off).<br />
$ $EDITOR postgresql.conf<br />
<br />
hot_standby = on<br />
* '''9.''' Create a recovery command file in the standby server; the following parameters are required for streaming replication.<br />
$ $EDITOR recovery.conf<br />
# Note that recovery.conf must be in $PGDATA directory.<br />
<br />
# Specifies whether to start the server as a standby. In streaming replication,<br />
# this parameter must to be set to on.<br />
standby_mode = 'on'<br />
<br />
# Specifies a connection string which is used for the standby server to connect<br />
# with the primary.<br />
primary_conninfo = 'host=192.168.0.10 port=5432 user=postgres'<br />
<br />
# Specifies a trigger file whose presence should cause streaming replication to<br />
# end (i.e., failover).<br />
trigger_file = '/path_to/trigger'<br />
<br />
# Specifies a command to load archive segments from the WAL archive. If<br />
# wal_keep_segments is a high enough number to retain the WAL segments<br />
# required for the standby server, this may not be necessary. But<br />
# a large workload can cause segments to be recycled before the standby<br />
# is fully synchronized, requiring you to start again from a new base backup.<br />
restore_command = 'cp /path_to/archive/%f "%p"'<br />
* '''10.''' Start postgres in the standby server. It will start streaming replication.<br />
* '''11.''' You can calculate the replication lag by comparing the current WAL write location on the primary with the last WAL location received/replayed by the standby. They can be retrieved using ''pg_current_xlog_location'' on the primary and the ''pg_last_xlog_receive_location''/''pg_last_xlog_replay_location'' on the standby, respectively.<br />
$ psql -c "SELECT pg_current_xlog_location()" -h192.168.0.10 (primary host)<br />
pg_current_xlog_location <br />
--------------------------<br />
0/2000000<br />
(1 row)<br />
<br />
$ psql -c "select pg_last_xlog_receive_location()" -h192.168.0.20 (standby host)<br />
pg_last_xlog_receive_location <br />
-------------------------------<br />
0/2000000<br />
(1 row)<br />
<br />
$ psql -c "select pg_last_xlog_replay_location()" -h192.168.0.20 (standby host)<br />
pg_last_xlog_replay_location <br />
------------------------------<br />
0/2000000<br />
(1 row)<br />
* '''12.''' You can also check the progress of streaming replication by using ''ps'' command.<br />
# The displayed LSNs indicate the byte position that the standby server has<br />
# written up to in the xlogs.<br />
[primary] $ ps -ef | grep sender<br />
postgres 6879 6831 0 10:31 ? 00:00:00 postgres: wal sender process postgres 127.0.0.1(44663) streaming 0/2000000<br />
<br />
[standby] $ ps -ef | grep receiver<br />
postgres 6878 6872 1 10:31 ? 00:00:01 postgres: wal receiver process streaming 0/2000000<br />
* How to do failover<br />
** Create the trigger file in the standby after the primary fails.<br />
* How to stop the primary or the standby server<br />
** Shut down it as usual (''pg_ctl stop'').<br />
* How to restart streaming replication after failover<br />
** Repeat the operations from '''6th'''; making a fresh backup, some configurations and starting the original primary as the standby. The primary server doesn't need to be stopped during these operations.<br />
* How to restart streaming replication after the standby fails<br />
** Restart postgres in the standby server after eliminating the cause of failure.<br />
* How to disconnect the standby from the primary<br />
** Create the trigger file in the standby while the primary is running. Then the standby would be brought up.<br />
* How to re-synchronize the stand-alone standby after isolation<br />
** Shut down the standby as usual. And repeat the operations from '''6th'''.<br />
* If you have more than one slave, promoting one will break the other(s). Update their recovery.conf settings to point to the new master, set recovery_target_timeline to 'latest', scp/rsync the pg_xlog directory, and restart the slave.<br />
<br />
= Todo =<br />
== v9.0 ==<br />
<br />
Moved to [[PostgreSQL_9.0_Open_Items]]<br />
<br />
=== Committed ===<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01455.php Retrying from archive and some refactoring around Read/FetchRecord().] - [http://archives.postgresql.org/pgsql-committers/2010-01/msg00395.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg02601.php SR wrongly treats the WAL-boundary.] - [http://archives.postgresql.org/pgsql-committers/2010-01/msg00396.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01715.php Adjust SR for some later changes about wal-skipping.] - [http://archives.postgresql.org/pgsql-committers/2010-01/msg00399.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00024.php VACUUM FULL unexpectedly writes an XLOG UNLOGGED record.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00038.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01754.php Add a message type header.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00037.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01536.php Documentation: Add a new "Replication" chapter.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00115.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00350.php Failed assertion during recovery of partial WAL file.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00124.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00712.php A PANIC error might occur in the standby because of a partially-filled archived WAL file.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00137.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00330.php Improve the standby messages.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00140.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01672.php pq_getbyte_if_available() is not working because the win32 socket emulation layer simply wasn't designed to deal with non-blocking sockets.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00198.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01488.php Walsender might emit unfit messages.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00239.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01236.php Streaming replication on win32, still broken.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00270.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00992.php Create new section for recovery.conf.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00295.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01824.php Assertion failure in walreceiver.] - [http://archives.postgresql.org/pgsql-committers/2010-02/msg00356.php commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01717.php Forbid a startup of walsender during recovery, and emit a suitable message? Or allow walsender to be started also during recovery?] - [http://archives.postgresql.org/message-id/20100316090955.9A5107541D0@cvs.postgresql.org commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01003.php How do we clean down the archive without using pg_standby?] - [http://archives.postgresql.org/message-id/20100318091718.BC14D7541D0@cvs.postgresql.org commit]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01510.php File-based log shipping without pg_standby doesn't replay the WAL files in pg_xlog.] - [http://archives.postgresql.org/pgsql-committers/2010-03/msg00356.php commit]<br />
<br />
== v9.1 ==<br />
=== Synchronization capability ===<br />
* Introduce the replication mode which can control how long transaction commit waits for replication before the commit command returns a "success" to a client. The valid modes are ''async'', ''recv'' and ''fsync''.<br />
** ''async'' doesn't make transaction commit wait for replication, i.e., asynchronous replication.<br />
** ''recv'' or ''fsync'' makes transaction commit wait for XLOG to be received or fsynced by the standby, respectively.<br />
** (''apply'' makes transaction commit wait for XLOG to be replayed by the standby. This mode will be supported in v9.2 or later)<br />
** The replication mode is specified in recovery.conf of the standby as well as other parameters for replication.<br />
*** The startup process reads the replication mode from recovery.conf and shares it to walreceiver via new shared-memory variable.<br />
*** Walreceiver also shares it to walsender by using the replication handshake message (existing protocol needs to be extended).<br />
** Based on the replication mode, walreceiver sends the reply meaning that replication is done up to the specified location to the primary.<br />
*** In async, walreceiver doesn't need to send any reply other than end-of-replication message.<br />
*** In recv or fsync, walreceiver sends the reply just after receiving or flushing XLOG, respectively.<br />
*** New message type for the reply needs to be defined. The reply is sent as CopyData message.<br />
** Walreceiver writes all the outstanding XLOG to disk before shutting down.<br />
** Walsender receives the reply from the standby, updates the location of the last record replicated, and announces completion of replication.<br />
*** New shared-memory variable to keep that location is required.<br />
** When processing the commit command, backend waits for XLOG to be replicated to only the standbys which are in the recv or fsync replication mode.<br />
*** Also smart shutdown waits for XLOG of shutdown checkpoint to be replicated.<br />
* Required optimization<br />
** Walsender should send outstanding XLOG without waiting wal_sender_delay.<br />
*** When processing the commit command, backend signals walsender to send outstanding XLOG immediately.<br />
** Backend should exit the wait loop as soon as the reply arrives at the primary.<br />
*** When receiving the reply, walsender signals backends to get up from the sleep and determine whether to exit the wait loop by checking the location of the last XLOG replicated.<br />
*** Only backends waiting for XLOG to be replicated up to the location contained in the reply are sent the signal.<br />
** Walsender waits for the signal from backends and the reply from the standby at the same time, by using select/poll.<br />
** Walsender reads XLOG from not only disk but also shared memory (wal buffers).<br />
** Walreceiver should flush XLOG file only when XLOG file is switched or the related page is flushed.<br />
*** When startup process or bgwriter flushes the buffer page, it checks whether the related XLOG has already been flushed via shared memory (location of the last XLOG flushed).<br />
*** It flushes the buffer page, if XLOG file has already been flushed.<br />
*** It signals walreceiver to flush XLOG file immediately and waits for the flush to complete, if XLOG file has not been flushed yet.<br />
** While the standby is catching up with the primary, those servers should ignore the replication mode and perform asynchronous replication.<br />
*** After those servers have almost gotten into synchronization, they perform replication based on the specified replication mode.<br />
*** New replication states like 'catching-up', 'sync', etc need to be defined, and the state machine for them is required on both servers.<br />
*** Current replication state can be monitored on both servers via SQL.<br />
* Required timeout<br />
** Add new parameter replication_timeout which is the maximum time to wait until XLOG is replicated to the standby.<br />
** Add new parameter (replication_timeout_action) to specify the reaction to replication_timeout.<br />
<br />
== Future release ==<br />
* '''Synchronization capability'''<br />
** Introduce the synchronization mode which can control how long transaction commit waits for replication before the commit command returns a "success" to a client. The valid modes are ''async'', ''recv'', ''fsync'' and ''apply''.<br />
*** ''async'' doesn't make transaction commit wait for replication, i.e., asynchronous replication.<br />
*** ''recv'', ''fsync'' and ''apply'' makes transaction commit wait for XLOG records to be received, fsynced and applied on the standby, respectively.<br />
** Change walsender to be able to read XLOG from not only the disk but also shared memory.<br />
** Add new parameter replication_timeout which is the maximum time to wait until XLOG records are replicated to the standby.<br />
** Add new parameter (replication_timeout_action) to specify the reaction to replication_timeout.<br />
* '''Monitoring'''<br />
** Provide the capability to check the progress and gap of streaming replication via one query. A collaboration of HS and SR is necessary to provide that capability on the standby side.<br />
** Provide the capability to check if the specified repliation is in progress via a query. Also more detailed status information might be necessary, e.g, the standby is catching up now, has already gotten into sync, and so on.<br />
** Change the stats collector to collect the statistics information about replication, e.g., average delay of replication time.<br />
** Develop the tool to calculate the latest XLOG position from XLOG files. This is necessary to check the gap of replication after the server fails.<br />
** Also develop the tool to extract the user-readable contents from XLOG files. This is necessary to see the contents of the gap, and manually restore them.<br />
* '''Easy to Use'''<br />
** Introduce the parameters like:<br />
*** replication_halt_timeout - replication will halt if no data has been sent for this much time.<br />
*** replication_halt_segments - replication will halt if number of WAL files in pg_xlog exceeds this threshold.<br />
*** These parameters allow us to avoid disk overflow.<br />
** Add new feature which transfers also base backup via the direct connection between the primary and the standby.<br />
** Add new hooks like walsender_hook and walreceiver_hook to cooperate with the add-on program for compression like pglesslog.<br />
** Provide a graceful termination of replication via a query on the primary. On the standby, a trigger file mechanism already provides that capability.<br />
** Support replication beyond timeline. The timeline history files need to be shipped from the primary to the standby.<br />
* '''Robustness'''<br />
** Support keepalive in libpq. This is useful for a client and the standby to detect a failure of the primary immediately.<br />
** [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01536.php New privilege for replication.]<br />
*** Currently superuser privilege is required when the standby connects to the primary. But there is the complaint that we should add new privilege for replication and use it instead of superuser because current approach is not good for security.<br />
* '''Miscellaneous'''<br />
** Standalone walreceiver tool, which connects to the primary, continuously receives and writes XLOG records, independently from postgres server.<br />
** Cascade streaming replication. Allow walsender to send XLOG to another standby during recovery.<br />
** WAL archiving during recovery.<br />
<br />
[[Category:Replication]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=18541User:Schmiddy2012-11-08T22:50:19Z<p>Schmiddy: /* TODO Items / Things to Investigate */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email]<br />
: [http://kupershmidt.org web]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=IRC2RWNames&diff=18540IRC2RWNames2012-11-08T22:46:10Z<p>Schmiddy: /* List of IRC nicks with their respective real world names */</p>
<hr />
<div>=== List of IRC nicks with their respective real world names ===<br />
<br />
You can find many PostgreSQL users and developers chatting in [irc://irc.freenode.net/postgresql #postgresql on freenode]. Here's more information about some of the regulars there:<br />
<br />
{| border="1"<br />
|-<br />
!Nickname || Real Name<br />
|-<br />
|ads || Andreas Scherbaum<br />
|-<br />
|agliodbs, aglio2 (freenode), jberkus (oftc) || Josh Berkus<br />
|-<br />
|ahammond || Andrew Hammond<br />
|-<br />
|alvherre || Alvaro Herrera<br />
|-<br />
|andres || Andres Freund<br />
|-<br />
|Assid || Satish Alwani<br />
|-<br />
|aurynn || Aurynn Shaw<br />
|-<br />
|BlueAidan/BlueAidan_work || [[user:davidblewett | David Blewett]]<br />
|-<br />
|bmomjian || Bruce Momjian<br />
|-<br />
|cbbrowne || Christopher Browne<br />
|-<br />
|cce || Clark C. Evans<br />
|-<br />
|chicagoben || Benjamin Johnson<br />
|-<br />
|crab || Abhijit Menon-Sen<br />
|-<br />
|Crad || Gavin M. Roy<br />
|- <br />
|daamien || Damien Clochard<br />
|-<br />
|DarcyB || Darcy Buskermolen<br />
|-<br />
|darkixion || Thom Brown<br />
|-<br />
|davidfetter || David Fetter<br />
|-<br />
|dbb || Brian Hamlin / darkblue_b<br />
|-<br />
|dcolish || [http://www.unencrypted.org Dan Colish]<br />
|-<br />
|dcramer || Dave Cramer<br />
|-<br />
|DeciBull, TheCougar || Jim C. Nasby<br />
|-<br />
|dennisb || Dennis Bj&ouml;rklund<br />
|-<br />
|depesz || Hubert Lubaczewski<br />
|-<br />
|devrimgunduz || Devrim G&uuml;nd&uuml;z<br />
|-<br />
|digicon || [http://digicondev.blogspot.com Zach Conrad]<br />
|-<br />
|dim || Dimitri Fontaine<br />
|-<br />
|direvus || Brendan Jurd<br />
|-<br />
|drbair || Ryan Bair<br />
|-<br />
|DrLou || Lou Picciano<br />
|-<br />
|duck_tape || Adi Alurkar<br />
|-<br />
|dvl || [http://langille.org/ Dan Langille]<br />
|-<br />
|eggyknap || Joshua Tolley<br />
|-<br />
|endpoint_david || David Christensen<br />
|-<br />
|eulerto || Euler Taveira<br />
|-<br />
|f3ew/devdas || Devdas Vasu Bhagat<br />
|-<br />
|feivel || Michael Meskes<br />
|-<br />
|elein || Elein Mustain<br />
|-<br />
|gleu || Guillaume Lelarge<br />
|-<br />
|gorthx || [[User:Gabrielle|Gabrielle Roth]]<br />
|-<br />
|grzm || Michael Glaesemann<br />
|-<br />
|gsmet || Guillaume Smet<br />
|-<br />
|gregs1104 || Greg Smith<br />
|-<br />
|G_SabinoMullane || Greg Sabino Mullane<br />
|-<br />
|HarrisonF || Harrison Fisk<br />
|-<br />
|ioguix || Jehan-Guillaume de Rorthais<br />
|-<br />
|indigo || Phil Frost<br />
|-<br />
|intgr || Marti Raudsepp<br />
|-<br />
|JanniCash || Jan Wieck<br />
|-<br />
|jconway || Joe Conway<br />
|-<br />
|jdavis, jdavis_ || Jeff Davis<br />
|-<br />
|jkatz05 || Jonathan S. Katz<br />
|-<br />
|johto || Marko Tiikkaja<br />
|-<br />
|jurka || Kris Jurka<br />
|-<br />
|justatheory || David Wheeler<br />
|-<br />
|jpa || Jean-Paul Argudo<br />
|-<br />
|jwp || James Pye<br />
|-<br />
|j_williams || Josh Williams<br />
|-<br />
|kgrittn || Kevin Grittner<br />
|-<br />
|klando || Cédric Villemain<br />
|-<br />
|larryrtx || Larry Rosenman<br />
|-<br />
|linuxpoet, postgresman || Joshua D. Drake<br />
|-<br />
|lluad || Steve Atkins<br />
|-<br />
|lsmith || Lukas Smith<br />
|-<br />
|magnush || Magnus Hagander<br />
|-<br />
|marco44 || Marc Cousin<br />
|-<br />
|markwkm || Mark Wong<br />
|-<br />
|mastermind || [[user:mastermind | Stefan Kaltenbrunner]]<br />
|-<br />
|mbalmer || [[user:mbalmer | Marc Balmer]]<br />
|-<br />
|merlin83 || Chua Khee Chin<br />
|-<br />
|merlinm || Merlin Moncure<br />
|-<br />
|metatrontech || Chris Travers<br />
|-<br />
|miracee || Susanne Ebrecht<br />
|-<br />
|Moosbert || Peter Eisentraut<br />
|-<br />
|neilc || Neil Conway<br />
|-<br />
|oicu || Andrew Dunstan<br />
|-<br />
|okbobcz || Pavel Stehule<br />
|-<br />
|pg_docbot || [[IRCBotSyntax]]<br />
|-<br />
|pgSnake || Dave Page<br />
|-<br />
|PJMODOS || Petr Jel&iacute;nek<br />
|-<br />
|Possible || Robert Ivens<br />
|-<br />
|postwait || Theo Schlossnagle<br />
|-<br />
|prothid || R Brenton Strickler<br />
|-<br />
|psoo || Bernd Helmle<br />
|-<br />
|pyarra || Philip Yarra<br />
|-<br />
|raptelan || [[user:Cshobe|Casey Allen Shobe]]<br />
|-<br />
|rhaas || Robert Haas<br />
|-<br />
|RhodiumToad (formerly AndrewSN) || Andrew Gierth<br />
|-<br />
|Robe || [[user:Robe | Michael Renner]]<br />
|-<br />
|rotellaro || Federico Campoli<br />
|-<br />
|rz || Kirill Simonov<br />
|-<br />
|SAS || Stéphane Schildknecht<br />
|-<br />
|schmiddy || Josh Kupershmidt<br />
|-<br />
|scrappy || Marc G. Fournier<br />
|-<br />
|selenamarie || Selena Deckelmann<br />
|-<br />
|SkippyDigits || Sherri Kalm<br />
|-<br />
|Snow-Man || Stephen Frost<br />
|-<br />
|Spritz || Matteo Beccati<br />
|-<br />
|sternocera || Peter Geoghegan<br />
|-<br />
|StuckMojo, MojoWork || Jon Erdman<br />
|-<br />
|swm || Gavin Sherry<br />
|-<br />
|vy || Volkan YAZICI<br />
|-<br />
|wulczer || Jan Urbański<br />
|-<br />
|xaprb || Baron Schwartz<br />
|-<br />
|xzilla, xzi11a || [[User:Xzilla|Robert Treat]]<br />
|}<br />
<br />
[[Category:Community]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=FAQ&diff=18423FAQ2012-10-23T05:34:09Z<p>Schmiddy: fix a few doc links to point to "current" instead of 8.x versions</p>
<hr />
<div>{{Languages}}<br />
[[:Category:FAQ|Additional FAQ Entries on this Wiki]]<br />
<br />
== Translations of this Document ==<br />
<br />
* [[Häufig gestellte Fragen|German]]<br />
* [[Perguntas Frequentes|Portuguese]]<br />
* [[Preguntas Frecuentes|Spanish]]<br />
* [[Часто Задаваемые Вопросы|Русский]]<br />
<br />
== Platform-specific questions ==<br />
<br />
Windows users should also read the [[Running & Installing PostgreSQL On Native Windows|platform FAQ for Windows]]. There are [[Frequently Asked Questions#Platform FAQs|FAQs for other platforms]] too.<br />
<br />
== General Questions ==<br />
<br />
=== What is PostgreSQL? How is it pronounced? What is Postgres? ===<br />
<br />
PostgreSQL is pronounced Post-Gres-Q-L. (For those curious about how<br />
to say "PostgreSQL", an [http://www.postgresql.org/files/postgresql.mp3 audio file] is available.)<br />
<br />
PostgreSQL is an object-relational database system that has the<br />
features of traditional proprietary database systems with enhancements<br />
to be found in next-generation DBMS systems. PostgreSQL is free and<br />
the complete source code is available.<br />
<br />
PostgreSQL development is performed by a team of mostly volunteer<br />
developers spread throughout the world and communicating via the<br />
Internet. It is a community project and is not controlled by any<br />
company. To get involved, see the [[Developer_FAQ | Developer FAQ]].<br />
<br />
Postgres is a widely-used nickname for PostgreSQL. It was the original<br />
name of the project at Berkeley and is strongly preferred over other<br />
nicknames. If you find 'PostgreSQL' hard to pronounce, call it<br />
'Postgres' instead.<br />
<br />
=== Who controls PostgreSQL? ===<br />
<br />
If you are looking for a PostgreSQL gatekeeper, central committee, or<br />
controlling company, give up --- there isn't one. We do have a core<br />
committee and CVS committers, but these groups are more for<br />
administrative purposes than control. The project is directed by the<br />
community of developers and users, which anyone can join. All you need<br />
to do is subscribe to the mailing lists and participate in the<br />
discussions. (See the [[Developer FAQ|Developer's FAQ]] for information on how to get<br />
involved in PostgreSQL development.)<br />
<br />
=== Who is the PostgreSQL Global Development Group? ===<br />
<br />
The "PGDG" is an international, unincorporated association of<br />
individuals and companies who have contributed to the PostgreSQL<br />
project. The PostgreSQL Core Team generally act as spokespeople<br />
for the PGDG.<br />
<br />
=== Who is the PostgreSQL Core Team? ===<br />
<br />
A committee of five to seven (currently six) senior contributors to<br />
PostgreSQL who do the following for the project: (a) set release dates,<br />
(b) handle confidential matters for the project, (c) act as spokespeople<br />
for the PGDG when required, and (d) arbitrate community decisions which<br />
are not settled by consensus. The current Core Team is listed on top of<br />
[http://www.postgresql.org/community/contributors/ the contributors page]<br />
<br />
=== What about the various PostgreSQL foundations? ===<br />
<br />
While the PostgreSQL project utilizes non-profit corporations in the<br />
USA, Europe, Brazil and Japan for fundraising and project coordination,<br />
these entities do not own the PostgreSQL code.<br />
<br />
=== What is the license of PostgreSQL? ===<br />
<br />
PostgreSQL is distributed under a license similar to BSD and MIT. Basically, it<br />
allows users to do anything they want with the code, including<br />
reselling binaries without the source code. The only restriction is<br />
that you not hold us legally liable for problems with the software.<br />
There is also the requirement that this copyright appear in all copies<br />
of the software. Here is the license we use:<br />
<br />
PostgreSQL Database Management System<br />
(formerly known as Postgres, then as Postgres95)<br />
<br />
Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group<br />
<br />
Portions Copyright (c) 1994, The Regents of the University of California<br />
<br />
Permission to use, copy, modify, and distribute this software and its<br />
documentation for any purpose, without fee, and without a written agreement<br />
is hereby granted, provided that the above copyright notice and this<br />
paragraph and the following two paragraphs appear in all copies.<br />
<br />
IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR<br />
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING<br />
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS<br />
DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE<br />
POSSIBILITY OF SUCH DAMAGE.<br />
<br />
THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,<br />
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY<br />
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS<br />
ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO<br />
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.<br />
<br />
=== What platforms does PostgreSQL support? ===<br />
<br />
In general, any modern Unix-compatible platform should be able to run<br />
PostgreSQL. The platforms that have received recent explicit testing<br />
can be seen in the [http://buildfarm.postgresql.org/ Build farm].<br />
The documentation contains more details about supported platforms at http://www.postgresql.org/docs/current/static/supported-platforms.html.<br />
<br />
PostgreSQL also runs natively on Microsoft Windows NT-based operating<br />
systems like Win2000 SP4, WinXP, and Win2003. A prepackaged installer<br />
is available at http://www.postgresql.org/download/windows.<br />
MSDOS-based versions of Windows (Win95, Win98, WinMe) can run<br />
PostgreSQL using Cygwin.<br />
<br />
=== Where can I get PostgreSQL? ===<br />
<br />
There are binary distributions for various operating systems and platforms; see [http://www.postgresql.org/download our download area].<br />
<br />
The source code can be obtained [http://www.postgresql.org/ftp/ via web browser] or [ftp://ftp.postgresql.org/pub/ through ftp].<br />
<br />
=== What is the most recent release? ===<br />
<br />
The latest release of PostgreSQL is shown on the front page of [http://www.postgresql.org/ our website].<br />
<br />
We typically have a major release every year, with minor releases every few months.<br />
Minor releases are usually made at the same time for all supported major-release branches.<br />
For more about major versus minor releases, see<br />
http://www.postgresql.org/support/versioning.<br />
<br />
=== Where can I get support? ===<br />
<br />
The PostgreSQL community provides assistance to many of its users via<br />
email. The main web site to subscribe to the email lists is<br />
http://www.postgresql.org/community/lists/. The general or bugs lists<br />
are a good place to start. For best results, consider reading the <br />
[[guide to reporting problems]] before you post to make sure you<br />
include enough information for people to help you.<br />
<br />
The major IRC channel is #postgresql on Freenode (irc.freenode.net).<br />
A Spanish one also<br />
exists on the same network, (#postgresql-es), a French one,<br />
(#postgresqlfr), and a Brazilian one, (#postgresql-br). There is also<br />
a PostgreSQL channel on EFNet.<br />
<br />
A list of support companies is available at<br />
http://www.postgresql.org/support/professional_support.<br />
<br />
=== How do I submit a bug report? ===<br />
<br />
Visit the PostgreSQL bug form at<br />
http://www.postgresql.org/support/submitbug to submit your bug<br />
report to the pgsql-bugs mailing list. Also check out our ftp<br />
site ftp://ftp.postgresql.org/pub/ to see if there is a more recent<br />
PostgreSQL version.<br />
<br />
For a prompt and helpful response, it is important for you to read the <br />
[[guide to reporting problems]] to make sure that you include the <br />
information required to fully understand and act on your report.<br />
<br />
Bugs submitted using the bug form or posted to any PostgreSQL mailing<br />
list typically generates one of the following replies:<br />
* It is not a bug, and why<br />
* It is a known bug and is already on the TODO list<br />
* The bug has been fixed in the current release<br />
* The bug has been fixed but is not packaged yet in an official release<br />
* A request is made for more detailed information:<br />
** Operating system<br />
** PostgreSQL version<br />
** Reproducible test case<br />
** Debugging information<br />
** [[Generating_a_stack_trace_of_a_PostgreSQL_backend|Debugger backtrace output]]<br />
* The bug is new. The following might happen:<br />
** A patch is created and will be included in the next major or minor release<br />
** The bug cannot be fixed immediately and is added to the TODO list<br />
<br />
=== How do I find out about known bugs or missing features? ===<br />
<br />
PostgreSQL supports an extended subset of SQL:2008. See our [[Todo|TODO list]]<br />
for known bugs, missing features, and future plans.<br />
<br />
A feature request usually results in one of the following replies:<br />
* The feature is already on the TODO list<br />
* The feature is not desired because:<br />
** It duplicates existing functionality that already follows the SQL standard<br />
** The feature would increase code complexity but add little benefit<br />
** The feature would be insecure or unreliable<br />
* The new feature is added to the TODO list<br />
<br />
PostgreSQL does not use a bug tracking system because we find it more<br />
efficient to respond directly to email and keep the TODO list<br />
up-to-date. In practice, bugs don't last very long in the software,<br />
and bugs that affect a large number of users are fixed rapidly. The<br />
only place to find all changes, improvements, and fixes in a<br />
PostgreSQL release is to read the CVS log messages. Even the release<br />
notes do not list every change made to the software.<br />
<br />
=== A bug I'm encountering is fixed in a newer minor release of PostgreSQL, but I don't want to upgrade. Can I get a patch for just this issue? ===<br />
<br />
No. Nobody will make a custom patch for you so you can (say) extract a fix from 8.4.3 and apply it to 8.4.1 . <br />
That's because there should never be any need to do that.<br />
<br />
PostgreSQL has a strict policy that only bug fixes are back-patched into point releases, as per the [http://www.postgresql.org/support/versioning version policy]. It is safe to upgrade from 8.4.1 to 8.4.3,<br />
for example. Binary compatibility will be maintained, no dump and reload is required, nothing will break, but bugs that might <br />
cause problems have been fixed. Even if you are not yet encountering a particular bug, you might later, and it is wise to upgrade promptly.<br />
You just have to install the update and re-start the database server, nothing more.<br />
<br />
Upgrading from 8.3 to 8.4, or 8.4 to 9.0, is a major upgrade that does not come with the same guarantees. However, if a bug<br />
is discovered in 9.0 then it will generally be fixed in all maintained older versions like 8.4 and 8.3 if it is safe and<br />
practical to do so.<br />
<br />
This means that if you're running 8.1.0, upgrading to 8.1.21 is <b>strongly</b> recommended and very safe. On the other hand,<br />
upgrading to the next major release, 8.2.x, may require changes to your app, and will certainly require a dump and reload.<br />
<br />
If you want to be careful about all upgrades, you should read the [http://www.postgresql.org/docs/current/static/release.html release notes] <br />
for each point release between your current one and the latest minor version of the same major release carefully. If you're<br />
exceptionally paranoid about upgrades, you can fetch the source code to each set of point release changes from [http://git.postgresql.org/ PostgreSQL's git repository] and examine it.<br />
<br />
It is strongly recommended that you <b>always</b> upgrade to the latest minor release. Avoid trying to extract and apply individual fixes<br />
from point releases; by doing so you're bypassing all the QA done by the PostgreSQL team when they prepare a release, and are creating your<br />
own custom version that <i>nobody else has ever used</i>. It's a lot safer to just update to the latest tested, safe release. <i>Patching your own custom, non-standard build will also take more time/effort, and will require the same amount of downtime as a normal upgrade.</i><br />
<br />
=== I have a program that says it wants PostgreSQL x.y.1. Can I use PostgreSQL x.y.2 instead? ===<br />
<br />
Any program that works with a particular version, like 8.4.1, should work with any other minor version in the same major version. That means that if a program says it wants (eg) 8.4.1, you can and should install the latest in the 8.4 series instead.<br />
<br />
See the previous question for more details.<br />
<br />
=== What documentation is available? ===<br />
<br />
PostgreSQL includes extensive documentation, including a large manual,<br />
manual pages, and some test examples. See the /doc directory. You can<br />
also browse the manuals online at http://www.postgresql.org/docs.<br />
<br />
There are a number of PostgreSQL<br />
books available for purchase; two of them are also available online. A list of books can be found at<br />
http://www.postgresql.org/docs/books/. One of the most popular ones is the one by Korry & Susan<br />
Douglas.<br />
<br />
There is also a collection of<br />
PostgreSQL technical articles on the <br />
[[Community_Generated_Articles%2C_Guides%2C_and_Documentation | wiki]].<br />
<br />
The command line client program psql has some \d commands to show<br />
information about types, operators, functions, aggregates, etc. - use<br />
\? to display the available commands.<br />
<br />
=== How can I learn SQL? ===<br />
<br />
First, consider the PostgreSQL-specific books mentioned above. Many of<br />
our users also like The Practical SQL Handbook, Bowman, Judith S., et<br />
al., Addison-Wesley. Others like The Complete Reference SQL, Groff et<br />
al., McGraw-Hill.<br />
<br />
Many people consider the PostgreSQL documentation to be an excellent guide<br />
for learning SQL its self, as well as for PostgreSQL's implementation of it.<br />
For best results use PostgreSQL alongside another full-featured SQL database as<br />
you learn, so you get used to SQL without becoming reliant on PostgreSQL-specific<br />
features. The PostgreSQL documentation generally mentions when features are PostgreSQL<br />
extensions of the standard.<br />
<br />
There are also many nice tutorials available online:<br />
* http://www.intermedia.net/support/sql/sqltut.shtm<br />
* http://sqlcourse.com<br />
* http://www.w3schools.com/sql/default.asp<br />
* http://mysite.verizon.net/Graeme_Birchall/id1.html<br />
* http://sqlzoo.net<br />
<br />
=== How do I submit a patch or join the development team? ===<br />
<br />
See the [[Developer FAQ|Developer's FAQ]].<br />
<br />
=== How does PostgreSQL compare to other DBMSs? ===<br />
<br />
There are several ways of measuring software: features, performance,<br />
reliability, support, and price.<br />
<br />
==== Features ====<br />
<br />
PostgreSQL has most features present in large proprietary DBMSs,<br />
like transactions, subselects, triggers, views, foreign key<br />
referential integrity, and sophisticated locking. We have some<br />
features they do not have, like user-defined types,<br />
inheritance, rules, and multi-version concurrency control to<br />
reduce lock contention.<br />
<br />
==== Performance ====<br />
<br />
PostgreSQL's performance is comparable to other proprietary and<br />
open source databases. It is faster for some things, slower for<br />
others. Our performance is usually +/-10% compared to other databases.<br />
<br />
==== Reliability ====<br />
<br />
We realize that a DBMS must be reliable, or it is worthless. We<br />
strive to release well-tested, stable code that has a minimum<br />
of bugs. Each release has at least one month of beta testing,<br />
and our release history shows that we can provide stable, solid<br />
releases that are ready for production use. We believe we<br />
compare favorably to other database software in this area.<br />
<br />
==== Support ====<br />
<br />
Our mailing lists provide contact with a large group of<br />
developers and users to help resolve any problems encountered.<br />
While we cannot guarantee a fix, proprietary DBMSs do not always<br />
supply a fix either. Direct access to developers, the user<br />
community, manuals, and the source code often make PostgreSQL<br />
support superior to other DBMSs. There is commercial<br />
per-incident support available for those who need it. (See [[#Where_can_I_get_support.3F | section 1.7]]).<br />
<br />
==== Price ====<br />
<br />
We are free for all use, both proprietary and open source.<br />
You can add our code to your product with no limitations,<br />
except those outlined in our BSD-style license stated above. <br />
<br />
=== Can PostgreSQL be embedded? ===<br />
<br />
PostgreSQL is designed as a client/server architecture, which requires<br />
separate processes for each client and server, and various helper<br />
processes. Many embedded architectures can support such requirements.<br />
However, if your embedded architecture requires the database server to<br />
run inside the application process, you cannot use Postgres and should<br />
select a lighter-weight database solution.<br />
<br />
Popular embeddable options include [http://sqlite.org SQLite] and [http://firebirdsql.org Firebird SQL].<br />
<br />
=== How do I unsubscribe from the PostgreSQL email lists? How do I avoid receiving duplicate emails? ===<br />
<br />
The PostgreSQL Majordomo page allows subscribing or unsubscribing from<br />
any of the PostgreSQL email lists. (You might need to have your<br />
Majordomo password emailed to you to log in.)<br />
<br />
All PostgreSQL email lists are configured so a group reply goes to the<br />
email list and the original email author. This is done so users<br />
receive the quickest possible email replies. If you would prefer not<br />
to receive duplicate email from the list in cases where you already<br />
receive an email directly, check eliminatecc from the Majordomo Change<br />
Settings page. You can also prevent yourself from receiving copies of<br />
emails you post to the lists by unchecking selfcopy.<br />
<br />
== User Client Questions ==<br />
<br />
=== What interfaces are available for PostgreSQL? ===<br />
<br />
The PostgreSQL install includes only the C and embedded C interfaces.<br />
All other interfaces are independent projects that are downloaded<br />
separately; being separate allows them to have their own release<br />
schedule and development teams.<br />
<br />
Some programming languages like PHP include an interface to<br />
PostgreSQL. Interfaces for languages like Perl, TCL, Python, and many<br />
others are available at http://pgfoundry.org.<br />
<br />
=== What tools are available for using PostgreSQL with Web pages? ===<br />
<br />
A nice introduction to Database-backed Web pages can be seen at:<br />
http://www.webreview.com<br />
<br />
For Web integration, PHP (http://www.php.net) is an excellent<br />
interface.<br />
<br />
For complex cases, many use the Perl and DBD::Pg with CGI.pm or<br />
mod_perl.<br />
<br />
=== Does PostgreSQL have a graphical user interface? ===<br />
<br />
There are a large number of GUI Tools that are available for<br />
PostgreSQL from both proprietary and open source developers. A detailed<br />
list can be found in the [[Community Guide to PostgreSQL GUI Tools]].<br />
<br />
== Administrative Questions ==<br />
<br />
=== How do I install PostgreSQL somewhere other than /usr/local/pgsql? ===<br />
<br />
Specify the --prefix option when running configure.<br />
<br />
=== I'm installing PostgreSQL and don't know the password for the postgres user ===<br />
<br />
Dave Page wrote a [http://pgsnake.blogspot.com/2010/07/postgresql-passwords-and-installers.html blog post] explaining what the different passwords are used for, and how to overcome common problems such as resetting them.<br />
<br />
=== How do I control connections from other hosts? ===<br />
<br />
By default, PostgreSQL only allows connections from the local machine<br />
using Unix domain sockets or TCP/IP connections. Other machines will<br />
not be able to connect unless you modify listen_addresses in the<br />
postgresql.conf file, enable host-based authentication by modifying<br />
the $PGDATA/pg_hba.conf file, and restart the database server.<br />
<br />
=== How do I tune the database engine for better performance? ===<br />
<br />
There are three major areas for potential performance improvement:<br />
<br />
==== Query Changes ====<br />
This involves modifying queries to obtain better performance:<br />
<br />
* Creation of indexes, including expression and partial indexes<br />
* Use of COPY instead of multiple INSERTs<br />
* Grouping of multiple statements into a single transaction to reduce commit overhead<br />
* Use of CLUSTER when retrieving many rows from an index<br />
* Use of LIMIT for returning a subset of a query's output<br />
* Use of Prepared queries<br />
* Use of ANALYZE to maintain accurate optimizer statistics<br />
* Regular use of VACUUM or pg_autovacuum<br />
* Dropping of indexes during large data changes<br />
<br />
==== Server Configuration ====<br />
A number of postgresql.conf settings affect performance. For<br />
more details, see Administration Guide/Server Run-time<br />
Environment/Run-time Configuration.<br />
<br />
==== Hardware Selection ====<br />
The effect of hardware on performance is detailed in<br />
http://www.powerpostgresql.com/PerfList/ and<br />
http://momjian.us/main/writings/pgsql/hw_performance/index.html.<br />
<br />
=== What debugging features are available? ===<br />
<br />
There are many log_* server configuration variables at http://www.postgresql.org/docs/current/interactive/runtime-config-logging.html that enable printing of query and process statistics which can be very useful for debugging and performance measurements.<br />
<br />
=== Why do I get "Sorry, too many clients" when trying to connect? ===<br />
<br />
You have reached the default limit of 100 database sessions. You need<br />
to increase the server's limit on how many concurrent backend<br />
processes it can start by changing the max_connections value in<br />
postgresql.conf and restarting the server.<br />
<br />
=== What is the upgrade process for PostgreSQL? ===<br />
<br />
See http://www.postgresql.org/support/versioning for a general<br />
discussion about upgrading, and<br />
http://www.postgresql.org/docs/current/static/install-upgrading.html<br />
for specific instructions.<br />
<br />
=== Will PostgreSQL handle recent daylight saving time changes in various countries? ===<br />
<br />
PostgreSQL releases 8.0 and up depend on the widely-used tzdata database<br />
(also called the zoneinfo database or the [http://www.twinsun.com/tz/tz-link.htm Olson timezone database]) for<br />
daylight savings information. To deal with a DST law change that affects you,<br />
install a new tzdata file set and restart the server.<br />
<br />
All PostgreSQL update releases include the latest available tzdata files,<br />
so keeping up-to-date on minor releases for your major version is usually<br />
sufficient for this.<br />
<br />
On platforms that receive regular software updates including new tzdata files,<br />
it may be more convenient to rely on the system's copy of the tzdata files.<br />
This is possible as a compile-time option. Most Linux distributions choose<br />
this approach for their pre-built versions of PostgreSQL.<br />
<br />
PostgreSQL releases before 8.0 always rely on the operating system's timezone<br />
information.<br />
<br />
=== What computer hardware should I use? ===<br />
<br />
Because PC hardware is mostly compatible, people tend to believe that<br />
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and<br />
quality motherboards are more reliable and have better performance<br />
than less expensive hardware. PostgreSQL will run on almost any<br />
hardware, but if reliability and performance are important it is wise<br />
to research your hardware options thoroughly. <br />
<br />
Database servers, unlike many other applications, are usually I/O and memory constrained, so it is wise to focus on the I/O subsystem first, then memory capacity, and lastly consider CPU issues. For example, a disk controller with a<br />
battery-backed cache is often the cheapest and easiest way to improve database performance. Our email lists can be used to discuss hardware options and tradeoffs.<br />
<br />
=== How does PostgreSQL use CPU resources? ===<br />
<br />
The PostgreSQL server is process-based (not threaded), and uses one operating system process per database session. A single database session (connection) cannot utilize more than one CPU. Of course, multiple sessions are automatically spread across all available CPUs by your operating system. Client applications can easily use threads and create multiple database connections from each thread.<br />
<br />
A single complex and CPU-intensive query is unable to use more than one CPU to do the processing for the query. The OS may still be able to use others for disk I/O etc, but you won't see much benefit from more than one spare core.<br />
<br />
=== Why does PostgreSQL have so many processes, even when idle? ===<br />
<br />
As noted in [[FAQ#How does PostgreSQL use CPU resources?|the answer above]], PostgreSQL is process based, so it starts one <code>postgres</code> (or <code>postgres.exe</code> on Windows) instance per connection. The postmaster (which accepts connections and starts new postgres instances for them) is always running. In addition, PostgreSQL generally has one or more "helper" processes like the stats collector, background writer, autovacuum daemon, walsender, etc, all of which show up as "postgres" instances in most system monitoring tools.<br />
<br />
Despite the number of processes, they actually use very little in the way of real resources. See [[FAQ#Why does PostgreSQL show up as using so much memory in my system monitoring tool?|the next answer]].<br />
<br />
=== Why does PostgreSQL use so much memory? ===<br />
<br />
Despite appearances, this is absolutely normal, and there's actually nowhere near as much memory being used as tools like <code>top</code> or the Windows process monitor say PostgreSQL is using.<br />
<br />
Tools like <code>top</code> and the Windows process monitor may show many <code>postgres</code> instances (see above), each of which appears to use a huge amount of memory. Often, when added up, the amount the postgres instances use is many times the amount of memory actually installed in the computer!<br />
<br />
This is a consequence of how these tools report memory use. They generally don't understand shared memory very well, and show it as if it was memory used individually and exclusively by each postgres instance. PostgreSQL uses a big chunk of shared memory to communicate between its backends and cache data. Because these tools count that shared memory block once per <code>postgres</code> instance instead of counting it once for <i>all</i> <code>postgres</code> instances, they massively over-estimate how much memory PostgreSQL is using.<br />
<br />
Furthermore, many versions of these tools don't report the entire shared memory block as being used by an individual instance immediately when it starts, but rather count the number of shared pages it has touched since starting. Over the lifetime of an instance, it will inevitably touch more and more of the shared memory until it has touched every page, so that its reported usage will gradually rise to include the entire shared memory block. This is frequently misinterpreted to be a memory leak; but it is no such thing, only a reporting artifact.<br />
<br />
== Operational Questions ==<br />
<br />
=== How do I SELECT only the first few rows of a query? A random row? ===<br />
<br />
To retrieve only a few rows, if you know at the number of rows needed<br />
at the time of the SELECT use LIMIT . If an index matches the ORDER BY<br />
it is possible the entire query does not have to be executed. If you<br />
don't know the number of rows at SELECT time, use a cursor and FETCH.<br />
<br />
To SELECT a random row, use:<br />
SELECT col<br />
FROM tab<br />
ORDER BY random()<br />
LIMIT 1;<br />
<br />
See also this [http://blog.rhodiumtoad.org.uk/2009/03/08/selecting-random-rows-from-a-table/ blog entry by Andrew Gierth]<br />
that has more information on this topic.<br />
<br />
=== How do I find out what tables, indexes, databases, and users are defined? How do I see the queries used by psql to display them? ===<br />
<br />
Use the \dt command to see tables in psql. For a complete list of<br />
commands inside psql you can use \?. Alternatively you can read the<br />
source code for psql in file pgsql/src/bin/psql/describe.c, it<br />
contains SQL commands that generate the output for psql's backslash<br />
commands. You can also start psql with the -E option so it will print<br />
out the queries it uses to execute the commands you give. PostgreSQL<br />
also provides an SQL compliant INFORMATION SCHEMA interface you can<br />
query to get information about the database.<br />
<br />
There are also system tables beginning with pg_ that describe these<br />
too.<br />
<br />
Use psql -l will list all databases.<br />
<br />
Also try the file pgsql/src/tutorial/syscat.source. It illustrates<br />
many of the SELECTs needed to get information from the database system<br />
tables.<br />
<br />
=== How do you change a column's data type? ===<br />
<br />
Changing the data type of a column can be done easily in 8.0 and later<br />
with ALTER TABLE ALTER COLUMN TYPE.<br />
<br />
In earlier releases, do this:<br />
BEGIN;<br />
ALTER TABLE tab ADD COLUMN new_col new_data_type;<br />
UPDATE tab SET new_col = CAST(old_col AS new_data_type);<br />
ALTER TABLE tab DROP COLUMN old_col;<br />
COMMIT;<br />
<br />
You might then want to do VACUUM FULL tab to reclaim the disk space<br />
used by the expired rows.<br />
<br />
=== What is the maximum size for a row, a table, and a database? ===<br />
<br />
These are the limits:<br />
<br />
Maximum size for a database? unlimited (32 TB databases exist)<br />
Maximum size for a table? 32 TB<br />
Maximum size for a row? 400 GB<br />
Maximum size for a field? 1 GB<br />
Maximum number of rows in a table? unlimited<br />
Maximum number of columns in a table? 250-1600 depending on column types<br />
Maximum number of indexes on a table? unlimited<br />
<br />
Of course, these are not actually unlimited, but limited to available<br />
disk space and memory/swap space. Performance may suffer when these<br />
values get unusually large.<br />
<br />
The maximum table size of 32 TB does not require large file support<br />
from the operating system. Large tables are stored as multiple 1 GB<br />
files so file system size limits are not important.<br />
<br />
The maximum table size, row size, and maximum number of columns can be<br />
quadrupled by increasing the default block size to 32k. The maximum<br />
table size can also be increased using table partitioning.<br />
<br />
One limitation is that indexes can not be created on columns longer<br />
than about 2,000 characters. Fortunately, such indexes are rarely<br />
needed. Uniqueness is best guaranteed by a function index of an MD5<br />
hash of the long column, and full text indexing allows for searching<br />
of words within the column.<br />
<br />
=== How much database disk space is required to store data from a typical text file? ===<br />
<br />
A PostgreSQL database may require up to five times the disk space to<br />
store data from a text file.<br />
<br />
As an example, consider a file of 100,000 lines with an integer and<br />
text description on each line. Suppose the text string averages<br />
twenty bytes in length. The flat file would be 2.8 MB. The size of the<br />
PostgreSQL database file containing this data can be estimated as 5.2<br />
MB:<br />
24 bytes: each row header (approximate)<br />
24 bytes: one int field and one text field<br />
+ 4 bytes: pointer on page to tuple<br />
----------------------------------------<br />
52 bytes per row<br />
<br />
The data page size in PostgreSQL is 8192 bytes (8 KB), so:<br />
<br />
8192 bytes per page<br />
------------------- = 158 rows per database page (rounded down)<br />
52 bytes per row<br />
<br />
100000 data rows<br />
------------------ = 633 database pages (rounded up)<br />
158 rows per page<br />
<br />
633 database pages * 8192 bytes per page = 5,185,536 bytes (5.2 MB)<br />
<br />
Indexes do not require as much overhead, but do contain the data that<br />
is being indexed, so they can be large also.<br />
<br />
NULLs are stored as bitmaps, so they use very little space.<br />
<br />
Note that long values may be compressed transparently.<br />
<br />
See also this presentation on the topic: [[Image:How_Long_Is_a_String.pdf]].<br />
<br />
=== Why are my queries slow? Why don't they use my indexes? ===<br />
<br />
Indexes are not used by every query. Indexes are used only if the<br />
table is larger than a minimum size, and the query selects only a<br />
small percentage of the rows in the table. This is because the random<br />
disk access caused by an index scan can be slower than a straight read<br />
through the table, or sequential scan.<br />
<br />
To determine if an index should be used, PostgreSQL must have<br />
statistics about the table. These statistics are collected using<br />
VACUUM ANALYZE, or simply ANALYZE. Using statistics, the optimizer<br />
knows how many rows are in the table, and can better determine if<br />
indexes should be used. Statistics are also valuable in determining<br />
optimal join order and join methods. Statistics collection should be<br />
performed periodically as the contents of the table change.<br />
<br />
Indexes are normally not used for ORDER BY or to perform joins. A<br />
sequential scan followed by an explicit sort is usually faster than an<br />
index scan of a large table. However, LIMIT combined with ORDER BY<br />
often will use an index because only a small portion of the table is<br />
returned.<br />
<br />
If you believe the optimizer is incorrect in choosing a sequential<br />
scan, use SET enable_seqscan TO 'off' and run query again to see if an<br />
index scan is indeed faster.<br />
<br />
When using wild-card operators such as LIKE or ~, indexes can only be<br />
used in certain circumstances:<br />
<br />
* The beginning of the search string must be anchored to the start of the string, i.e.<br />
** LIKE patterns must not start with % or _.<br />
** ~ (regular expression) patterns must start with ^.<br />
<br />
* The search string can not start with a character class, e.g. [a-e].<br />
<br />
* Case-insensitive searches such as ILIKE and ~* do not utilize indexes. Instead, use expression indexes, which are described in [[#How_do_I_perform_regular_expression_searches_and_case-insensitive_regular_expression_searches.3F_How_do_I_use_an_index_for_case-insensitive_searches.3F | section 4.8]].<br />
<br />
* C locale must be used during initdb because sorting in a non-C locale often doesn't match the behavior of LIKE. You can create a special text_pattern_ops index that will work in such cases, but note it is only helpful for LIKE indexing.<br />
<br />
It is also possible to use full text indexing for word searches.<br />
<br />
The [[SlowQueryQuestions]] article contains some more tips and guidance.<br />
<br />
=== How do I see how the query optimizer is evaluating my query? ===<br />
<br />
This is done with the EXPLAIN command; see [[Using EXPLAIN]].<br />
<br />
=== How do I change the sort ordering of textual data? ===<br />
<br />
PostgreSQL sorts textual data according to the ordering that is defined by the current locale, which is selected during initdb.<br />
(In 8.4 and up it will be possible to select a different locale when creating a new database.)<br />
If you don't like the ordering then you need to use a different locale.<br />
In particular, most locales other than "C" sort according to dictionary order, which largely ignores punctuation and spacing.<br />
If that's not what you want then you need "C" locale.<br />
<br />
=== How do I perform regular expression searches and case-insensitive regular expression searches? How do I use an index for case-insensitive searches? ===<br />
<br />
The ~ operator does regular expression matching, and ~* does<br />
case-insensitive regular expression matching. The case-insensitive<br />
variant of LIKE is called ILIKE.<br />
<br />
Case-insensitive equality comparisons are normally expressed as:<br />
SELECT *<br />
FROM tab<br />
WHERE lower(col) = 'abc';<br />
<br />
This will not use a standard index on "col". However, if you create an<br />
expression index on "lower(col)", it will be used:<br />
CREATE INDEX tabindex ON tab (lower(col));<br />
<br />
If the above index is created as UNIQUE, then the column can store<br />
upper and lowercase characters, but it cannot contain identical values that<br />
differ only in case. To force a particular case to be stored in the<br />
column, use a CHECK constraint or a trigger.<br />
<br />
In PostgreSQL 8.4 and later, you can also use the contributed [http://www.postgresql.org/docs/current/static/citext.html CITEXT data type], which internally implements the "lower()" calls, so that you can effectively treat it as a fully case-insensitive data type. CITEXT is also [https://svn.kineticode.com/citext/trunk/ available for 8.3], and an earlier version that treats only ASCII characters case-insensitively on 8.2 and earlier is available on [http://pgfoundry.org/projects/citext/ pgFoundry].<br />
<br />
=== In a query, how do I detect if a field is NULL? How do I concatenate possible NULLs? How can I sort on whether a field is NULL or not? ===<br />
<br />
You can test the value with IS NULL or IS NOT NULL, like this:<br />
SELECT *<br />
FROM tab<br />
WHERE col IS NULL;<br />
<br />
Concatenating a NULL with something else produces another NULL.<br />
If that's not what you want, you can replace the NULL(s) using<br />
COALESCE(), like this:<br />
<nowiki>SELECT COALESCE(col1, '') || COALESCE(col2, '')</nowiki><br />
FROM tab;<br />
<br />
To sort by the NULL status, use an IS NULL or IS NOT NULL test<br />
in your ORDER BY clause. Things that are true will sort higher than<br />
things that are false, so the following will put NULL entries at the<br />
front of the output:<br />
SELECT *<br />
FROM tab<br />
ORDER BY (col IS NOT NULL), col;<br />
<br />
In PostgreSQL 8.3 and up, you can also control sort ordering of NULLs<br />
using the recently-standardized NULLS FIRST/NULLS LAST modifiers,<br />
like this:<br />
SELECT *<br />
FROM tab<br />
ORDER BY col NULLS FIRST;<br />
<br />
=== What is the difference between the various character types? ===<br />
<br />
{| border="1"<br />
!Type<br />
!Internal Name<br />
!Notes<br />
|-<br />
|VARCHAR(n) <br />
|varchar<br />
|size specifies maximum length, no padding<br />
|- <br />
|CHAR(n)<br />
|bpchar<br />
|blank-padded to the specified fixed length<br />
|-<br />
|TEXT<br />
|text<br />
|no specific upper limit on length<br />
|-<br />
|BYTEA<br />
|bytea<br />
|variable-length byte array (null-byte safe)<br />
|-<br />
|"char" (with the quotes)<br />
|char<br />
|one byte<br />
|}<br />
<br />
You will see the internal name when examining system catalogs and in<br />
some error messages.<br />
<br />
The first four types above are "varlena" types (i.e., the field length<br />
is explicitly stored on disk, followed by the data). Thus the actual<br />
space used is slightly greater than the expected size. However, long<br />
values are also subject to compression, so the space on disk might<br />
also be less than expected.<br />
<br />
VARCHAR(n) is best when storing variable-length strings if a specific<br />
upper limit on the string length is required by the application.<br />
TEXT is for strings of "unlimited" length (though all fields in PostgreSQL<br />
are subject to a maximum value length of one gigabyte).<br />
<br />
CHAR(n) is for storing strings that are all the same length. CHAR(n)<br />
pads with blanks to the specified length, while VARCHAR(n) only stores<br />
the characters supplied. BYTEA is for storing binary data,<br />
particularly values that include zero bytes. All these types have similar<br />
performance characteristics, except that the blank-padding involved<br />
in CHAR(n) requires additional storage and some extra runtime.<br />
<br />
The "char" type (the quotes are required to distinguish it from CHAR(n))<br />
is a specialized datatype that can store exactly one byte. It is found in<br />
the system catalogs but its use in user tables is generally discouraged.<br />
<br />
=== How do I create a serial/auto-incrementing field? ===<br />
<br />
PostgreSQL supports a SERIAL data type. Actually, this isn't quite<br />
a real type. It's a shorthand for creating an integer column that<br />
is fed from a sequence.<br />
<br />
For example, this:<br />
CREATE TABLE person (<br />
id SERIAL,<br />
name TEXT<br />
);<br />
is automatically translated into this:<br />
CREATE SEQUENCE person_id_seq;<br />
CREATE TABLE person (<br />
id INTEGER NOT NULL DEFAULT nextval('person_id_seq'),<br />
name TEXT<br />
);<br />
<br />
The automatically created sequence is named ''table''_''serialcolumn''_seq,<br />
where ''table'' and ''serialcolumn'' are the names of the table and SERIAL<br />
column, respectively. See the CREATE SEQUENCE manual page for more<br />
information about sequences.<br />
<br />
There is also BIGSERIAL, which is like SERIAL except that the resulting<br />
column is of type BIGINT instead of INTEGER. Use this type if you think<br />
that you might need more than 2 billion serial values over the lifespan<br />
of the table.<br />
<br />
Note that sequences may contain "holes" or "gaps" as a normal part of operation. It is entirely normal for generated keys to go 1, 4, 5, 6, 9, ... . See [[#Why are there gaps in the numbering of my sequence/SERIAL column? Why aren't my sequence numbers reused on transaction abort?|the FAQ entry on sequence gaps]].<br />
<br />
=== How do I get the value of a SERIAL insert? ===<br />
<br />
The simplest way is to retrieve the assigned SERIAL value with<br />
RETURNING. Using the example table in the previous question, it would look like this:<br />
INSERT INTO person (name) VALUES ('Blaise Pascal') RETURNING id;<br />
<br />
You can also call nextval() and use that value in the INSERT, or call<br />
currval() after the INSERT.<br />
<br />
=== Doesn't currval() lead to a race condition with other users? ===<br />
<br />
No. currval() returns the latest sequence value assigned by your session,<br />
independently of what is happening in other sessions.<br />
<br />
=== Why are there gaps in the numbering of my sequence/SERIAL column? Why aren't my sequence numbers reused on transaction abort? ===<br />
<br />
To improve concurrency, sequence values are given out to running<br />
transactions on-demand; the sequence object is not kept locked but is<br />
immediately available for another transaction to get another sequence<br />
value. This causes gaps in numbering from aborted transactions, as documented in the [http://www.postgresql.org/docs/current/static/functions-sequence.html NOTE section for the nextval() function].<br />
<br />
Additionally, an unclean server shutdown will cause sequences to increment on recovery, because PostgreSQL keeps a cache of sequence numbers to hand out and in an unclean shutdown it isn't sure which of those cached numbers has already been used. Since sequences are allowed to have gaps anyway it takes the safe option and increments the sequence.<br />
<br />
Another cause for gaps in sequence is the use of the CACHE clause in [http://www.postgresql.org/docs/current/static/sql-createsequence.html CREATE SEQUENCE].<br />
<br />
In general, you should not rely on SERIAL keys or SEQUENCEs being gapless, nor should you make assumptions about their order; [http://www.postgresql.org/docs/current/static/sql-createsequence.html#AEN69802 it is ''not'' guaranteed that id n+1 was inserted after id n except when both were generated within the same transaction]. Compare synthetic keys for equality and only for equality.<br />
<br />
Gap-less sequences are possible, but are very bad for performance. At most one transaction at a time can be inserting rows from a gapless sequence. There is no built-in SERIAL or SEQUENCE equivalent for gap-less sequences, but one is [http://stackoverflow.com/a/9985219/398670 trivial to implement]. Information on gapless sequence implementations can be found in the mailing list archives, on Stack Overflow, and in [http://www.varlena.com/GeneralBits/130.php this useful article]. Avoid using a gap-less sequence unless it is an absolute business requirement. Consider dynamically generating the gap-less numbering on demand for display, using the [http://www.postgresql.org/docs/current/static/tutorial-window.html row_number() window function], or adding it in a batch process that runs periodically.<br />
<br />
See also: [http://www.neilconway.org/docs/sequences/ FAQ: Using sequences in PostgreSQL].<br />
<br />
=== What is an OID? ===<br />
<br />
If a table is created WITH OIDS, each row includes an OID column that is automatically filled in during INSERT.<br />
OIDs are sequentially assigned 4-byte integers. Initially they are unique<br />
across the entire installation. However, the OID counter wraps around at 4 billion,<br />
and after that OIDs may be duplicated.<br />
<br />
It is possible to prevent duplication of OIDs within a single table by<br />
creating a unique index on the OID column (but note that the WITH OIDS<br />
clause doesn't by itself create such an index).<br />
The system checks the index to see if a newly<br />
generated OID is already present, and if so generates a new OID and<br />
repeats. This works well so long as no OID-containing table has<br />
more than a small fraction of 4 billion rows. <br />
<br />
PostgreSQL uses OIDs for object identifiers in the system catalogs,<br />
where the size limit is unlikely to be a problem.<br />
<br />
To uniquely number rows in user tables, it is best to use SERIAL<br />
rather than an OID column, or BIGSERIAL if the table is expected to<br />
have more than 2 billion entries over its lifespan.<br />
<br />
=== What is a CTID? ===<br />
<br />
CTIDs identify specific physical rows by their block and<br />
offset positions within a table.<br />
They are used by index entries to point to physical rows.<br />
A logical row's CTID changes when it is updated, so the CTID<br />
cannot be used as a long-term row identifier. But it is sometimes<br />
useful to identify a row within a transaction when no competing<br />
update is expected.<br />
<br />
=== Why do I get the error "ERROR: Memory exhausted in AllocSetAlloc()"? ===<br />
<br />
You probably have run out of virtual memory on your system, or your<br />
kernel has a low limit for certain resources. Try this before starting<br />
the server:<br />
ulimit -d 262144<br />
limit datasize 256m<br />
<br />
Depending on your shell, only one of these may succeed, but it will<br />
set your process data segment limit much higher and perhaps allow the<br />
query to complete. This command applies to the current process, and<br />
all subprocesses created after the command is run. If you are having a<br />
problem with the SQL client because the backend is returning too much<br />
data, try it before starting the client.<br />
<br />
=== How do I tell what PostgreSQL version I am running? ===<br />
<br />
Run this query: SELECT version();<br />
<br />
=== Is there a way to leave an audit trail of database operations? ===<br />
<br />
There's nothing built-in, but it's not too difficult to build such<br />
facilities yourself.<br />
<br />
Simple example right in the official docs:<br />
http://www.postgresql.org/docs/current/static/plpgsql-trigger.html#PLPGSQL-TRIGGER-AUDIT-EXAMPLE<br />
<br />
Project targeting this feature: http://pgfoundry.org/projects/tablelog/<br />
<br />
Background information and other sample implementations: <br />
http://it.toolbox.com/blogs/database-soup/simple-data-auditing-19014<br />
http://www.go4expert.com/forums/showthread.php?t=7252<br />
http://www.alberton.info/postgresql_table_audit.html<br />
<br />
=== How do I create a column that will default to the current time? ===<br />
<br />
Use CURRENT_TIMESTAMP:<br />
CREATE TABLE test (x int, modtime TIMESTAMP DEFAULT CURRENT_TIMESTAMP );<br />
<br />
=== How do I perform an outer join? ===<br />
<br />
PostgreSQL supports outer joins using the SQL standard syntax. Here<br />
are two examples:<br />
SELECT *<br />
FROM t1 LEFT OUTER JOIN t2 ON (t1.col = t2.col);<br />
<br />
or<br />
SELECT *<br />
FROM t1 LEFT OUTER JOIN t2 USING (col);<br />
<br />
These identical queries join t1.col to t2.col, and also return any<br />
unjoined rows in t1 (those with no match in t2). A RIGHT join would<br />
add unjoined rows of t2. A FULL join would return the matched rows<br />
plus all unjoined rows from t1 and t2. The word OUTER is optional and<br />
is assumed in LEFT, RIGHT, and FULL joins. Ordinary joins are called<br />
INNER joins.<br />
<br />
=== How do I perform queries using multiple databases? ===<br />
<br />
There is no way to query a database other than the current one.<br />
Because PostgreSQL loads database-specific system catalogs, it is<br />
uncertain how a cross-database query should even behave.<br />
<br />
contrib/dblink allows cross-database queries using function calls. Of<br />
course, a client can also make simultaneous connections to different<br />
databases and merge the results on the client side.<br />
<br />
=== How do I return multiple rows or columns from a function? ===<br />
<br />
It is easy using set-returning functions,<br />
[[Return more than one row of data from PL/pgSQL functions]].<br />
<br />
=== Why do I get "relation with OID ##### does not exist" errors when accessing temporary tables in PL/PgSQL functions? ===<br />
<br />
In PostgreSQL versions < 8.3, PL/PgSQL caches function scripts, and an<br />
unfortunate side effect is that if a PL/PgSQL function accesses a<br />
temporary table, and that table is later dropped and recreated, and<br />
the function called again, the function will fail because the cached<br />
function contents still point to the old temporary table. The solution<br />
is to use EXECUTE for temporary table access in PL/PgSQL. This will<br />
cause the query to be reparsed every time.<br />
<br />
This problem does not occur in PostgreSQL 8.3 and later.<br />
<br />
=== What replication solutions are available? ===<br />
<br />
Though "replication" is a single term, there are several technologies<br />
for doing replication, with advantages and disadvantages for each.<br />
Our documentation contains a good introduction to this topic at<br />
http://www.postgresql.org/docs/current/static/high-availability.html and a<br />
grid listing replication software and features is at<br />
[[Replication, Clustering, and Connection Pooling]]<br />
<br />
Master/slave replication allows a single master to receive read/write<br />
queries, while slaves can only accept read/SELECT queries. The most<br />
popular freely available master-slave PostgreSQL replication solution<br />
is Slony-I.<br />
<br />
Multi-master replication allows read/write queries to be sent to<br />
multiple replicated computers. This capability also has a severe<br />
impact on performance due to the need to synchronize changes between<br />
servers. PGCluster is the most popular such solution freely available<br />
for PostgreSQL.<br />
<br />
There are also proprietary and hardware-based replication solutions<br />
available supporting a variety of replication models.<br />
<br />
=== Is possible to create a shared-storage postgresql server cluster? ===<br />
<br />
PostgreSQL does not support clustering using [[Shared_Storage|shared storage]] on a SAN, SCSI backplane,<br />
iSCSI volume, or other shared media. Such "RAC-style" clustering isn't supported.<br />
Only replication-based clustering is currently supported.<br />
<br />
See [[Replication, Clustering, and Connection Pooling]] information for details.<br />
<br />
[[Shared_Storage|Shared-storage]] 'failover' is possible, but it is not safe to have more than one<br />
postmaster running and accessing the data store at the same time. Heartbeat and<br />
[http://en.wikipedia.org/wiki/STONITH STONITH] or some other hard-disconnect option are recommended.<br />
<br />
=== Why are my table and column names not recognized in my query? Why is capitalization not preserved? ===<br />
<br />
The most common cause of unrecognized names is the use of<br />
double-quotes around table or column names during table creation. When<br />
double-quotes are used, table and column names (called identifiers)<br />
are stored case-sensitive, meaning you must use double-quotes when<br />
referencing the names in a query. Some interfaces, like pgAdmin,<br />
automatically double-quote identifiers during table creation. So, for<br />
identifiers to be recognized, you must either:<br />
<br />
* Avoid double-quoting identifiers when creating tables<br />
* Use only lowercase characters in identifiers<br />
* Double-quote identifiers when referencing them in queries<br />
<br />
=== I lost the database password. What can I do to recover it? ===<br />
<br />
You can't. However, you can reset it to something else. To do this, you<br />
<br />
* edit pg_hba.conf to allow ''trust'' authorization temporarily<br />
* Reload the config file (pg_ctl reload)<br />
* Connect and issue ALTER ROLE / PASSWORD to set the new password<br />
* edit pg_hba.conf again and restore the previous settings<br />
* Reload the config file again<br />
<br />
=== Does PostgreSQL have stored procedures? ===<br />
<br />
PostgreSQL doesn't. However PostgreSQL have very powerful functions and user-defined functions capabilities that can do most things that other RDBMS stored routines (procedures and functions) can do and in many cases more.<br />
<br />
These functions can be of different types and can be implemented in several programming languages.<br />
(Refer to documentation for more details. [http://www.postgresql.org/docs/current/static/xfunc.html User-Defined Functions])<br />
<br />
PostgreSQL functions can be invoked in many ways. If you want to invoke a function as you would call a stored procedure in other RDBMS (typically a function with side-effects but whose result you don't care for example because it returns void), one option would be to use [http://www.postgresql.org/docs/current/static/plpgsql.html PL/pgSQL Language] for your procedure and the [http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-NORESULT PERFORM] command. Example:<br />
PERFORM theNameOfTheFunction(arg1, arg2);<br />
<br />
Note that invoking instead:<br />
SELECT theNameOfTheFunction(arg1, arg2);<br />
would produce a result even if the function returns void (this result would be one row containing a void value).<br />
<br />
[http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-NORESULT PERFORM] could thus be used to discard this unuseful result.<br />
<br />
The main limitations on Pg's stored functions - as compared to true stored procedures - are:<br />
<br />
* inability to return multiple result sets<br />
* no support for autonomous transactions (<code>BEGIN</code>, <code>COMMIT</code> and <code>ROLLBACK</code> within a function)<br />
* no support for the SQL-standard <code>CALL</code> syntax, though the ODBC and JDBC drivers will translate calls for you.<br />
<br />
=== Why don't BEGIN, ROLLBACK and COMMIT work in stored procedures/functions? ===<br />
<br />
PostgreSQL doesn't support autonomous transactions in its stored functions. Like all PostgreSQL queries, stored functions always run in a transaction and cannot operate outside a transaction.<br />
<br />
If you need a stored procedure to manage transactions, you can look into the dblink interface or do the work from a client-side script instead. In some cases you can do what you need to using [http://www.postgresql.org/docs/current/static/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING exception blocks in PL/PgSQL], because each BEGIN/EXCEPTION/END block creates a subtransaction.<br />
<br />
=== Why is "SELECT count(*) FROM bigtable;" slow? ===<br />
<br />
It can't be answered directly from an index. PostgreSQL has to check the visibility for each record, so it<br />
forces a sequential scan of the entire table. If you want, you can keep track of the number of rows yourself with triggers, but beware that this will slow down write access to the table.<br />
<br />
You can get an estimation. The reltuples column in [http://www.postgresql.org/docs/current/static/catalog-pg-class.html pg_class] contains the information from the latest [http://www.postgresql.org/docs/current/static/sql-analyze.html ANALYZE] of the table. On a large table this is often accurate to within a few thousandths of a percent, which is accurate enough for many purposes.<br />
<br />
An "exact" count is often not exact for very long, anyway; due to [http://www.postgresql.org/docs/current/static/mvcc-intro.html MVCC] concurrency, the count will be accurate as of the moment the SELECT count(*) query (or, for stricter [http://www.postgresql.org/docs/current/static/transaction-iso.html transaction isolation] levels, its transaction) ''started'', and may well be out-of-date by the time the query completes. In a transaction mix where the table is being modified, two count(*) executions which return at the same moment might have different values, if a modifying transaction committed between their start times.<br />
<br />
For more information, see [[Slow Counting]].<br />
<br />
=== Why is my query much slower when run as a prepared query? ===<br />
<br />
When PostgreSQL has the full query with all parameters known by planning time, it can use statistics in the table to find out if the values used in the query are very common or very uncommon in a column. This lets it change the way it fetches the data to be more efficient, as it knows to expect lots or very few results from a certain part of the query. For example, it might choose an sequential scan instead of doing an index scan if you search for 'active=y' and it knows that 99% of the records in the table have 'active=y', because in this case a sequential scan will be much faster.<br />
<br />
In a prepared query, PostgreSQL doesn't have the value of all parameters when it's creating the plan. It has to try to pick a "safe" plan that should work fairly well no matter what value you supply as the parameter when you execute the prepared query. Unfortunately, this plan might not be very appropriate if the value you supply is vastly more common, or vastly less common, than is average for some randomly selected values in the table.<br />
<br />
If you suspect this issue is affecting you, start by using the [http://www.postgresql.org/docs/current/static/sql-explain.html EXPLAIN] command to compare the slow and fast queries. Look at the output of <code>EXPLAIN SELECT query...</code> and compare it to the result of <code>PREPARE query... ; EXPLAIN EXECUTE query...</code> to see if the plans are notably different. <code>EXPLAIN ANALYZE</code> may give you more information, such as row count estimates and counts.<br />
<br />
Usually people having this problem are trying to use prepared queries as a security measure to prevent SQL injection, rather than as a performance tuning option for expensive-to-plan queries frequently executed with a variety of different parameters. Those people should consider using client-side prepared statements if their client interface (eg PgJDBC) supports it.<br />
<br />
At present, PostgreSQL does not offer a way to request re-planning of a prepared statement using a particular set of parameter values; doing so somewhat defeats the purpose of server-side prepared statements. Running a statistics check to see if a particular parameter value is notably outside the norm and automatically re-planning in that case has been discussed, but not agreed upon or implemented as yet.<br />
<br />
See [[Using_EXPLAIN]]. If you're going to ask for help on the mailing lists, please read the [[Guide to reporting problems]].<br />
<br />
=== Why is my query much slower when run in a function than standalone? ===<br />
<br />
See [[FAQ#Why is my query much slower when run as a prepared query?]]. Queries in PL/PgSQL functions are prepared and cached, so they execute in much the same way as if you'd <code>PREPARE</code>d then <code>EXECUTE</code>d the query yourself.<br />
<br />
If you're having really severe issues with this that improving the table statistics or adjusting your query don't help with, you can work around it by forcing PL/PgSQL to re-prepare your query at every execution. To do this, use the <code>EXECUTE ... USING</code> statement in PL/PgSQL to supply your query as a textual string. Alternately, the [http://www.postgresql.org/docs/current/static/functions-string.html quote_literal or quote_nullable] functions may be used to escape parameters substituted into query text.<br />
<br />
=== Why do my strings sort incorrectly? ===<br />
<br />
First, make sure you are using the locale you want to be using. Use <code>SHOW lc_collate</code> to show the database-wide locale in effect. If you are using per-column collations, check those. If everything is how you want it, then read on.<br />
<br />
PostgreSQL uses the C library's locale facilities for sorting strings. So if the sort order of the strings is not what you expect, the issue is likely in the C library. You can verify the C library's idea of sorting using the <code>sort</code> utility on a text file, e.g.,<br />
<br />
LC_COLLATE=xx_YY.utf8 sort testfile.txt<br />
<br />
If this results in the same order that PostgreSQL gives you, then the problem is outside of PostgreSQL.<br />
<br />
PostgreSQL deviates from the libc behavior in so far as it breaks ties by sorting strings in byte order. This should rarely make a<br />
difference in practice, and is usually not the source of the problem when users complain about the sort order, but it could affect cases where, for example, combining and precombined Unicode characters are mixed.<br />
<br />
If the problem is in the C library, you will have to take it up with your operating system maintainers. Note, however, that while actual bugs in locale definitions of C libraries have been known to exist, it is more likely that the C library is correct, where "correct" means it follows some recognized international or national standard. Possibly, you are expecting one of multiple equally valid interpretations of a language's sorting rules.<br />
<br />
Common complaint patterns include:<br />
<br />
* Spaces and special characters: The sorting algorithm normally works in multiple passes. First, all the letters are compared, ignoring spaces and punctuation. Then, spaces and punctuation are compared to break ties. (This is a simplification of what actually happens.) It's not possible to change this without changing the locale definitions themselves (and even then it's difficult). You might want to restructure your data slightly to avoid this problem. For example, if you are sorting a name field, you could split the field into first and last name fields, avoiding the space in between.<br />
<br />
* Upper/lower case: Locales other than the C locale generally sort upper and lower case letters together. So the order will be something like a A b B c C ... instead of the A B C ... a b c ... that a sort based on ASCII byte values will give. That is correct.<br />
<br />
* German locale: sort order of ä as a or ae. Both of these are valid (see http://de.wikipedia.org/wiki/Alphabetische_Sortierung), but most C libraries only provide the first one. Fixing this would require creating a custom locale. This is possible, but will take some work.<br />
<br />
* It is not in ASCII/byte order. No, it's not, it's not supposed to be. ASCII is an encoding, not a sort order. If you want this, you can use the C locale, but then you use the ability to non-ASCII characters.<br />
<br />
That said, if you are on Mac OS X or a BSD-family operating system, and you are using UTF-8, then give up. The locale definitions on<br />
those operating systems are broken.<br />
<br />
[[Category:FAQ]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=FAQ&diff=18421FAQ2012-10-22T18:32:42Z<p>Schmiddy: /* Why are there gaps in the numbering of my sequence/SERIAL column? Why aren't my sequence numbers reused on transaction abort? */ fix link</p>
<hr />
<div>{{Languages}}<br />
[[:Category:FAQ|Additional FAQ Entries on this Wiki]]<br />
<br />
== Translations of this Document ==<br />
<br />
* [[Häufig gestellte Fragen|German]]<br />
* [[Perguntas Frequentes|Portuguese]]<br />
* [[Preguntas Frecuentes|Spanish]]<br />
* [[Часто Задаваемые Вопросы|Русский]]<br />
<br />
== Platform-specific questions ==<br />
<br />
Windows users should also read the [[Running & Installing PostgreSQL On Native Windows|platform FAQ for Windows]]. There are [[Frequently Asked Questions#Platform FAQs|FAQs for other platforms]] too.<br />
<br />
== General Questions ==<br />
<br />
=== What is PostgreSQL? How is it pronounced? What is Postgres? ===<br />
<br />
PostgreSQL is pronounced Post-Gres-Q-L. (For those curious about how<br />
to say "PostgreSQL", an [http://www.postgresql.org/files/postgresql.mp3 audio file] is available.)<br />
<br />
PostgreSQL is an object-relational database system that has the<br />
features of traditional proprietary database systems with enhancements<br />
to be found in next-generation DBMS systems. PostgreSQL is free and<br />
the complete source code is available.<br />
<br />
PostgreSQL development is performed by a team of mostly volunteer<br />
developers spread throughout the world and communicating via the<br />
Internet. It is a community project and is not controlled by any<br />
company. To get involved, see the [[Developer_FAQ | Developer FAQ]].<br />
<br />
Postgres is a widely-used nickname for PostgreSQL. It was the original<br />
name of the project at Berkeley and is strongly preferred over other<br />
nicknames. If you find 'PostgreSQL' hard to pronounce, call it<br />
'Postgres' instead.<br />
<br />
=== Who controls PostgreSQL? ===<br />
<br />
If you are looking for a PostgreSQL gatekeeper, central committee, or<br />
controlling company, give up --- there isn't one. We do have a core<br />
committee and CVS committers, but these groups are more for<br />
administrative purposes than control. The project is directed by the<br />
community of developers and users, which anyone can join. All you need<br />
to do is subscribe to the mailing lists and participate in the<br />
discussions. (See the [[Developer FAQ|Developer's FAQ]] for information on how to get<br />
involved in PostgreSQL development.)<br />
<br />
=== Who is the PostgreSQL Global Development Group? ===<br />
<br />
The "PGDG" is an international, unincorporated association of<br />
individuals and companies who have contributed to the PostgreSQL<br />
project. The PostgreSQL Core Team generally act as spokespeople<br />
for the PGDG.<br />
<br />
=== Who is the PostgreSQL Core Team? ===<br />
<br />
A committee of five to seven (currently six) senior contributors to<br />
PostgreSQL who do the following for the project: (a) set release dates,<br />
(b) handle confidential matters for the project, (c) act as spokespeople<br />
for the PGDG when required, and (d) arbitrate community decisions which<br />
are not settled by consensus. The current Core Team is listed on top of<br />
[http://www.postgresql.org/community/contributors/ the contributors page]<br />
<br />
=== What about the various PostgreSQL foundations? ===<br />
<br />
While the PostgreSQL project utilizes non-profit corporations in the<br />
USA, Europe, Brazil and Japan for fundraising and project coordination,<br />
these entities do not own the PostgreSQL code.<br />
<br />
=== What is the license of PostgreSQL? ===<br />
<br />
PostgreSQL is distributed under a license similar to BSD and MIT. Basically, it<br />
allows users to do anything they want with the code, including<br />
reselling binaries without the source code. The only restriction is<br />
that you not hold us legally liable for problems with the software.<br />
There is also the requirement that this copyright appear in all copies<br />
of the software. Here is the license we use:<br />
<br />
PostgreSQL Database Management System<br />
(formerly known as Postgres, then as Postgres95)<br />
<br />
Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group<br />
<br />
Portions Copyright (c) 1994, The Regents of the University of California<br />
<br />
Permission to use, copy, modify, and distribute this software and its<br />
documentation for any purpose, without fee, and without a written agreement<br />
is hereby granted, provided that the above copyright notice and this<br />
paragraph and the following two paragraphs appear in all copies.<br />
<br />
IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR<br />
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING<br />
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS<br />
DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE<br />
POSSIBILITY OF SUCH DAMAGE.<br />
<br />
THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,<br />
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY<br />
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS<br />
ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO<br />
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.<br />
<br />
=== What platforms does PostgreSQL support? ===<br />
<br />
In general, any modern Unix-compatible platform should be able to run<br />
PostgreSQL. The platforms that have received recent explicit testing<br />
can be seen in the [http://buildfarm.postgresql.org/ Build farm].<br />
The documentation contains more details about supported platforms at http://www.postgresql.org/docs/current/static/supported-platforms.html.<br />
<br />
PostgreSQL also runs natively on Microsoft Windows NT-based operating<br />
systems like Win2000 SP4, WinXP, and Win2003. A prepackaged installer<br />
is available at http://www.postgresql.org/download/windows.<br />
MSDOS-based versions of Windows (Win95, Win98, WinMe) can run<br />
PostgreSQL using Cygwin.<br />
<br />
=== Where can I get PostgreSQL? ===<br />
<br />
There are binary distributions for various operating systems and platforms; see [http://www.postgresql.org/download our download area].<br />
<br />
The source code can be obtained [http://www.postgresql.org/ftp/ via web browser] or [ftp://ftp.postgresql.org/pub/ through ftp].<br />
<br />
=== What is the most recent release? ===<br />
<br />
The latest release of PostgreSQL is shown on the front page of [http://www.postgresql.org/ our website].<br />
<br />
We typically have a major release every year, with minor releases every few months.<br />
Minor releases are usually made at the same time for all supported major-release branches.<br />
For more about major versus minor releases, see<br />
http://www.postgresql.org/support/versioning.<br />
<br />
=== Where can I get support? ===<br />
<br />
The PostgreSQL community provides assistance to many of its users via<br />
email. The main web site to subscribe to the email lists is<br />
http://www.postgresql.org/community/lists/. The general or bugs lists<br />
are a good place to start. For best results, consider reading the <br />
[[guide to reporting problems]] before you post to make sure you<br />
include enough information for people to help you.<br />
<br />
The major IRC channel is #postgresql on Freenode (irc.freenode.net).<br />
A Spanish one also<br />
exists on the same network, (#postgresql-es), a French one,<br />
(#postgresqlfr), and a Brazilian one, (#postgresql-br). There is also<br />
a PostgreSQL channel on EFNet.<br />
<br />
A list of support companies is available at<br />
http://www.postgresql.org/support/professional_support.<br />
<br />
=== How do I submit a bug report? ===<br />
<br />
Visit the PostgreSQL bug form at<br />
http://www.postgresql.org/support/submitbug to submit your bug<br />
report to the pgsql-bugs mailing list. Also check out our ftp<br />
site ftp://ftp.postgresql.org/pub/ to see if there is a more recent<br />
PostgreSQL version.<br />
<br />
For a prompt and helpful response, it is important for you to read the <br />
[[guide to reporting problems]] to make sure that you include the <br />
information required to fully understand and act on your report.<br />
<br />
Bugs submitted using the bug form or posted to any PostgreSQL mailing<br />
list typically generates one of the following replies:<br />
* It is not a bug, and why<br />
* It is a known bug and is already on the TODO list<br />
* The bug has been fixed in the current release<br />
* The bug has been fixed but is not packaged yet in an official release<br />
* A request is made for more detailed information:<br />
** Operating system<br />
** PostgreSQL version<br />
** Reproducible test case<br />
** Debugging information<br />
** [[Generating_a_stack_trace_of_a_PostgreSQL_backend|Debugger backtrace output]]<br />
* The bug is new. The following might happen:<br />
** A patch is created and will be included in the next major or minor release<br />
** The bug cannot be fixed immediately and is added to the TODO list<br />
<br />
=== How do I find out about known bugs or missing features? ===<br />
<br />
PostgreSQL supports an extended subset of SQL:2008. See our [[Todo|TODO list]]<br />
for known bugs, missing features, and future plans.<br />
<br />
A feature request usually results in one of the following replies:<br />
* The feature is already on the TODO list<br />
* The feature is not desired because:<br />
** It duplicates existing functionality that already follows the SQL standard<br />
** The feature would increase code complexity but add little benefit<br />
** The feature would be insecure or unreliable<br />
* The new feature is added to the TODO list<br />
<br />
PostgreSQL does not use a bug tracking system because we find it more<br />
efficient to respond directly to email and keep the TODO list<br />
up-to-date. In practice, bugs don't last very long in the software,<br />
and bugs that affect a large number of users are fixed rapidly. The<br />
only place to find all changes, improvements, and fixes in a<br />
PostgreSQL release is to read the CVS log messages. Even the release<br />
notes do not list every change made to the software.<br />
<br />
=== A bug I'm encountering is fixed in a newer minor release of PostgreSQL, but I don't want to upgrade. Can I get a patch for just this issue? ===<br />
<br />
No. Nobody will make a custom patch for you so you can (say) extract a fix from 8.4.3 and apply it to 8.4.1 . <br />
That's because there should never be any need to do that.<br />
<br />
PostgreSQL has a strict policy that only bug fixes are back-patched into point releases, as per the [http://www.postgresql.org/support/versioning version policy]. It is safe to upgrade from 8.4.1 to 8.4.3,<br />
for example. Binary compatibility will be maintained, no dump and reload is required, nothing will break, but bugs that might <br />
cause problems have been fixed. Even if you are not yet encountering a particular bug, you might later, and it is wise to upgrade promptly.<br />
You just have to install the update and re-start the database server, nothing more.<br />
<br />
Upgrading from 8.3 to 8.4, or 8.4 to 9.0, is a major upgrade that does not come with the same guarantees. However, if a bug<br />
is discovered in 9.0 then it will generally be fixed in all maintained older versions like 8.4 and 8.3 if it is safe and<br />
practical to do so.<br />
<br />
This means that if you're running 8.1.0, upgrading to 8.1.21 is <b>strongly</b> recommended and very safe. On the other hand,<br />
upgrading to the next major release, 8.2.x, may require changes to your app, and will certainly require a dump and reload.<br />
<br />
If you want to be careful about all upgrades, you should read the [http://www.postgresql.org/docs/current/static/release.html release notes] <br />
for each point release between your current one and the latest minor version of the same major release carefully. If you're<br />
exceptionally paranoid about upgrades, you can fetch the source code to each set of point release changes from [http://git.postgresql.org/ PostgreSQL's git repository] and examine it.<br />
<br />
It is strongly recommended that you <b>always</b> upgrade to the latest minor release. Avoid trying to extract and apply individual fixes<br />
from point releases; by doing so you're bypassing all the QA done by the PostgreSQL team when they prepare a release, and are creating your<br />
own custom version that <i>nobody else has ever used</i>. It's a lot safer to just update to the latest tested, safe release. <i>Patching your own custom, non-standard build will also take more time/effort, and will require the same amount of downtime as a normal upgrade.</i><br />
<br />
=== I have a program that says it wants PostgreSQL x.y.1. Can I use PostgreSQL x.y.2 instead? ===<br />
<br />
Any program that works with a particular version, like 8.4.1, should work with any other minor version in the same major version. That means that if a program says it wants (eg) 8.4.1, you can and should install the latest in the 8.4 series instead.<br />
<br />
See the previous question for more details.<br />
<br />
=== What documentation is available? ===<br />
<br />
PostgreSQL includes extensive documentation, including a large manual,<br />
manual pages, and some test examples. See the /doc directory. You can<br />
also browse the manuals online at http://www.postgresql.org/docs.<br />
<br />
There are a number of PostgreSQL<br />
books available for purchase; two of them are also available online. A list of books can be found at<br />
http://www.postgresql.org/docs/books/. One of the most popular ones is the one by Korry & Susan<br />
Douglas.<br />
<br />
There is also a collection of<br />
PostgreSQL technical articles on the <br />
[[Community_Generated_Articles%2C_Guides%2C_and_Documentation | wiki]].<br />
<br />
The command line client program psql has some \d commands to show<br />
information about types, operators, functions, aggregates, etc. - use<br />
\? to display the available commands.<br />
<br />
=== How can I learn SQL? ===<br />
<br />
First, consider the PostgreSQL-specific books mentioned above. Many of<br />
our users also like The Practical SQL Handbook, Bowman, Judith S., et<br />
al., Addison-Wesley. Others like The Complete Reference SQL, Groff et<br />
al., McGraw-Hill.<br />
<br />
Many people consider the PostgreSQL documentation to be an excellent guide<br />
for learning SQL its self, as well as for PostgreSQL's implementation of it.<br />
For best results use PostgreSQL alongside another full-featured SQL database as<br />
you learn, so you get used to SQL without becoming reliant on PostgreSQL-specific<br />
features. The PostgreSQL documentation generally mentions when features are PostgreSQL<br />
extensions of the standard.<br />
<br />
There are also many nice tutorials available online:<br />
* http://www.intermedia.net/support/sql/sqltut.shtm<br />
* http://sqlcourse.com<br />
* http://www.w3schools.com/sql/default.asp<br />
* http://mysite.verizon.net/Graeme_Birchall/id1.html<br />
* http://sqlzoo.net<br />
<br />
=== How do I submit a patch or join the development team? ===<br />
<br />
See the [[Developer FAQ|Developer's FAQ]].<br />
<br />
=== How does PostgreSQL compare to other DBMSs? ===<br />
<br />
There are several ways of measuring software: features, performance,<br />
reliability, support, and price.<br />
<br />
==== Features ====<br />
<br />
PostgreSQL has most features present in large proprietary DBMSs,<br />
like transactions, subselects, triggers, views, foreign key<br />
referential integrity, and sophisticated locking. We have some<br />
features they do not have, like user-defined types,<br />
inheritance, rules, and multi-version concurrency control to<br />
reduce lock contention.<br />
<br />
==== Performance ====<br />
<br />
PostgreSQL's performance is comparable to other proprietary and<br />
open source databases. It is faster for some things, slower for<br />
others. Our performance is usually +/-10% compared to other databases.<br />
<br />
==== Reliability ====<br />
<br />
We realize that a DBMS must be reliable, or it is worthless. We<br />
strive to release well-tested, stable code that has a minimum<br />
of bugs. Each release has at least one month of beta testing,<br />
and our release history shows that we can provide stable, solid<br />
releases that are ready for production use. We believe we<br />
compare favorably to other database software in this area.<br />
<br />
==== Support ====<br />
<br />
Our mailing lists provide contact with a large group of<br />
developers and users to help resolve any problems encountered.<br />
While we cannot guarantee a fix, proprietary DBMSs do not always<br />
supply a fix either. Direct access to developers, the user<br />
community, manuals, and the source code often make PostgreSQL<br />
support superior to other DBMSs. There is commercial<br />
per-incident support available for those who need it. (See [[#Where_can_I_get_support.3F | section 1.7]]).<br />
<br />
==== Price ====<br />
<br />
We are free for all use, both proprietary and open source.<br />
You can add our code to your product with no limitations,<br />
except those outlined in our BSD-style license stated above. <br />
<br />
=== Can PostgreSQL be embedded? ===<br />
<br />
PostgreSQL is designed as a client/server architecture, which requires<br />
separate processes for each client and server, and various helper<br />
processes. Many embedded architectures can support such requirements.<br />
However, if your embedded architecture requires the database server to<br />
run inside the application process, you cannot use Postgres and should<br />
select a lighter-weight database solution.<br />
<br />
Popular embeddable options include [http://sqlite.org SQLite] and [http://firebirdsql.org Firebird SQL].<br />
<br />
=== How do I unsubscribe from the PostgreSQL email lists? How do I avoid receiving duplicate emails? ===<br />
<br />
The PostgreSQL Majordomo page allows subscribing or unsubscribing from<br />
any of the PostgreSQL email lists. (You might need to have your<br />
Majordomo password emailed to you to log in.)<br />
<br />
All PostgreSQL email lists are configured so a group reply goes to the<br />
email list and the original email author. This is done so users<br />
receive the quickest possible email replies. If you would prefer not<br />
to receive duplicate email from the list in cases where you already<br />
receive an email directly, check eliminatecc from the Majordomo Change<br />
Settings page. You can also prevent yourself from receiving copies of<br />
emails you post to the lists by unchecking selfcopy.<br />
<br />
== User Client Questions ==<br />
<br />
=== What interfaces are available for PostgreSQL? ===<br />
<br />
The PostgreSQL install includes only the C and embedded C interfaces.<br />
All other interfaces are independent projects that are downloaded<br />
separately; being separate allows them to have their own release<br />
schedule and development teams.<br />
<br />
Some programming languages like PHP include an interface to<br />
PostgreSQL. Interfaces for languages like Perl, TCL, Python, and many<br />
others are available at http://pgfoundry.org.<br />
<br />
=== What tools are available for using PostgreSQL with Web pages? ===<br />
<br />
A nice introduction to Database-backed Web pages can be seen at:<br />
http://www.webreview.com<br />
<br />
For Web integration, PHP (http://www.php.net) is an excellent<br />
interface.<br />
<br />
For complex cases, many use the Perl and DBD::Pg with CGI.pm or<br />
mod_perl.<br />
<br />
=== Does PostgreSQL have a graphical user interface? ===<br />
<br />
There are a large number of GUI Tools that are available for<br />
PostgreSQL from both proprietary and open source developers. A detailed<br />
list can be found in the [[Community Guide to PostgreSQL GUI Tools]].<br />
<br />
== Administrative Questions ==<br />
<br />
=== How do I install PostgreSQL somewhere other than /usr/local/pgsql? ===<br />
<br />
Specify the --prefix option when running configure.<br />
<br />
=== I'm installing PostgreSQL and don't know the password for the postgres user ===<br />
<br />
Dave Page wrote a [http://pgsnake.blogspot.com/2010/07/postgresql-passwords-and-installers.html blog post] explaining what the different passwords are used for, and how to overcome common problems such as resetting them.<br />
<br />
=== How do I control connections from other hosts? ===<br />
<br />
By default, PostgreSQL only allows connections from the local machine<br />
using Unix domain sockets or TCP/IP connections. Other machines will<br />
not be able to connect unless you modify listen_addresses in the<br />
postgresql.conf file, enable host-based authentication by modifying<br />
the $PGDATA/pg_hba.conf file, and restart the database server.<br />
<br />
=== How do I tune the database engine for better performance? ===<br />
<br />
There are three major areas for potential performance improvement:<br />
<br />
==== Query Changes ====<br />
This involves modifying queries to obtain better performance:<br />
<br />
* Creation of indexes, including expression and partial indexes<br />
* Use of COPY instead of multiple INSERTs<br />
* Grouping of multiple statements into a single transaction to reduce commit overhead<br />
* Use of CLUSTER when retrieving many rows from an index<br />
* Use of LIMIT for returning a subset of a query's output<br />
* Use of Prepared queries<br />
* Use of ANALYZE to maintain accurate optimizer statistics<br />
* Regular use of VACUUM or pg_autovacuum<br />
* Dropping of indexes during large data changes<br />
<br />
==== Server Configuration ====<br />
A number of postgresql.conf settings affect performance. For<br />
more details, see Administration Guide/Server Run-time<br />
Environment/Run-time Configuration.<br />
<br />
==== Hardware Selection ====<br />
The effect of hardware on performance is detailed in<br />
http://www.powerpostgresql.com/PerfList/ and<br />
http://momjian.us/main/writings/pgsql/hw_performance/index.html.<br />
<br />
=== What debugging features are available? ===<br />
<br />
There are many log_* server configuration variables at http://www.postgresql.org/docs/current/interactive/runtime-config-logging.html that enable printing of query and process statistics which can be very useful for debugging and performance measurements.<br />
<br />
=== Why do I get "Sorry, too many clients" when trying to connect? ===<br />
<br />
You have reached the default limit of 100 database sessions. You need<br />
to increase the server's limit on how many concurrent backend<br />
processes it can start by changing the max_connections value in<br />
postgresql.conf and restarting the server.<br />
<br />
=== What is the upgrade process for PostgreSQL? ===<br />
<br />
See http://www.postgresql.org/support/versioning for a general<br />
discussion about upgrading, and<br />
http://www.postgresql.org/docs/current/static/install-upgrading.html<br />
for specific instructions.<br />
<br />
=== Will PostgreSQL handle recent daylight saving time changes in various countries? ===<br />
<br />
PostgreSQL releases 8.0 and up depend on the widely-used tzdata database<br />
(also called the zoneinfo database or the [http://www.twinsun.com/tz/tz-link.htm Olson timezone database]) for<br />
daylight savings information. To deal with a DST law change that affects you,<br />
install a new tzdata file set and restart the server.<br />
<br />
All PostgreSQL update releases include the latest available tzdata files,<br />
so keeping up-to-date on minor releases for your major version is usually<br />
sufficient for this.<br />
<br />
On platforms that receive regular software updates including new tzdata files,<br />
it may be more convenient to rely on the system's copy of the tzdata files.<br />
This is possible as a compile-time option. Most Linux distributions choose<br />
this approach for their pre-built versions of PostgreSQL.<br />
<br />
PostgreSQL releases before 8.0 always rely on the operating system's timezone<br />
information.<br />
<br />
=== What computer hardware should I use? ===<br />
<br />
Because PC hardware is mostly compatible, people tend to believe that<br />
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and<br />
quality motherboards are more reliable and have better performance<br />
than less expensive hardware. PostgreSQL will run on almost any<br />
hardware, but if reliability and performance are important it is wise<br />
to research your hardware options thoroughly. <br />
<br />
Database servers, unlike many other applications, are usually I/O and memory constrained, so it is wise to focus on the I/O subsystem first, then memory capacity, and lastly consider CPU issues. For example, a disk controller with a<br />
battery-backed cache is often the cheapest and easiest way to improve database performance. Our email lists can be used to discuss hardware options and tradeoffs.<br />
<br />
=== How does PostgreSQL use CPU resources? ===<br />
<br />
The PostgreSQL server is process-based (not threaded), and uses one operating system process per database session. A single database session (connection) cannot utilize more than one CPU. Of course, multiple sessions are automatically spread across all available CPUs by your operating system. Client applications can easily use threads and create multiple database connections from each thread.<br />
<br />
A single complex and CPU-intensive query is unable to use more than one CPU to do the processing for the query. The OS may still be able to use others for disk I/O etc, but you won't see much benefit from more than one spare core.<br />
<br />
=== Why does PostgreSQL have so many processes, even when idle? ===<br />
<br />
As noted in [[FAQ#How does PostgreSQL use CPU resources?|the answer above]], PostgreSQL is process based, so it starts one <code>postgres</code> (or <code>postgres.exe</code> on Windows) instance per connection. The postmaster (which accepts connections and starts new postgres instances for them) is always running. In addition, PostgreSQL generally has one or more "helper" processes like the stats collector, background writer, autovacuum daemon, walsender, etc, all of which show up as "postgres" instances in most system monitoring tools.<br />
<br />
Despite the number of processes, they actually use very little in the way of real resources. See [[FAQ#Why does PostgreSQL show up as using so much memory in my system monitoring tool?|the next answer]].<br />
<br />
=== Why does PostgreSQL use so much memory? ===<br />
<br />
Despite appearances, this is absolutely normal, and there's actually nowhere near as much memory being used as tools like <code>top</code> or the Windows process monitor say PostgreSQL is using.<br />
<br />
Tools like <code>top</code> and the Windows process monitor may show many <code>postgres</code> instances (see above), each of which appears to use a huge amount of memory. Often, when added up, the amount the postgres instances use is many times the amount of memory actually installed in the computer!<br />
<br />
This is a consequence of how these tools report memory use. They generally don't understand shared memory very well, and show it as if it was memory used individually and exclusively by each postgres instance. PostgreSQL uses a big chunk of shared memory to communicate between its backends and cache data. Because these tools count that shared memory block once per <code>postgres</code> instance instead of counting it once for <i>all</i> <code>postgres</code> instances, they massively over-estimate how much memory PostgreSQL is using.<br />
<br />
Furthermore, many versions of these tools don't report the entire shared memory block as being used by an individual instance immediately when it starts, but rather count the number of shared pages it has touched since starting. Over the lifetime of an instance, it will inevitably touch more and more of the shared memory until it has touched every page, so that its reported usage will gradually rise to include the entire shared memory block. This is frequently misinterpreted to be a memory leak; but it is no such thing, only a reporting artifact.<br />
<br />
== Operational Questions ==<br />
<br />
=== How do I SELECT only the first few rows of a query? A random row? ===<br />
<br />
To retrieve only a few rows, if you know at the number of rows needed<br />
at the time of the SELECT use LIMIT . If an index matches the ORDER BY<br />
it is possible the entire query does not have to be executed. If you<br />
don't know the number of rows at SELECT time, use a cursor and FETCH.<br />
<br />
To SELECT a random row, use:<br />
SELECT col<br />
FROM tab<br />
ORDER BY random()<br />
LIMIT 1;<br />
<br />
See also this [http://blog.rhodiumtoad.org.uk/2009/03/08/selecting-random-rows-from-a-table/ blog entry by Andrew Gierth]<br />
that has more information on this topic.<br />
<br />
=== How do I find out what tables, indexes, databases, and users are defined? How do I see the queries used by psql to display them? ===<br />
<br />
Use the \dt command to see tables in psql. For a complete list of<br />
commands inside psql you can use \?. Alternatively you can read the<br />
source code for psql in file pgsql/src/bin/psql/describe.c, it<br />
contains SQL commands that generate the output for psql's backslash<br />
commands. You can also start psql with the -E option so it will print<br />
out the queries it uses to execute the commands you give. PostgreSQL<br />
also provides an SQL compliant INFORMATION SCHEMA interface you can<br />
query to get information about the database.<br />
<br />
There are also system tables beginning with pg_ that describe these<br />
too.<br />
<br />
Use psql -l will list all databases.<br />
<br />
Also try the file pgsql/src/tutorial/syscat.source. It illustrates<br />
many of the SELECTs needed to get information from the database system<br />
tables.<br />
<br />
=== How do you change a column's data type? ===<br />
<br />
Changing the data type of a column can be done easily in 8.0 and later<br />
with ALTER TABLE ALTER COLUMN TYPE.<br />
<br />
In earlier releases, do this:<br />
BEGIN;<br />
ALTER TABLE tab ADD COLUMN new_col new_data_type;<br />
UPDATE tab SET new_col = CAST(old_col AS new_data_type);<br />
ALTER TABLE tab DROP COLUMN old_col;<br />
COMMIT;<br />
<br />
You might then want to do VACUUM FULL tab to reclaim the disk space<br />
used by the expired rows.<br />
<br />
=== What is the maximum size for a row, a table, and a database? ===<br />
<br />
These are the limits:<br />
<br />
Maximum size for a database? unlimited (32 TB databases exist)<br />
Maximum size for a table? 32 TB<br />
Maximum size for a row? 400 GB<br />
Maximum size for a field? 1 GB<br />
Maximum number of rows in a table? unlimited<br />
Maximum number of columns in a table? 250-1600 depending on column types<br />
Maximum number of indexes on a table? unlimited<br />
<br />
Of course, these are not actually unlimited, but limited to available<br />
disk space and memory/swap space. Performance may suffer when these<br />
values get unusually large.<br />
<br />
The maximum table size of 32 TB does not require large file support<br />
from the operating system. Large tables are stored as multiple 1 GB<br />
files so file system size limits are not important.<br />
<br />
The maximum table size, row size, and maximum number of columns can be<br />
quadrupled by increasing the default block size to 32k. The maximum<br />
table size can also be increased using table partitioning.<br />
<br />
One limitation is that indexes can not be created on columns longer<br />
than about 2,000 characters. Fortunately, such indexes are rarely<br />
needed. Uniqueness is best guaranteed by a function index of an MD5<br />
hash of the long column, and full text indexing allows for searching<br />
of words within the column.<br />
<br />
=== How much database disk space is required to store data from a typical text file? ===<br />
<br />
A PostgreSQL database may require up to five times the disk space to<br />
store data from a text file.<br />
<br />
As an example, consider a file of 100,000 lines with an integer and<br />
text description on each line. Suppose the text string averages<br />
twenty bytes in length. The flat file would be 2.8 MB. The size of the<br />
PostgreSQL database file containing this data can be estimated as 5.2<br />
MB:<br />
24 bytes: each row header (approximate)<br />
24 bytes: one int field and one text field<br />
+ 4 bytes: pointer on page to tuple<br />
----------------------------------------<br />
52 bytes per row<br />
<br />
The data page size in PostgreSQL is 8192 bytes (8 KB), so:<br />
<br />
8192 bytes per page<br />
------------------- = 158 rows per database page (rounded down)<br />
52 bytes per row<br />
<br />
100000 data rows<br />
------------------ = 633 database pages (rounded up)<br />
158 rows per page<br />
<br />
633 database pages * 8192 bytes per page = 5,185,536 bytes (5.2 MB)<br />
<br />
Indexes do not require as much overhead, but do contain the data that<br />
is being indexed, so they can be large also.<br />
<br />
NULLs are stored as bitmaps, so they use very little space.<br />
<br />
Note that long values may be compressed transparently.<br />
<br />
See also this presentation on the topic: [[Image:How_Long_Is_a_String.pdf]].<br />
<br />
=== Why are my queries slow? Why don't they use my indexes? ===<br />
<br />
Indexes are not used by every query. Indexes are used only if the<br />
table is larger than a minimum size, and the query selects only a<br />
small percentage of the rows in the table. This is because the random<br />
disk access caused by an index scan can be slower than a straight read<br />
through the table, or sequential scan.<br />
<br />
To determine if an index should be used, PostgreSQL must have<br />
statistics about the table. These statistics are collected using<br />
VACUUM ANALYZE, or simply ANALYZE. Using statistics, the optimizer<br />
knows how many rows are in the table, and can better determine if<br />
indexes should be used. Statistics are also valuable in determining<br />
optimal join order and join methods. Statistics collection should be<br />
performed periodically as the contents of the table change.<br />
<br />
Indexes are normally not used for ORDER BY or to perform joins. A<br />
sequential scan followed by an explicit sort is usually faster than an<br />
index scan of a large table. However, LIMIT combined with ORDER BY<br />
often will use an index because only a small portion of the table is<br />
returned.<br />
<br />
If you believe the optimizer is incorrect in choosing a sequential<br />
scan, use SET enable_seqscan TO 'off' and run query again to see if an<br />
index scan is indeed faster.<br />
<br />
When using wild-card operators such as LIKE or ~, indexes can only be<br />
used in certain circumstances:<br />
<br />
* The beginning of the search string must be anchored to the start of the string, i.e.<br />
** LIKE patterns must not start with % or _.<br />
** ~ (regular expression) patterns must start with ^.<br />
<br />
* The search string can not start with a character class, e.g. [a-e].<br />
<br />
* Case-insensitive searches such as ILIKE and ~* do not utilize indexes. Instead, use expression indexes, which are described in [[#How_do_I_perform_regular_expression_searches_and_case-insensitive_regular_expression_searches.3F_How_do_I_use_an_index_for_case-insensitive_searches.3F | section 4.8]].<br />
<br />
* C locale must be used during initdb because sorting in a non-C locale often doesn't match the behavior of LIKE. You can create a special text_pattern_ops index that will work in such cases, but note it is only helpful for LIKE indexing.<br />
<br />
It is also possible to use full text indexing for word searches.<br />
<br />
The [[SlowQueryQuestions]] article contains some more tips and guidance.<br />
<br />
=== How do I see how the query optimizer is evaluating my query? ===<br />
<br />
This is done with the EXPLAIN command; see [[Using EXPLAIN]].<br />
<br />
=== How do I change the sort ordering of textual data? ===<br />
<br />
PostgreSQL sorts textual data according to the ordering that is defined by the current locale, which is selected during initdb.<br />
(In 8.4 and up it will be possible to select a different locale when creating a new database.)<br />
If you don't like the ordering then you need to use a different locale.<br />
In particular, most locales other than "C" sort according to dictionary order, which largely ignores punctuation and spacing.<br />
If that's not what you want then you need "C" locale.<br />
<br />
=== How do I perform regular expression searches and case-insensitive regular expression searches? How do I use an index for case-insensitive searches? ===<br />
<br />
The ~ operator does regular expression matching, and ~* does<br />
case-insensitive regular expression matching. The case-insensitive<br />
variant of LIKE is called ILIKE.<br />
<br />
Case-insensitive equality comparisons are normally expressed as:<br />
SELECT *<br />
FROM tab<br />
WHERE lower(col) = 'abc';<br />
<br />
This will not use a standard index on "col". However, if you create an<br />
expression index on "lower(col)", it will be used:<br />
CREATE INDEX tabindex ON tab (lower(col));<br />
<br />
If the above index is created as UNIQUE, then the column can store<br />
upper and lowercase characters, but it cannot contain identical values that<br />
differ only in case. To force a particular case to be stored in the<br />
column, use a CHECK constraint or a trigger.<br />
<br />
In PostgreSQL 8.4 and later, you can also use the contributed [http://www.postgresql.org/docs/8.4/static/citext.html CITEXT data type], which internally implements the "lower()" calls, so that you can effectively treat it as a fully case-insensitive data type. CITEXT is also [https://svn.kineticode.com/citext/trunk/ available for 8.3], and an earlier version that treats only ASCII characters case-insensitively on 8.2 and earlier is available on [http://pgfoundry.org/projects/citext/ pgFoundry].<br />
<br />
=== In a query, how do I detect if a field is NULL? How do I concatenate possible NULLs? How can I sort on whether a field is NULL or not? ===<br />
<br />
You can test the value with IS NULL or IS NOT NULL, like this:<br />
SELECT *<br />
FROM tab<br />
WHERE col IS NULL;<br />
<br />
Concatenating a NULL with something else produces another NULL.<br />
If that's not what you want, you can replace the NULL(s) using<br />
COALESCE(), like this:<br />
<nowiki>SELECT COALESCE(col1, '') || COALESCE(col2, '')</nowiki><br />
FROM tab;<br />
<br />
To sort by the NULL status, use an IS NULL or IS NOT NULL test<br />
in your ORDER BY clause. Things that are true will sort higher than<br />
things that are false, so the following will put NULL entries at the<br />
front of the output:<br />
SELECT *<br />
FROM tab<br />
ORDER BY (col IS NOT NULL), col;<br />
<br />
In PostgreSQL 8.3 and up, you can also control sort ordering of NULLs<br />
using the recently-standardized NULLS FIRST/NULLS LAST modifiers,<br />
like this:<br />
SELECT *<br />
FROM tab<br />
ORDER BY col NULLS FIRST;<br />
<br />
=== What is the difference between the various character types? ===<br />
<br />
{| border="1"<br />
!Type<br />
!Internal Name<br />
!Notes<br />
|-<br />
|VARCHAR(n) <br />
|varchar<br />
|size specifies maximum length, no padding<br />
|- <br />
|CHAR(n)<br />
|bpchar<br />
|blank-padded to the specified fixed length<br />
|-<br />
|TEXT<br />
|text<br />
|no specific upper limit on length<br />
|-<br />
|BYTEA<br />
|bytea<br />
|variable-length byte array (null-byte safe)<br />
|-<br />
|"char" (with the quotes)<br />
|char<br />
|one byte<br />
|}<br />
<br />
You will see the internal name when examining system catalogs and in<br />
some error messages.<br />
<br />
The first four types above are "varlena" types (i.e., the field length<br />
is explicitly stored on disk, followed by the data). Thus the actual<br />
space used is slightly greater than the expected size. However, long<br />
values are also subject to compression, so the space on disk might<br />
also be less than expected.<br />
<br />
VARCHAR(n) is best when storing variable-length strings if a specific<br />
upper limit on the string length is required by the application.<br />
TEXT is for strings of "unlimited" length (though all fields in PostgreSQL<br />
are subject to a maximum value length of one gigabyte).<br />
<br />
CHAR(n) is for storing strings that are all the same length. CHAR(n)<br />
pads with blanks to the specified length, while VARCHAR(n) only stores<br />
the characters supplied. BYTEA is for storing binary data,<br />
particularly values that include zero bytes. All these types have similar<br />
performance characteristics, except that the blank-padding involved<br />
in CHAR(n) requires additional storage and some extra runtime.<br />
<br />
The "char" type (the quotes are required to distinguish it from CHAR(n))<br />
is a specialized datatype that can store exactly one byte. It is found in<br />
the system catalogs but its use in user tables is generally discouraged.<br />
<br />
=== How do I create a serial/auto-incrementing field? ===<br />
<br />
PostgreSQL supports a SERIAL data type. Actually, this isn't quite<br />
a real type. It's a shorthand for creating an integer column that<br />
is fed from a sequence.<br />
<br />
For example, this:<br />
CREATE TABLE person (<br />
id SERIAL,<br />
name TEXT<br />
);<br />
is automatically translated into this:<br />
CREATE SEQUENCE person_id_seq;<br />
CREATE TABLE person (<br />
id INTEGER NOT NULL DEFAULT nextval('person_id_seq'),<br />
name TEXT<br />
);<br />
<br />
The automatically created sequence is named ''table''_''serialcolumn''_seq,<br />
where ''table'' and ''serialcolumn'' are the names of the table and SERIAL<br />
column, respectively. See the CREATE SEQUENCE manual page for more<br />
information about sequences.<br />
<br />
There is also BIGSERIAL, which is like SERIAL except that the resulting<br />
column is of type BIGINT instead of INTEGER. Use this type if you think<br />
that you might need more than 2 billion serial values over the lifespan<br />
of the table.<br />
<br />
Note that sequences may contain "holes" or "gaps" as a normal part of operation. It is entirely normal for generated keys to go 1, 4, 5, 6, 9, ... . See [[#Why are there gaps in the numbering of my sequence/SERIAL column? Why aren't my sequence numbers reused on transaction abort?|the FAQ entry on sequence gaps]].<br />
<br />
=== How do I get the value of a SERIAL insert? ===<br />
<br />
The simplest way is to retrieve the assigned SERIAL value with<br />
RETURNING. Using the example table in the previous question, it would look like this:<br />
INSERT INTO person (name) VALUES ('Blaise Pascal') RETURNING id;<br />
<br />
You can also call nextval() and use that value in the INSERT, or call<br />
currval() after the INSERT.<br />
<br />
=== Doesn't currval() lead to a race condition with other users? ===<br />
<br />
No. currval() returns the latest sequence value assigned by your session,<br />
independently of what is happening in other sessions.<br />
<br />
=== Why are there gaps in the numbering of my sequence/SERIAL column? Why aren't my sequence numbers reused on transaction abort? ===<br />
<br />
To improve concurrency, sequence values are given out to running<br />
transactions on-demand; the sequence object is not kept locked but is<br />
immediately available for another transaction to get another sequence<br />
value. This causes gaps in numbering from aborted transactions, as documented in the [http://www.postgresql.org/docs/current/static/functions-sequence.html NOTE section for the nextval() function].<br />
<br />
Additionally, an unclean server shutdown will cause sequences to increment on recovery, because PostgreSQL keeps a cache of sequence numbers to hand out and in an unclean shutdown it isn't sure which of those cached numbers has already been used. Since sequences are allowed to have gaps anyway it takes the safe option and increments the sequence.<br />
<br />
Another cause for gaps in sequence is the use of the CACHE clause in [http://www.postgresql.org/docs/current/static/sql-createsequence.html CREATE SEQUENCE].<br />
<br />
In general, you should not rely on SERIAL keys or SEQUENCEs being gapless, nor should you make assumptions about their order; [http://www.postgresql.org/docs/current/static/sql-createsequence.html#AEN69802 it is ''not'' guaranteed that id n+1 was inserted after id n except when both were generated within the same transaction]. Compare synthetic keys for equality and only for equality.<br />
<br />
Gap-less sequences are possible, but are very bad for performance. At most one transaction at a time can be inserting rows from a gapless sequence. There is no built-in SERIAL or SEQUENCE equivalent for gap-less sequences, but one is [http://stackoverflow.com/a/9985219/398670 trivial to implement]. Information on gapless sequence implementations can be found in the mailing list archives, on Stack Overflow, and in [http://www.varlena.com/GeneralBits/130.php this useful article]. Avoid using a gap-less sequence unless it is an absolute business requirement. Consider dynamically generating the gap-less numbering on demand for display, using the [http://www.postgresql.org/docs/current/static/tutorial-window.html row_number() window function], or adding it in a batch process that runs periodically.<br />
<br />
See also: [http://www.neilconway.org/docs/sequences/ FAQ: Using sequences in PostgreSQL].<br />
<br />
=== What is an OID? ===<br />
<br />
If a table is created WITH OIDS, each row includes an OID column that is automatically filled in during INSERT.<br />
OIDs are sequentially assigned 4-byte integers. Initially they are unique<br />
across the entire installation. However, the OID counter wraps around at 4 billion,<br />
and after that OIDs may be duplicated.<br />
<br />
It is possible to prevent duplication of OIDs within a single table by<br />
creating a unique index on the OID column (but note that the WITH OIDS<br />
clause doesn't by itself create such an index).<br />
The system checks the index to see if a newly<br />
generated OID is already present, and if so generates a new OID and<br />
repeats. This works well so long as no OID-containing table has<br />
more than a small fraction of 4 billion rows. <br />
<br />
PostgreSQL uses OIDs for object identifiers in the system catalogs,<br />
where the size limit is unlikely to be a problem.<br />
<br />
To uniquely number rows in user tables, it is best to use SERIAL<br />
rather than an OID column, or BIGSERIAL if the table is expected to<br />
have more than 2 billion entries over its lifespan.<br />
<br />
=== What is a CTID? ===<br />
<br />
CTIDs identify specific physical rows by their block and<br />
offset positions within a table.<br />
They are used by index entries to point to physical rows.<br />
A logical row's CTID changes when it is updated, so the CTID<br />
cannot be used as a long-term row identifier. But it is sometimes<br />
useful to identify a row within a transaction when no competing<br />
update is expected.<br />
<br />
=== Why do I get the error "ERROR: Memory exhausted in AllocSetAlloc()"? ===<br />
<br />
You probably have run out of virtual memory on your system, or your<br />
kernel has a low limit for certain resources. Try this before starting<br />
the server:<br />
ulimit -d 262144<br />
limit datasize 256m<br />
<br />
Depending on your shell, only one of these may succeed, but it will<br />
set your process data segment limit much higher and perhaps allow the<br />
query to complete. This command applies to the current process, and<br />
all subprocesses created after the command is run. If you are having a<br />
problem with the SQL client because the backend is returning too much<br />
data, try it before starting the client.<br />
<br />
=== How do I tell what PostgreSQL version I am running? ===<br />
<br />
Run this query: SELECT version();<br />
<br />
=== Is there a way to leave an audit trail of database operations? ===<br />
<br />
There's nothing built-in, but it's not too difficult to build such<br />
facilities yourself.<br />
<br />
Simple example right in the official docs:<br />
http://www.postgresql.org/docs/8.3/static/plpgsql-trigger.html#PLPGSQL-TRIGGER-AUDIT-EXAMPLE<br />
<br />
Project targeting this feature: http://pgfoundry.org/projects/tablelog/<br />
<br />
Background information and other sample implementations: <br />
http://it.toolbox.com/blogs/database-soup/simple-data-auditing-19014<br />
http://www.go4expert.com/forums/showthread.php?t=7252<br />
http://www.alberton.info/postgresql_table_audit.html<br />
<br />
=== How do I create a column that will default to the current time? ===<br />
<br />
Use CURRENT_TIMESTAMP:<br />
CREATE TABLE test (x int, modtime TIMESTAMP DEFAULT CURRENT_TIMESTAMP );<br />
<br />
=== How do I perform an outer join? ===<br />
<br />
PostgreSQL supports outer joins using the SQL standard syntax. Here<br />
are two examples:<br />
SELECT *<br />
FROM t1 LEFT OUTER JOIN t2 ON (t1.col = t2.col);<br />
<br />
or<br />
SELECT *<br />
FROM t1 LEFT OUTER JOIN t2 USING (col);<br />
<br />
These identical queries join t1.col to t2.col, and also return any<br />
unjoined rows in t1 (those with no match in t2). A RIGHT join would<br />
add unjoined rows of t2. A FULL join would return the matched rows<br />
plus all unjoined rows from t1 and t2. The word OUTER is optional and<br />
is assumed in LEFT, RIGHT, and FULL joins. Ordinary joins are called<br />
INNER joins.<br />
<br />
=== How do I perform queries using multiple databases? ===<br />
<br />
There is no way to query a database other than the current one.<br />
Because PostgreSQL loads database-specific system catalogs, it is<br />
uncertain how a cross-database query should even behave.<br />
<br />
contrib/dblink allows cross-database queries using function calls. Of<br />
course, a client can also make simultaneous connections to different<br />
databases and merge the results on the client side.<br />
<br />
=== How do I return multiple rows or columns from a function? ===<br />
<br />
It is easy using set-returning functions,<br />
[[Return more than one row of data from PL/pgSQL functions]].<br />
<br />
=== Why do I get "relation with OID ##### does not exist" errors when accessing temporary tables in PL/PgSQL functions? ===<br />
<br />
In PostgreSQL versions < 8.3, PL/PgSQL caches function scripts, and an<br />
unfortunate side effect is that if a PL/PgSQL function accesses a<br />
temporary table, and that table is later dropped and recreated, and<br />
the function called again, the function will fail because the cached<br />
function contents still point to the old temporary table. The solution<br />
is to use EXECUTE for temporary table access in PL/PgSQL. This will<br />
cause the query to be reparsed every time.<br />
<br />
This problem does not occur in PostgreSQL 8.3 and later.<br />
<br />
=== What replication solutions are available? ===<br />
<br />
Though "replication" is a single term, there are several technologies<br />
for doing replication, with advantages and disadvantages for each.<br />
Our documentation contains a good introduction to this topic at<br />
http://www.postgresql.org/docs/8.3/static/high-availability.html and a<br />
grid listing replication software and features is at<br />
[[Replication, Clustering, and Connection Pooling]]<br />
<br />
Master/slave replication allows a single master to receive read/write<br />
queries, while slaves can only accept read/SELECT queries. The most<br />
popular freely available master-slave PostgreSQL replication solution<br />
is Slony-I.<br />
<br />
Multi-master replication allows read/write queries to be sent to<br />
multiple replicated computers. This capability also has a severe<br />
impact on performance due to the need to synchronize changes between<br />
servers. PGCluster is the most popular such solution freely available<br />
for PostgreSQL.<br />
<br />
There are also proprietary and hardware-based replication solutions<br />
available supporting a variety of replication models.<br />
<br />
=== Is possible to create a shared-storage postgresql server cluster? ===<br />
<br />
PostgreSQL does not support clustering using [[Shared_Storage|shared storage]] on a SAN, SCSI backplane,<br />
iSCSI volume, or other shared media. Such "RAC-style" clustering isn't supported.<br />
Only replication-based clustering is currently supported.<br />
<br />
See [[Replication, Clustering, and Connection Pooling]] information for details.<br />
<br />
[[Shared_Storage|Shared-storage]] 'failover' is possible, but it is not safe to have more than one<br />
postmaster running and accessing the data store at the same time. Heartbeat and<br />
[http://en.wikipedia.org/wiki/STONITH STONITH] or some other hard-disconnect option are recommended.<br />
<br />
=== Why are my table and column names not recognized in my query? Why is capitalization not preserved? ===<br />
<br />
The most common cause of unrecognized names is the use of<br />
double-quotes around table or column names during table creation. When<br />
double-quotes are used, table and column names (called identifiers)<br />
are stored case-sensitive, meaning you must use double-quotes when<br />
referencing the names in a query. Some interfaces, like pgAdmin,<br />
automatically double-quote identifiers during table creation. So, for<br />
identifiers to be recognized, you must either:<br />
<br />
* Avoid double-quoting identifiers when creating tables<br />
* Use only lowercase characters in identifiers<br />
* Double-quote identifiers when referencing them in queries<br />
<br />
=== I lost the database password. What can I do to recover it? ===<br />
<br />
You can't. However, you can reset it to something else. To do this, you<br />
<br />
* edit pg_hba.conf to allow ''trust'' authorization temporarily<br />
* Reload the config file (pg_ctl reload)<br />
* Connect and issue ALTER ROLE / PASSWORD to set the new password<br />
* edit pg_hba.conf again and restore the previous settings<br />
* Reload the config file again<br />
<br />
=== Does PostgreSQL have stored procedures? ===<br />
<br />
PostgreSQL doesn't. However PostgreSQL have very powerful functions and user-defined functions capabilities that can do most things that other RDBMS stored routines (procedures and functions) can do and in many cases more.<br />
<br />
These functions can be of different types and can be implemented in several programming languages.<br />
(Refer to documentation for more details. [http://www.postgresql.org/docs/current/static/xfunc.html User-Defined Functions])<br />
<br />
PostgreSQL functions can be invoked in many ways. If you want to invoke a function as you would call a stored procedure in other RDBMS (typically a function with side-effects but whose result you don't care for example because it returns void), one option would be to use [http://www.postgresql.org/docs/current/static/plpgsql.html PL/pgSQL Language] for your procedure and the [http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-NORESULT PERFORM] command. Example:<br />
PERFORM theNameOfTheFunction(arg1, arg2);<br />
<br />
Note that invoking instead:<br />
SELECT theNameOfTheFunction(arg1, arg2);<br />
would produce a result even if the function returns void (this result would be one row containing a void value).<br />
<br />
[http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-NORESULT PERFORM] could thus be used to discard this unuseful result.<br />
<br />
The main limitations on Pg's stored functions - as compared to true stored procedures - are:<br />
<br />
* inability to return multiple result sets<br />
* no support for autonomous transactions (<code>BEGIN</code>, <code>COMMIT</code> and <code>ROLLBACK</code> within a function)<br />
* no support for the SQL-standard <code>CALL</code> syntax, though the ODBC and JDBC drivers will translate calls for you.<br />
<br />
=== Why don't BEGIN, ROLLBACK and COMMIT work in stored procedures/functions? ===<br />
<br />
PostgreSQL doesn't support autonomous transactions in its stored functions. Like all PostgreSQL queries, stored functions always run in a transaction and cannot operate outside a transaction.<br />
<br />
If you need a stored procedure to manage transactions, you can look into the dblink interface or do the work from a client-side script instead. In some cases you can do what you need to using [http://www.postgresql.org/docs/current/static/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING exception blocks in PL/PgSQL], because each BEGIN/EXCEPTION/END block creates a subtransaction.<br />
<br />
=== Why is "SELECT count(*) FROM bigtable;" slow? ===<br />
<br />
It can't be answered directly from an index. PostgreSQL has to check the visibility for each record, so it<br />
forces a sequential scan of the entire table. If you want, you can keep track of the number of rows yourself with triggers, but beware that this will slow down write access to the table.<br />
<br />
You can get an estimation. The reltuples column in [http://www.postgresql.org/docs/current/static/catalog-pg-class.html pg_class] contains the information from the latest [http://www.postgresql.org/docs/current/static/sql-analyze.html ANALYZE] of the table. On a large table this is often accurate to within a few thousandths of a percent, which is accurate enough for many purposes.<br />
<br />
An "exact" count is often not exact for very long, anyway; due to [http://www.postgresql.org/docs/current/static/mvcc-intro.html MVCC] concurrency, the count will be accurate as of the moment the SELECT count(*) query (or, for stricter [http://www.postgresql.org/docs/current/static/transaction-iso.html transaction isolation] levels, its transaction) ''started'', and may well be out-of-date by the time the query completes. In a transaction mix where the table is being modified, two count(*) executions which return at the same moment might have different values, if a modifying transaction committed between their start times.<br />
<br />
For more information, see [[Slow Counting]].<br />
<br />
=== Why is my query much slower when run as a prepared query? ===<br />
<br />
When PostgreSQL has the full query with all parameters known by planning time, it can use statistics in the table to find out if the values used in the query are very common or very uncommon in a column. This lets it change the way it fetches the data to be more efficient, as it knows to expect lots or very few results from a certain part of the query. For example, it might choose an sequential scan instead of doing an index scan if you search for 'active=y' and it knows that 99% of the records in the table have 'active=y', because in this case a sequential scan will be much faster.<br />
<br />
In a prepared query, PostgreSQL doesn't have the value of all parameters when it's creating the plan. It has to try to pick a "safe" plan that should work fairly well no matter what value you supply as the parameter when you execute the prepared query. Unfortunately, this plan might not be very appropriate if the value you supply is vastly more common, or vastly less common, than is average for some randomly selected values in the table.<br />
<br />
If you suspect this issue is affecting you, start by using the [http://www.postgresql.org/docs/current/static/sql-explain.html EXPLAIN] command to compare the slow and fast queries. Look at the output of <code>EXPLAIN SELECT query...</code> and compare it to the result of <code>PREPARE query... ; EXPLAIN EXECUTE query...</code> to see if the plans are notably different. <code>EXPLAIN ANALYZE</code> may give you more information, such as row count estimates and counts.<br />
<br />
Usually people having this problem are trying to use prepared queries as a security measure to prevent SQL injection, rather than as a performance tuning option for expensive-to-plan queries frequently executed with a variety of different parameters. Those people should consider using client-side prepared statements if their client interface (eg PgJDBC) supports it.<br />
<br />
At present, PostgreSQL does not offer a way to request re-planning of a prepared statement using a particular set of parameter values; doing so somewhat defeats the purpose of server-side prepared statements. Running a statistics check to see if a particular parameter value is notably outside the norm and automatically re-planning in that case has been discussed, but not agreed upon or implemented as yet.<br />
<br />
See [[Using_EXPLAIN]]. If you're going to ask for help on the mailing lists, please read the [[Guide to reporting problems]].<br />
<br />
=== Why is my query much slower when run in a function than standalone? ===<br />
<br />
See [[FAQ#Why is my query much slower when run as a prepared query?]]. Queries in PL/PgSQL functions are prepared and cached, so they execute in much the same way as if you'd <code>PREPARE</code>d then <code>EXECUTE</code>d the query yourself.<br />
<br />
If you're having really severe issues with this that improving the table statistics or adjusting your query don't help with, you can work around it by forcing PL/PgSQL to re-prepare your query at every execution. To do this, use the <code>EXECUTE ... USING</code> statement in PL/PgSQL to supply your query as a textual string. Alternately, the [http://www.postgresql.org/docs/current/static/functions-string.html quote_literal or quote_nullable] functions may be used to escape parameters substituted into query text.<br />
<br />
=== Why do my strings sort incorrectly? ===<br />
<br />
First, make sure you are using the locale you want to be using. Use <code>SHOW lc_collate</code> to show the database-wide locale in effect. If you are using per-column collations, check those. If everything is how you want it, then read on.<br />
<br />
PostgreSQL uses the C library's locale facilities for sorting strings. So if the sort order of the strings is not what you expect, the issue is likely in the C library. You can verify the C library's idea of sorting using the <code>sort</code> utility on a text file, e.g.,<br />
<br />
LC_COLLATE=xx_YY.utf8 sort testfile.txt<br />
<br />
If this results in the same order that PostgreSQL gives you, then the problem is outside of PostgreSQL.<br />
<br />
PostgreSQL deviates from the libc behavior in so far as it breaks ties by sorting strings in byte order. This should rarely make a<br />
difference in practice, and is usually not the source of the problem when users complain about the sort order, but it could affect cases where, for example, combining and precombined Unicode characters are mixed.<br />
<br />
If the problem is in the C library, you will have to take it up with your operating system maintainers. Note, however, that while actual bugs in locale definitions of C libraries have been known to exist, it is more likely that the C library is correct, where "correct" means it follows some recognized international or national standard. Possibly, you are expecting one of multiple equally valid interpretations of a language's sorting rules.<br />
<br />
Common complaint patterns include:<br />
<br />
* Spaces and special characters: The sorting algorithm normally works in multiple passes. First, all the letters are compared, ignoring spaces and punctuation. Then, spaces and punctuation are compared to break ties. (This is a simplification of what actually happens.) It's not possible to change this without changing the locale definitions themselves (and even then it's difficult). You might want to restructure your data slightly to avoid this problem. For example, if you are sorting a name field, you could split the field into first and last name fields, avoiding the space in between.<br />
<br />
* Upper/lower case: Locales other than the C locale generally sort upper and lower case letters together. So the order will be something like a A b B c C ... instead of the A B C ... a b c ... that a sort based on ASCII byte values will give. That is correct.<br />
<br />
* German locale: sort order of ä as a or ae. Both of these are valid (see http://de.wikipedia.org/wiki/Alphabetische_Sortierung), but most C libraries only provide the first one. Fixing this would require creating a custom locale. This is possible, but will take some work.<br />
<br />
* It is not in ASCII/byte order. No, it's not, it's not supposed to be. ASCII is an encoding, not a sort order. If you want this, you can use the C locale, but then you use the ability to non-ASCII characters.<br />
<br />
That said, if you are on Mac OS X or a BSD-family operating system, and you are using UTF-8, then give up. The locale definitions on<br />
those operating systems are broken.<br />
<br />
[[Category:FAQ]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Pg_reorg&diff=18402Pg reorg2012-10-16T03:33:34Z<p>Schmiddy: trim out features, bugs and wishlist section of this page: we are using the github issue tracker and wiki now, and this page should just be a pointer</p>
<hr />
<div>{{DISPLAYTITLE:pg_reorg}}<br />
<br />
== Project Organization ==<br />
<br />
The pg_reorg developers are slowly moving the project from its old home on [http://pgfoundry.org/projects/reorg/ pgfoundry] to [https://github.com/reorg/pg_reorg github]. The current stable release of pg_reorg is [http://pgfoundry.org/frs/?group_id=1000411&release_id=1873 1.1.7], and the bleeding edge is [https://github.com/reorg/pg_reorg git master]. The project mailing list is still hosted on pgfoundry at [http://lists.pgfoundry.org/mailman/listinfo/reorg-general reorg-general]. Please report bugs on the [https://github.com/reorg/pg_reorg/issues github issue tracker]. <br />
<br />
Official documentation page: [http://reorg.projects.postgresql.org/pg_reorg.html pg_reorg], though documentation is being moved to the [https://github.com/reorg/pg_reorg/wiki github wiki]<br />
<br />
== Why use pg_reorg? ==<br />
pg_reorg is handy when you have a large table which has become bloated (see [[Show database bloat]] for a useful bloat-detection query). If you are able to hold an AccessExclusive lock on the table for an extended period, you have it easy: just use [http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER] or [[VACUUM FULL]]. However, if your table is busy being accessed by queries which can't wait hours while a CLUSTER/VACUUM FULL completes, you need a solution which will de-bloat your table and indexes while allowing concurrent reads and writes of the table. pg_reorg allows you to do precisely this. See also [http://www.depesz.com/2011/07/06/bloat-happens/ depesz's summary].</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Pg_reorg&diff=18397Pg reorg2012-10-15T02:18:10Z<p>Schmiddy: </p>
<hr />
<div>{{DISPLAYTITLE:pg_reorg}}<br />
<br />
== Project Organization ==<br />
<br />
The pg_reorg developers are slowly moving the project from its old home on [http://pgfoundry.org/projects/reorg/ pgfoundry] to [https://github.com/reorg/pg_reorg github]. The current stable release of pg_reorg is [http://pgfoundry.org/frs/?group_id=1000411&release_id=1873 1.1.7]. The project mailing list is still hosted on pgfoundry; please report bugs, issues and feature requests on the [http://lists.pgfoundry.org/mailman/listinfo/reorg-general reorg-general list].<br />
<br />
Official documentation page: [http://reorg.projects.postgresql.org/pg_reorg.html pg_reorg], though this wiki page may serve as an unofficial documentation point.<br />
<br />
== Why use pg_reorg? ==<br />
pg_reorg is handy when you have a large table which has become bloated (see [[Show database bloat]] for a useful bloat-detection query). If you are able to hold an AccessExclusive lock on the table for an extended period, you have it easy: just use [http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER] or [[VACUUM FULL]]. However, if your table is busy being accessed by queries which can't wait hours while a CLUSTER/VACUUM FULL completes, you need a solution which will de-bloat your table and indexes while allowing concurrent reads and writes of the table. pg_reorg allows you to do precisely this. See also [http://www.depesz.com/2011/07/06/bloat-happens/ depesz's summary].<br />
<br />
== Pending Features ==<br />
A few features have been added to cvs, git master or another git branch, and are not yet in a stable release. TODO: break this list out into items to be fixed in REL1_1, items committed to git master, and items in other git branches.<br />
# Bundle pg_reorg as an extension, so CREATE EXTENSION can be used<br />
# Make pg_reorg available on pgxn<br />
# Use ALTER TABLE ... ENABLE ALWAYS so pg_reorg can operate on a Slony slave node: [http://pgfoundry.org/pipermail/reorg-general/2012-October/000094.html]<br />
# Print status message while waiting on table lock: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000069.html]<br />
# Column name quoting: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000071.html]<br />
# Trigger to prevent TRUNCATE on target table.<br />
<br />
== Bugs, Known Issues ==<br />
# Problem using pg_reorg on a newly-promoted streaming replication slave: [http://pgfoundry.org/tracker/index.php?func=detail&aid=1011203&group_id=1000411&atid=1376| #1011203] [http://archives.postgresql.org/pgsql-bugs/2012-09/msg00278.php]<br />
# It is generally unsafe to perform DDL on your table while it is being reorg'ed, except for VACUUM or ANALYZE. Although the primary benefit of pg_reorg is that it does not hold a high-level lock on the target table while it is being reorg'ed, this also leaves us with no hard and fast way to prevent unsafe DDL. Perhaps we could add in some sanity checks of pg_class, pg_tablespace, etc. before and after the reorg finishes, and throw an error if any unexpected changes in the table attributes are detected.<br />
# pg_reorg chokes when an invalid index is left behind e.g. by CREATE INDEX CONCURRENTLY [http://pgfoundry.org/pipermail/reorg-general/2012-October/000101.html]<br />
<br />
== Wishlist ==<br />
# Concurrent index builds using multiple connections.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Pg_reorg&diff=18396Pg reorg2012-10-15T02:17:45Z<p>Schmiddy: Change title to "pg_reorg" using DISPLAYTITLE</p>
<hr />
<div>=== pg_reorg ===<br />
<br />
{{DISPLAYTITLE:pg_reorg}}<br />
<br />
== Project Organization ==<br />
<br />
The pg_reorg developers are slowly moving the project from its old home on [http://pgfoundry.org/projects/reorg/ pgfoundry] to [https://github.com/reorg/pg_reorg github]. The current stable release of pg_reorg is [http://pgfoundry.org/frs/?group_id=1000411&release_id=1873 1.1.7]. The project mailing list is still hosted on pgfoundry; please report bugs, issues and feature requests on the [http://lists.pgfoundry.org/mailman/listinfo/reorg-general reorg-general list].<br />
<br />
Official documentation page: [http://reorg.projects.postgresql.org/pg_reorg.html pg_reorg], though this wiki page may serve as an unofficial documentation point.<br />
<br />
== Why use pg_reorg? ==<br />
pg_reorg is handy when you have a large table which has become bloated (see [[Show database bloat]] for a useful bloat-detection query). If you are able to hold an AccessExclusive lock on the table for an extended period, you have it easy: just use [http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER] or [[VACUUM FULL]]. However, if your table is busy being accessed by queries which can't wait hours while a CLUSTER/VACUUM FULL completes, you need a solution which will de-bloat your table and indexes while allowing concurrent reads and writes of the table. pg_reorg allows you to do precisely this. See also [http://www.depesz.com/2011/07/06/bloat-happens/ depesz's summary].<br />
<br />
== Pending Features ==<br />
A few features have been added to cvs, git master or another git branch, and are not yet in a stable release. TODO: break this list out into items to be fixed in REL1_1, items committed to git master, and items in other git branches.<br />
# Bundle pg_reorg as an extension, so CREATE EXTENSION can be used<br />
# Make pg_reorg available on pgxn<br />
# Use ALTER TABLE ... ENABLE ALWAYS so pg_reorg can operate on a Slony slave node: [http://pgfoundry.org/pipermail/reorg-general/2012-October/000094.html]<br />
# Print status message while waiting on table lock: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000069.html]<br />
# Column name quoting: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000071.html]<br />
# Trigger to prevent TRUNCATE on target table.<br />
<br />
== Bugs, Known Issues ==<br />
# Problem using pg_reorg on a newly-promoted streaming replication slave: [http://pgfoundry.org/tracker/index.php?func=detail&aid=1011203&group_id=1000411&atid=1376| #1011203] [http://archives.postgresql.org/pgsql-bugs/2012-09/msg00278.php]<br />
# It is generally unsafe to perform DDL on your table while it is being reorg'ed, except for VACUUM or ANALYZE. Although the primary benefit of pg_reorg is that it does not hold a high-level lock on the target table while it is being reorg'ed, this also leaves us with no hard and fast way to prevent unsafe DDL. Perhaps we could add in some sanity checks of pg_class, pg_tablespace, etc. before and after the reorg finishes, and throw an error if any unexpected changes in the table attributes are detected.<br />
# pg_reorg chokes when an invalid index is left behind e.g. by CREATE INDEX CONCURRENTLY [http://pgfoundry.org/pipermail/reorg-general/2012-October/000101.html]<br />
<br />
== Wishlist ==<br />
# Concurrent index builds using multiple connections.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Pg_reorg&diff=18395Pg reorg2012-10-15T02:15:29Z<p>Schmiddy: /* Bugs, Known Issues */</p>
<hr />
<div>=== pg_reorg ===<br />
<br />
== Project Organization ==<br />
<br />
The pg_reorg developers are slowly moving the project from its old home on [http://pgfoundry.org/projects/reorg/ pgfoundry] to [https://github.com/reorg/pg_reorg github]. The current stable release of pg_reorg is [http://pgfoundry.org/frs/?group_id=1000411&release_id=1873 1.1.7]. The project mailing list is still hosted on pgfoundry; please report bugs, issues and feature requests on the [http://lists.pgfoundry.org/mailman/listinfo/reorg-general reorg-general list].<br />
<br />
Official documentation page: [http://reorg.projects.postgresql.org/pg_reorg.html pg_reorg], though this wiki page may serve as an unofficial documentation point.<br />
<br />
== Why use pg_reorg? ==<br />
pg_reorg is handy when you have a large table which has become bloated (see [[Show database bloat]] for a useful bloat-detection query). If you are able to hold an AccessExclusive lock on the table for an extended period, you have it easy: just use [http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER] or [[VACUUM FULL]]. However, if your table is busy being accessed by queries which can't wait hours while a CLUSTER/VACUUM FULL completes, you need a solution which will de-bloat your table and indexes while allowing concurrent reads and writes of the table. pg_reorg allows you to do precisely this. See also [http://www.depesz.com/2011/07/06/bloat-happens/ depesz's summary].<br />
<br />
== Pending Features ==<br />
A few features have been added to cvs, git master or another git branch, and are not yet in a stable release. TODO: break this list out into items to be fixed in REL1_1, items committed to git master, and items in other git branches.<br />
# Bundle pg_reorg as an extension, so CREATE EXTENSION can be used<br />
# Make pg_reorg available on pgxn<br />
# Use ALTER TABLE ... ENABLE ALWAYS so pg_reorg can operate on a Slony slave node: [http://pgfoundry.org/pipermail/reorg-general/2012-October/000094.html]<br />
# Print status message while waiting on table lock: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000069.html]<br />
# Column name quoting: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000071.html]<br />
# Trigger to prevent TRUNCATE on target table.<br />
<br />
== Bugs, Known Issues ==<br />
# Problem using pg_reorg on a newly-promoted streaming replication slave: [http://pgfoundry.org/tracker/index.php?func=detail&aid=1011203&group_id=1000411&atid=1376| #1011203] [http://archives.postgresql.org/pgsql-bugs/2012-09/msg00278.php]<br />
# It is generally unsafe to perform DDL on your table while it is being reorg'ed, except for VACUUM or ANALYZE. Although the primary benefit of pg_reorg is that it does not hold a high-level lock on the target table while it is being reorg'ed, this also leaves us with no hard and fast way to prevent unsafe DDL. Perhaps we could add in some sanity checks of pg_class, pg_tablespace, etc. before and after the reorg finishes, and throw an error if any unexpected changes in the table attributes are detected.<br />
# pg_reorg chokes when an invalid index is left behind e.g. by CREATE INDEX CONCURRENTLY [http://pgfoundry.org/pipermail/reorg-general/2012-October/000101.html]<br />
<br />
== Wishlist ==<br />
# Concurrent index builds using multiple connections.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Pg_reorg&diff=18394Pg reorg2012-10-15T01:14:26Z<p>Schmiddy: </p>
<hr />
<div>=== pg_reorg ===<br />
<br />
== Project Organization ==<br />
<br />
The pg_reorg developers are slowly moving the project from its old home on [http://pgfoundry.org/projects/reorg/ pgfoundry] to [https://github.com/reorg/pg_reorg github]. The current stable release of pg_reorg is [http://pgfoundry.org/frs/?group_id=1000411&release_id=1873 1.1.7]. The project mailing list is still hosted on pgfoundry; please report bugs, issues and feature requests on the [http://lists.pgfoundry.org/mailman/listinfo/reorg-general reorg-general list].<br />
<br />
Official documentation page: [http://reorg.projects.postgresql.org/pg_reorg.html pg_reorg], though this wiki page may serve as an unofficial documentation point.<br />
<br />
== Why use pg_reorg? ==<br />
pg_reorg is handy when you have a large table which has become bloated (see [[Show database bloat]] for a useful bloat-detection query). If you are able to hold an AccessExclusive lock on the table for an extended period, you have it easy: just use [http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER] or [[VACUUM FULL]]. However, if your table is busy being accessed by queries which can't wait hours while a CLUSTER/VACUUM FULL completes, you need a solution which will de-bloat your table and indexes while allowing concurrent reads and writes of the table. pg_reorg allows you to do precisely this. See also [http://www.depesz.com/2011/07/06/bloat-happens/ depesz's summary].<br />
<br />
== Pending Features ==<br />
A few features have been added to cvs, git master or another git branch, and are not yet in a stable release. TODO: break this list out into items to be fixed in REL1_1, items committed to git master, and items in other git branches.<br />
# Bundle pg_reorg as an extension, so CREATE EXTENSION can be used<br />
# Make pg_reorg available on pgxn<br />
# Use ALTER TABLE ... ENABLE ALWAYS so pg_reorg can operate on a Slony slave node: [http://pgfoundry.org/pipermail/reorg-general/2012-October/000094.html]<br />
# Print status message while waiting on table lock: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000069.html]<br />
# Column name quoting: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000071.html]<br />
# Trigger to prevent TRUNCATE on target table.<br />
<br />
== Bugs, Known Issues ==<br />
# Problem using pg_reorg on a newly-promoted streaming replication slave: [http://pgfoundry.org/tracker/index.php?func=detail&aid=1011203&group_id=1000411&atid=1376| #1011203] [http://archives.postgresql.org/pgsql-bugs/2012-09/msg00278.php]<br />
# It is generally unsafe to perform DDL on your table while it is being reorg'ed, except for VACUUM or ANALYZE. Although the primary benefit of pg_reorg is that it does not hold a high-level lock on the target table while it is being reorg'ed, this also leaves us with no hard and fast way to prevent unsafe DDL. Perhaps we could add in some sanity checks of pg_class, pg_tablespace, etc. before and after the reorg finishes, and throw an error if any unexpected changes in the table attributes are detected.<br />
# pg_reorg chokes when an invalid index is left behind e.g. by CREATE INDEX CONCURRENTLY<br />
<br />
== Wishlist ==<br />
# Concurrent index builds using multiple connections.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Pg_reorg&diff=18393Pg reorg2012-10-14T22:55:08Z<p>Schmiddy: Start a wiki page for pg_reorg project</p>
<hr />
<div>=== pg_reorg ===<br />
<br />
== Project Organization ==<br />
<br />
The pg_reorg developers are slowly moving the project from its old home on [http://pgfoundry.org/projects/reorg/ pgfoundry] to [https://github.com/reorg/pg_reorg github]. The current stable release of pg_reorg is [http://pgfoundry.org/frs/?group_id=1000411&release_id=1873 1.1.7]. The project mailing list is still hosted on pgfoundry; please report bugs, issues and feature requests on the [http://lists.pgfoundry.org/mailman/listinfo/reorg-general reorg-general list].<br />
<br />
Official documentation page: [http://reorg.projects.postgresql.org/pg_reorg.html pg_reorg], though this wiki page may serve as an unofficial documentation point.<br />
<br />
== Why use pg_reorg? ==<br />
pg_reorg is handy when you have a large table which has become bloated (see [[Show database bloat]] for a useful bloat-detection query). If you are able to hold an AccessExclusive lock on the table for an extended period, you have it easy: just use [http://www.postgresql.org/docs/current/static/sql-cluster.html CLUSTER] or [[VACUUM FULL]]. However, if your table is busy being accessed by queries which can't wait hours while a CLUSTER/VACUUM FULL completes, you need a solution which will de-bloat your table and indexes while allowing concurrent reads and writes of the table. pg_reorg allows you to do precisely this. See also [http://www.depesz.com/2011/07/06/bloat-happens/ depesz's summary].<br />
<br />
== Pending Features ==<br />
A few features have been added to cvs, git master or another git branch, and are not yet in a stable release. TODO: break this list out into items to be fixed in REL1_1, items committed to git master, and items in other git branches.<br />
# Bundle pg_reorg as an extension, so CREATE EXTENSION can be used<br />
# Make pg_reorg available on pgxn<br />
# Use ALTER TABLE ... ENABLE ALWAYS so pg_reorg can operate on a Slony slave node: [http://pgfoundry.org/pipermail/reorg-general/2012-October/000094.html]<br />
# Print status message while waiting on table lock: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000069.html]<br />
# Column name quoting: [http://pgfoundry.org/pipermail/reorg-general/2012-September/000071.html]<br />
# Trigger to prevent TRUNCATE on target table.<br />
<br />
== Bugs, Known Issues ==<br />
# Problem using pg_reorg on a newly-promoted streaming replication slave: [http://pgfoundry.org/tracker/index.php?func=detail&aid=1011203&group_id=1000411&atid=1376| #1011203] [http://archives.postgresql.org/pgsql-bugs/2012-09/msg00278.php]<br />
# It is generally unsafe to perform DDL on your table while it is being reorg'ed, except for VACUUM or ANALYZE. Although the primary benefit of pg_reorg is that it does not hold a high-level lock on the target table while it is being reorg'ed, this also leaves us with no hard and fast way to prevent unsafe DDL. Perhaps we could add in some sanity checks of pg_class, pg_tablespace, etc. before and after the reorg finishes, and throw an error if any unexpected changes in the table attributes are detected.<br />
<br />
== Wishlist ==<br />
# Concurrent index builds using multiple connections.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=What%27s_new_in_PostgreSQL_9.2&diff=18114What's new in PostgreSQL 9.22012-08-29T19:24:36Z<p>Schmiddy: /* Replication improvements */ grammar & spelling</p>
<hr />
<div>{{Languages}}<br />
<br />
This document showcases many of the latest developments in PostgreSQL 9.2, compared to the last major release &ndash; PostgreSQL 9.1. There are many improvements in this release, so this wiki page covers many of the more important changes in detail. The full list of changes is itemised in ''Release Notes''.<br />
<br />
<br />
=Major new features=<br />
<br />
==Index-only scans <!-- Robert Haas, Ibrar Ahmed, Heikki Linnakangas, Tom Lane -->==<br />
<br />
In PostgreSQL, indexes have no "visibility" information. It means that when you access a record by its index, PostgreSQL has to visit the real tuple in the table to be sure it is visible to you: the tuple the index points to may simply be an old version of the record you are looking for.<br />
<br />
It can be a very big performance problem: the index is mostly ordered, so accessing its records is quite efficient, while the records may be scattered all over the place (that's a reason why PostgreSQL has a cluster command, but that's another story). In 9.2, PostgreSQL will use an "Index Only Scan" when possible, and not access the record itself if it doesn't need to.<br />
<br />
There is still no visibility information in the index. So in order to do this, PostgreSQL uses the visibility map ([http://www.postgresql.org/docs/devel/static/storage-vm.html visibility map]) , which tells it whether the whole content of a (usually) 8K page is visible to all transactions or not. When the index record points to a tuple contained in an «all visible» page, PostgreSQL won't have to access the tuple, it will be able to build it directly from the index. Of course, all the columns requested by the query must be in the index.<br />
<br />
The visibility map is maintained by VACUUM (it sets the visible bit), and by the backends doing SQL work (they unset the visible bit).<br />
<br />
If the data has been read only since the last VACUUM then the data is All Visible and the index only scan feature can improve performance.<br />
<br />
Here is an example.<br />
<br />
CREATE TABLE demo_ios (col1 float, col2 float, col3 text);<br />
<br />
In this table, we'll put random data, in order to have "scattered" data. We'll put 100 million records, to have a big recordset, and have it not fit in memory (that's a 4GB-ram machine). This is an ideal case, made for this demo. The gains won't be that big in real life.<br />
<br />
INSERT INTO demo_ios SELECT generate_series(1,100000000),random(), 'mynotsolongstring';<br />
<br />
SELECT pg_size_pretty(pg_total_relation_size('demo_ios'));<br />
pg_size_pretty <br />
----------------<br />
6512 MB<br />
<br />
Let's pretend that the query is this:<br />
<br />
SELECT col1,col2 FROM demo_ios where col2 BETWEEN 0.01 AND 0.02<br />
<br />
In order to use an index only scan on this query, we need an index on col2,col1 (col2 first, as it is used in the WHERE clause).<br />
<br />
CREATE index idx_demo_ios on demo_ios(col2,col1);<br />
<br />
We vacuum the visibility map to be up-to-date:<br />
<br />
VACUUM demo_ios;<br />
<br />
All the timing you'll see below are done on a cold OS and PostgreSQL cache (that's where the gains are, as the purpose on Index Only Scans is to reduce I/O).<br />
<br />
Let's first try without Index Only Scans:<br />
<br />
SET enable_indexonlyscan to off;<br />
<br />
EXPLAIN (analyze,buffers) select col1,col2 FROM demo_ios where col2 between 0.01 and 0.02;<br />
QUERY PLAN <br />
----------------------------------------------------------------------------------------------------------------------------------------<br />
Bitmap Heap Scan on demo_ios (cost=25643.01..916484.44 rows=993633 width=16) (actual time=763.391..362963.899 rows=1000392 loops=1)<br />
Recheck Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))<br />
Rows Removed by Index Recheck: 68098621<br />
Buffers: shared hit=2 read=587779<br />
-> Bitmap Index Scan on idx_demo_ios (cost=0.00..25394.60 rows=993633 width=0) (actual time=759.011..759.011 rows=1000392 loops=1)<br />
Index Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))<br />
Buffers: shared hit=2 read=3835<br />
Total runtime: 364390.127 ms<br />
<br />
<br />
With Index Only Scans:<br />
<br />
explain (analyze,buffers) select col1,col2 from demo_ios where col2 between 0.01 and 0.02;<br />
QUERY PLAN <br />
-----------------------------------------------------------------------------------------------------------------------------------------------<br />
Index Only Scan using idx_demo_ios on demo_ios (cost=0.00..35330.93 rows=993633 width=16) (actual time=58.100..3250.589 rows=1000392 loops=1)<br />
Index Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))<br />
Heap Fetches: 0<br />
Buffers: shared hit=923073 read=3848<br />
Total runtime: 4297.405 ms<br />
<br />
<br />
<br />
As nothing is free, there are a few things to keep in mind:<br />
<br />
* Adding indexes for index only scans obviously adds indexes to your table. So updates will be slower.<br />
* You will index columns that weren't indexed before. So there will be less opportunities for HOT updates.<br />
* Gains will probably be smaller in real life situations, especially when data is changed between VACUUMs<br />
<br />
This required making visibility map changes crash-safe, so visibility map bit changes are now WAL-logged.<br />
<br />
==Replication improvements <!-- Fujii Masao, Simon Riggs, Magnus Hagander, Jun Ishizuka -->==<br />
<br />
Streaming Replication becomes more polished with this release. <br />
<br />
One of the main remaining gripes about streaming replication is that all the slaves have to be connected to the same and unique master, consuming its resources. Moreover, in case of a failover, it could be complicated to reconnect all the remaining slaves to the newly promoted master, if one is not using a tool like repmgr. <br />
<br />
* With 9.2, a standby can also send replication changes, allowing cascading replication.<br />
<br />
Let's build this. We start with an already working 9.2 database.<br />
<br />
We set it up for replication:<br />
<br />
postgresql.conf:<br />
wal_level=hot_standby #(could be archive too)<br />
max_wal_senders=5<br />
hot_standby=on<br />
<br />
You'll probably also want to activate archiving in production, it won't be done here.<br />
<br />
pg_hba.conf (do not use trust in production):<br />
host replication replication_user 0.0.0.0/0 md5<br />
<br />
Create the user:<br />
create user replication_user replication password 'secret';<br />
<br />
Clone the database:<br />
<br />
pg_basebackup -h localhost -U replication_user -D data2<br />
Password:<br />
<br />
We have a brand new cluster in the data2 directory. We'll change the port so that it can start (postgresql.conf):<br />
port=5433<br />
<br />
We add a recovery.conf to tell it how to stream from the master database:<br />
standby_mode = on<br />
primary_conninfo = 'host=localhost port=5432 user=replication_user password=secret' <br />
<br />
pg_ctl -D data2 start<br />
server starting<br />
LOG: database system was interrupted; last known up at 2012-07-03 17:58:09 CEST<br />
LOG: creating missing WAL directory "pg_xlog/archive_status"<br />
LOG: entering standby mode<br />
LOG: streaming replication successfully connected to primary<br />
LOG: redo starts at 0/9D000020<br />
LOG: consistent recovery state reached at 0/9D0000B8<br />
LOG: database system is ready to accept read only connections<br />
<br />
Now, let's add a second slave, which will use this slave:<br />
<br />
<br />
pg_basebackup -h localhost -U replication_user -D data3 -p 5433<br />
Password: <br />
<br />
We edit data3's postgresql.conf to change the port:<br />
port=5434<br />
<br />
We modify the recovery.conf to stream from the slave:<br />
standby_mode = on<br />
primary_conninfo = 'host=localhost port=5433 user=replication_user password=secret' # e.g. 'host=localhost port=5432'<br />
<br />
We start the cluster:<br />
pg_ctl -D data3 start<br />
server starting<br />
LOG: database system was interrupted while in recovery at log time 2012-07-03 17:58:09 CEST<br />
HINT: If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.<br />
LOG: creating missing WAL directory "pg_xlog/archive_status"<br />
LOG: entering standby mode<br />
LOG: streaming replication successfully connected to primary<br />
LOG: redo starts at 0/9D000020<br />
LOG: consistent recovery state reached at 0/9E000000<br />
LOG: database system is ready to accept read only connections<br />
<br />
Now, everything modified on the master cluster get streamed to the first slave, and from there to the second slave. This second replication has to be monitored from the first slave (the master knows nothing about it).<br />
<br />
<br />
* As you may have noticed from the example, pg_basebackup now works from slaves.<br />
<br />
* There is another use case that wasn't covered: what if a user didn't care for having a full fledged slave, and only wanted to stream the WAL files to another location, to benefit from the reduced data loss without the burden of maintaining a slave ?<br />
<br />
pg_receivexlog is provided just for this purpose: it pretends to be a PostgreSQL slave, but only stores the log files as they are streamed, in a directory:<br />
pg_receivexlog -D /tmp/new_logs -h localhost -U replication_user<br />
<br />
will connect to the master (or a slave), and start creating files: <br />
ls /tmp/new_logs/<br />
00000001000000000000009E.partial<br />
<br />
Files are of the segment size, so they can be used for a normal recovery of the database. It's the same as an archive command, but with a much smaller granularity.<br />
Remember to rename the last segment to remove the .partial suffix before using it with PITR or other.<br />
<br />
* synchronous_commit has a new value: remote_write. It can be used when there is a synchronous slave (synchronous_standby_names is set), meaning that the master doesn't have to wait for the slave to have written the data to disk, only for the slave to have acknowledged the data. With this set, data is protected from a crash on the master, but could still be lost if the slave crashed at the same time (i.e. before having written the in flight data to disk). As this is a quite remote possibility, some people will be interested in this compromise.<br />
<br />
==JSON datatype==<br />
<br />
The JSON datatype is meant for storing JSON-structured data. It will validate that the input JSON string is correct JSON:<br />
<br />
=# SELECT '{"username":"john","posts":121,"emailaddress":"john@nowhere.com"}'::json;<br />
json <br />
-------------------------------------------------------------------<br />
{"username":"john","posts":121,"emailaddress":"john@nowhere.com"}<br />
(1 row)<br />
<br />
=# SELECT '{"username","posts":121,"emailaddress":"john@nowhere.com"}'::json;<br />
ERROR: invalid input syntax for type json at character 8<br />
DETAIL: Expected ":", but found ",".<br />
CONTEXT: JSON data, line 1: {"username",...<br />
STATEMENT: SELECT '{"username","posts":121,"emailaddress":"john@nowhere.com"}'::json;<br />
ERROR: invalid input syntax for type json<br />
LINE 1: SELECT '{"username","posts":121,"emailaddress":"john@nowhere...<br />
^<br />
DETAIL: Expected ":", but found ",".<br />
CONTEXT: JSON data, line 1: {"username",...<br />
<br />
You can also convert a row type to JSON:<br />
<br />
=#SELECT * FROM demo ;<br />
username | posts | emailaddress <br />
----------+-------+---------------------<br />
john | 121 | john@nowhere.com<br />
mickael | 215 | mickael@nowhere.com<br />
(2 rows)<br />
<br />
=# SELECT row_to_json(demo) FROM demo;<br />
row_to_json <br />
-------------------------------------------------------------------------<br />
{"username":"john","posts":121,"emailaddress":"john@nowhere.com"}<br />
{"username":"mickael","posts":215,"emailaddress":"mickael@nowhere.com"}<br />
(2 rows)<br />
<br />
Or an array type:<br />
<br />
<br />
=# select array_to_json(array_agg(demo)) from demo;<br />
array_to_json <br />
---------------------------------------------------------------------------------------------------------------------------------------------<br />
[{"username":"john","posts":121,"emailaddress":"john@nowhere.com"},{"username":"mickael","posts":215,"emailaddress":"mickael@nowhere.com"}]<br />
(1 row)<br />
<br />
== Range Types ==<br />
<br />
Range types are used to store a range of data of a given type. There are a few pre-defined types. They are integer (int4range), bigint (int8range), numeric (numrange), timestamp without timezone (tsrange), timestamp with timezone (tstzrange), and date (daterange).<br />
<br />
Ranges can be made of continuous (numeric, timestamp...) or discrete (integer, date...) data types. They can be open (the bound isn't part of the range) or closed (the bound is part of the range). A bound can also be infinite.<br />
<br />
Without these datatypes, most people solve the range problems by using two columns in a table. These range types are much more powerful, as you can use many operators on them:<br />
<br />
Here is the intersection between then 1000(open)-2000(closed) and 1000(closed)-1200(closed) numeric range:<br />
<br />
SELECT '(1000,2000]'::numrange * '[1000,1200]'::numrange;<br />
?column? <br />
-------------<br />
(1000,1200]<br />
(1 row)<br />
<br />
So you can query on things like: «give me all ranges that intersect this»:<br />
<br />
=# SELECT * from test_range ;<br />
period <br />
-----------------------------------------------------<br />
["2012-01-01 00:00:00+01","2012-01-02 12:00:00+01"]<br />
["2012-01-01 00:00:00+01","2012-03-01 00:00:00+01"]<br />
["2008-01-01 00:00:00+01","2015-01-01 00:00:00+01"]<br />
(3 rows)<br />
<br />
<br />
=# SELECT * FROM test_range WHERE period && '[2012-01-03 00:00:00,2012-01-03 12:00:00]'; <br />
period <br />
-----------------------------------------------------<br />
["2012-01-01 00:00:00+01","2012-03-01 00:00:00+01"]<br />
["2008-01-01 00:00:00+01","2015-01-01 00:00:00+01"]<br />
(2 rows)<br />
<br />
This query could use an index defined like this:<br />
<br />
=# CREATE INDEX idx_test_range on test_range USING gist (period);<br />
<br />
<br />
<br />
You can also use these range data types to define exclusion constraints:<br />
<br />
CREATE EXTENSION btree_gist ;<br />
CREATE TABLE reservation (room_id int, period tstzrange);<br />
ALTER TABLE reservation ADD EXCLUDE USING GIST (room_id WITH =, period WITH &&);<br />
<br />
This means that now it is forbidden to have two records in this table where room_id is equal and period overlaps. The extension btree_gist is required to create a GiST index on room_id (it's an integer, it is usually indexed with a btree index).<br />
<br />
<br />
=# INSERT INTO reservation VALUES (1,'(2012-08-23 14:00:00,2012-08-23 15:00:00)');<br />
INSERT 0 1<br />
=# INSERT INTO reservation VALUES (2,'(2012-08-23 14:00:00,2012-08-23 15:00:00)');<br />
INSERT 0 1<br />
=# INSERT INTO reservation VALUES (1,'(2012-08-23 14:45:00,2012-08-23 15:15:00)');<br />
ERROR: conflicting key value violates exclusion constraint "reservation_room_id_period_excl"<br />
DETAIL: Key (room_id, period)=(1, ("2012-08-23 14:45:00+02","2012-08-23 15:15:00+02")) <br />
conflicts with existing key (room_id, period)=(1, ("2012-08-23 14:00:00+02","2012-08-23 15:00:00+02")).<br />
STATEMENT: INSERT INTO reservation VALUES (1,'(2012-08-23 14:45:00,2012-08-23 15:15:00)');<br />
<br />
One can also declare new range types.<br />
<br />
=Performance improvements=<br />
<br />
This version has performance improvements on a very large range of domains (non-exaustive):<br />
<br />
* The most visible will probably be the Index Only Scans, which has already been introduced in this document.<br />
<br />
* The lock contention of several big locks has been significantly reduced, leading to better multi-processor scalability, for machines with over 32 cores mostly. <!-- Robert Haas --><br />
<br />
* The performance of in-memory sorts has been improved by up to 25% in some situations, with certain specialized sort functions introduced. <!-- Peter Geoghegan --><br />
<br />
* An idle PostgreSQL server now makes less wakeups, leading to lower power consumption<!--Peter Geoghegan-->. This is especially useful on virtualized and embedded environments.<br />
<br />
* COPY has been improved, it will generate less WAL volume and fewer locks of a table's pages. <!-- Heikki Linnakangas --><br />
<br />
* Statistics are collected on array contents<!-- Alexander Korotkov -->, allowing for better estimations of selectivity on array operations.<br />
<br />
* Text-to-anytype concatenation and quote_literal/quote_nullable functions are not volatile any more, enabling better optimization in some cases <!-- Marti Raudsepp --><br />
<br />
* The system can now track IO durations <!--Ants Aasma --><br />
<br />
This one deserves a little explanation, as it can be a little tricky. Tracking IO durations means asking repeatedly the time to the operating system. Depending on the operating system and the hardware, this can be quite cheap, or extremely costly. The most import factor here is where the system gets its time from. It could be directly retrieved from the processor (TSC), dedicated hardware such as HPET, or an ACPI call. What's most important is that the cost of getting time can vary from a factor of thousands.<br />
<br />
If you are interested in this timing data, it's better to first check if your system will support it without to much of a performance hit. PostgreSQL provides you with the pg_test_timing tool:<br />
<br />
<pre><br />
$ pg_test_timing <br />
Testing timing overhead for 3 seconds.<br />
Per loop time including overhead: 28.02 nsec<br />
Histogram of timing durations:<br />
< usec: count percent<br />
32: 41 0.00004%<br />
16: 1405 0.00131%<br />
8: 200 0.00019%<br />
4: 388 0.00036%<br />
2: 2982558 2.78523%<br />
1: 104100166 97.21287%<br />
</pre><br />
<br />
Here, everything is good: getting time costs around 28 nanoseconds, and has a very small variation. Anything under 100 nanoseconds should be good for production. If you get higher values, you may still find a way to tune your system. You'd better check on the [http://www.postgresql.org/docs/9.2/static/pgtesttiming.html documentation].<br />
<br />
Anyway, here is the data you'll be able to collect if your system is ready for this:<br />
<br />
First, you'll get per-database statistics, which will now give accurate informations about which database is doing most I/O:<br />
<br />
<pre><br />
=# SELECT * FROM pg_stat_database WHERE datname = 'mydb';<br />
-[ RECORD 1 ]--+------------------------------<br />
datid | 16384<br />
datname | mydb<br />
numbackends | 1<br />
xact_commit | 270<br />
xact_rollback | 2<br />
blks_read | 1961<br />
blks_hit | 17944<br />
tup_returned | 269035<br />
tup_fetched | 8850<br />
tup_inserted | 16<br />
tup_updated | 4<br />
tup_deleted | 45<br />
conflicts | 0<br />
temp_files | 0<br />
temp_bytes | 0<br />
deadlocks | 0<br />
blk_read_time | 583.774<br />
blk_write_time | 0<br />
stats_reset | 2012-07-03 17:18:54.796817+02<br />
</pre><br />
We see here that mydb has only consumed 583.774 milliseconds of read time.<br />
<br />
Explain will benefit from this too:<br />
<pre><br />
=# EXPLAIN (analyze,buffers) SELECT count(*) FROM mots ;<br />
QUERY PLAN<br />
----------------------------------------------------------------------------------------------------------------<br />
Aggregate (cost=1669.95..1669.96 rows=1 width=0) (actual time=21.943..21.943 rows=1 loops=1)<br />
Buffers: shared read=493<br />
I/O Timings: read=2.578<br />
-> Seq Scan on mots (cost=0.00..1434.56 rows=94156 width=0) (actual time=0.059..12.933 rows=94156 loops=1)<br />
Buffers: shared read=493<br />
I/O Timings: read=2.578<br />
Total runtime: 22.059 ms<br />
</pre><br />
We now have a separate information about the time taken to retrieve data from the operating system. Obviously, here, the data was in the operating system's cache (2 milliseconds to read 493 blocks).<br />
<br />
And last, if you have enabled pg_stat_statements:<br />
<pre><br />
select * from pg_stat_statements where query ~ 'words';<br />
-[ RECORD 1 ]-------+---------------------------<br />
userid | 10<br />
dbid | 16384<br />
query | select count(*) from words;<br />
calls | 2<br />
total_time | 78.332<br />
rows | 2<br />
shared_blks_hit | 0<br />
shared_blks_read | 986<br />
shared_blks_dirtied | 0<br />
shared_blks_written | 0<br />
local_blks_hit | 0<br />
local_blks_read | 0<br />
local_blks_dirtied | 0<br />
local_blks_written | 0<br />
temp_blks_read | 0<br />
temp_blks_written | 0<br />
blk_read_time | 58.427<br />
blk_write_time | 0<br />
</pre><br />
<br />
* As for every version, the optimizer has received its share of improvements <!-- Tom Lane--><br />
** Prepared statements used to be optimized once, without any knowledge of the parameters' values. With 9.2, the planner will use specific plans regarding to the parameters sent (the query will be planned at execution), except if the query is executed several times and the planner decides that the generic plan is not too much more expensive than the specific plans.<br />
** A new feature has been added: parameterized paths. Simply put, it means that a sub-part of a query plan can use parameters it has got from a parent node. It fixes several bad plans that could occur, especially when the optimizer couldn't reorder joins to put nested loops where it would have been efficient.<br />
<br />
This example is straight from the developpers mailing lists <!-- Andres Freund -->:<br />
<br />
<pre><br />
CREATE TABLE a (<br />
a_id serial PRIMARY KEY NOT NULL,<br />
b_id integer<br />
);<br />
CREATE INDEX a__b_id ON a USING btree (b_id);<br />
<br />
<br />
CREATE TABLE b (<br />
b_id serial NOT NULL,<br />
c_id integer<br />
);<br />
CREATE INDEX b__c_id ON b USING btree (c_id);<br />
<br />
<br />
CREATE TABLE c (<br />
c_id serial PRIMARY KEY NOT NULL,<br />
value integer UNIQUE<br />
);<br />
<br />
INSERT INTO b (b_id, c_id)<br />
SELECT g.i, g.i FROM generate_series(1, 50000) g(i);<br />
<br />
INSERT INTO a(b_id)<br />
SELECT g.i FROM generate_series(1, 50000) g(i);<br />
<br />
INSERT INTO c(c_id,value)<br />
VALUES (1,1);<br />
</pre><br />
<br />
So we have a referencing b, b referencing c.<br />
<br />
Here is an example of a query working badly with PostgreSQL 9.1:<br />
<br />
<pre><br />
EXPLAIN ANALYZE SELECT 1<br />
FROM<br />
c<br />
WHERE<br />
EXISTS (<br />
SELECT *<br />
FROM a<br />
JOIN b USING (b_id)<br />
WHERE b.c_id = c.c_id)<br />
AND c.value = 1;<br />
QUERY PLAN<br />
-----------------------------------------------------------------------------------------------------------------------<br />
Nested Loop Semi Join (cost=1347.00..3702.27 rows=1 width=0) (actual time=13.799..13.802 rows=1 loops=1)<br />
Join Filter: (c.c_id = b.c_id)<br />
-> Index Scan using c_value_key on c (cost=0.00..8.27 rows=1 width=4) (actual time=0.006..0.008 rows=1 loops=1)<br />
Index Cond: (value = 1)<br />
-> Hash Join (cost=1347.00..3069.00 rows=50000 width=4) (actual time=13.788..13.788 rows=1 loops=1)<br />
Hash Cond: (a.b_id = b.b_id)<br />
-> Seq Scan on a (cost=0.00..722.00 rows=50000 width=4) (actual time=0.007..0.007 rows=1 loops=1)<br />
-> Hash (cost=722.00..722.00 rows=50000 width=8) (actual time=13.760..13.760 rows=50000 loops=1)<br />
Buckets: 8192 Batches: 1 Memory Usage: 1954kB<br />
-> Seq Scan on b (cost=0.00..722.00 rows=50000 width=8) (actual time=0.008..5.702 rows=50000 loops=1)<br />
Total runtime: 13.842 ms<br />
</pre><br />
<br />
Not that bad, 13 milliseconds. Still, we are doing sequential scans on a and b, when our common sense tells us that c.value=1 should be used to filter rows more aggressively.<br />
<br />
Here's what 9.2 does with this query:<br />
<br />
<pre><br />
QUERY PLAN<br />
----------------------------------------------------------------------------------------------------------------------------<br />
Nested Loop Semi Join (cost=0.00..16.97 rows=1 width=0) (actual time=0.035..0.037 rows=1 loops=1)<br />
-> Index Scan using c_value_key on c (cost=0.00..8.27 rows=1 width=4) (actual time=0.007..0.009 rows=1 loops=1)<br />
Index Cond: (value = 1)<br />
-> Nested Loop (cost=0.00..8.69 rows=1 width=4) (actual time=0.025..0.025 rows=1 loops=1)<br />
-> Index Scan using b__c_id on b (cost=0.00..8.33 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=1)<br />
Index Cond: (c_id = c.c_id)<br />
-> Index Only Scan using a__b_id on a (cost=0.00..0.35 rows=1 width=4) (actual time=0.014..0.014 rows=1 loops=1)<br />
Index Cond: (b_id = b.b_id)<br />
Total runtime: 0.089 ms<br />
</pre><br />
<br />
The «parameterized path» is:<br />
<pre><br />
-> Nested Loop (cost=0.00..8.69 rows=1 width=4) (actual time=0.025..0.025 rows=1 loops=1)<br />
-> Index Scan using b__c_id on b (cost=0.00..8.33 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=1)<br />
Index Cond: (c_id = c.c_id)<br />
-> Index Only Scan using a__b_id on a (cost=0.00..0.35 rows=1 width=4) (actual time=0.014..0.014 rows=1 loops=1)<br />
Index Cond: (b_id = b.b_id)<br />
Total runtime: 0.089 ms<br />
</pre><br />
<br />
This part of the plan depends on a parent node (c_id=c.c_id). This part of the plan is called each time with a different parameter coming from the parent node.<br />
<br />
This plan is of course much faster, as there is no need to fully scan a, and to fully scan AND hash b.<br />
<br />
=SP-GiST=<br />
<br />
SP-GiST stands for Space Partitionned GiST, GiST being Generalized Search Tree. GiST is an index type, and has been available for quite a while in PostgreSQL. GiST is already very efficient at indexing complex data types, but performance tends to suffer when the source data isn't uniformly distributed. SP-GiST tries to fix that.<br />
<br />
As all indexing methods available in PostgreSQL, SP-GiST is a generic indexing method, meaning its purpose is to index whatever you'll throw at it, using operators you'll provide. It means that if you want to create a new datatype, and make it indexable through SP-GiST, you'll have to follow the documented API.<br />
<br />
SP-GiST can be used to implement 3 type of indexes: trie (suffix) indexing, Quadtree (data is divided into quadrants), and k-d tree (k-dimensional tree).<br />
<br />
For now, SP-GiST is provided with operator families called "quad_point_ops", "kd_point_ops" and "text_ops".<br />
<br />
As their names indicate, the first one indexes point types, using a quadtree, the second one indexes point types using a k-d tree, and the third one indexes text, using suffix.<br />
<br />
=pg_stat_statements=<br />
<br />
This contrib module has received a lot of improvements in this version:<br />
<br />
* Queries are normalized: queries that are identical except for their constant values will be considered the same, as long as their post-parse analysis query tree (that is, the internal representation of the query before rule expansion) are the same. This also implies that differences that are not semantically essential to the query, such as variations in whitespace or alias names, or the use of one particular syntax over another equivalent one will not differentiate queries.<br />
<br />
<pre><br />
=#SELECT * FROM words WHERE word= 'foo';<br />
word <br />
------<br />
(0 ligne)<br />
<br />
=# SELECT * FROM words WHERE word= 'bar';<br />
word <br />
------<br />
bar<br />
<br />
=#select * from pg_stat_statements where query like '%words where%';<br />
-[ RECORD 1 ]-------+-----------------------------------<br />
userid | 10<br />
dbid | 16384<br />
query | SELECT * FROM words WHERE word= ?;<br />
calls | 2<br />
total_time | 142.314<br />
rows | 1<br />
shared_blks_hit | 3<br />
shared_blks_read | 5<br />
shared_blks_dirtied | 0<br />
shared_blks_written | 0<br />
local_blks_hit | 0<br />
local_blks_read | 0<br />
local_blks_dirtied | 0<br />
local_blks_written | 0<br />
temp_blks_read | 0<br />
temp_blks_written | 0<br />
blk_read_time | 142.165<br />
blk_write_time | 0<br />
<br />
</pre><br />
<br />
The two queries are shown as one in pg_stat_statements.<br />
<br />
* For prepared statements, the execution part (execute statement) is charged on the prepare statement. That way it is easier to use, and avoids the double-counting there was with PostgreSQL 9.1.<br />
<br />
* pg_stat_statements displays timing in milliseconds, to be consistent with other system views.<br />
<br />
= Explain improvements=<br />
<br />
* Timing can now be disabled with EXPLAIN (analyze on, timing off), leading to lower overhead on platforms where getting the current time is expensive <!--Tomas Vondra--><br />
<br />
=# EXPLAIN (analyze on,timing off) SELECT * FROM reservation ;<br />
QUERY PLAN <br />
----------------------------------------------------------------------------------------<br />
Seq Scan on reservation (cost=0.00..22.30 rows=1230 width=36) (actual rows=2 loops=1)<br />
Total runtime: 0.045 ms<br />
<br />
<br />
<br />
* Have EXPLAIN ANALYZE report the number of rows rejected by filter steps <!--(Marko Tiikkaja)--><br />
<br />
This new feature makes it much easier to know how many rows are removed by a filter (and spot potential places to put indexes):<br />
<br />
=# EXPLAIN ANALYZE SELECT * FROM test WHERE a ~ 'tra';<br />
QUERY PLAN <br />
---------------------------------------------------------------------------------------------------------------<br />
Seq Scan on test (cost=0.00..106876.56 rows=2002 width=11) (actual time=2.914..8538.285 rows=120256 loops=1)<br />
Filter: (a ~ 'tra'::text)<br />
Rows Removed by Filter: 5905600<br />
Total runtime: 8549.539 ms<br />
(4 rows)<br />
<br />
=Backward compatibility=<br />
<br />
These changes may incur regressions in your applications.<br />
<br />
==Ensure that xpath() escapes special characters in string values <!-- (Florian Pflug)--> ==<br />
<br />
Before 9.2:<br />
<pre><br />
SELECT (XPATH('/*/text()', '<root>&amp;lt;</root>'))[1];<br />
xpath <br />
-------<br />
<<br />
<br />
'<' Isn't valid XML.<br />
</pre><br />
With 9.2:<br />
<pre><br />
SELECT (XPATH('/*/text()', '<root>&amp;lt;</root>'))[1];<br />
xpath <br />
-------<br />
&amp;lt;<br />
</pre><br />
<br />
==Remove hstore's => operator <!-- (Robert Haas)-->==<br />
Up to 9.1, one could use the => operator to create a hstore. Hstore is a contrib, used to store key/values pairs in a column.<br />
<br />
In 9.1:<br />
<pre><br />
=# SELECT 'a'=>'b';<br />
?column? <br />
----------<br />
"a"=>"b"<br />
(1 row)<br />
<br />
=# SELECT pg_typeof('a'=>'b');<br />
pg_typeof <br />
-----------<br />
hstore<br />
(1 row)<br />
</pre><br />
<br />
With 9.2:<br />
<pre><br />
SELECT 'a'=>'b';<br />
ERROR: operator does not exist: unknown => unknown at character 11<br />
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.<br />
STATEMENT: SELECT 'a'=>'b';<br />
ERROR: operator does not exist: unknown => unknown<br />
LINE 1: SELECT 'a'=>'b';<br />
^<br />
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.<br />
</pre><br />
<br />
It doesn't mean one cannot use '=>' in hstores, it just isn't an operator anymore:<br />
<br />
<pre><br />
=# select hstore('a=>b');<br />
hstore <br />
----------<br />
"a"=>"b"<br />
(1 row)<br />
<br />
=# select hstore('a','b');<br />
hstore <br />
----------<br />
"a"=>"b"<br />
(1 row)<br />
</pre><br />
are still two valid ways to input a hstore.<br />
<br />
"=>" is removed as an operator as it is a reserved keyword in SQL.<br />
<br />
<br />
==Have pg_relation_size() and friends return NULL if the object does not exist <!-- (Phil Sorber)-->==<br />
<br />
A relation could be dropped by a concurrent session, while one was doing a pg_relation_size on it, leading to a SQL exception. Now, it merely returns NULL for this record.<br />
<br />
<br />
==Remove the spclocation field from pg_tablespace <!-- (Magnus Hagander)-->==<br />
<br />
The spclocation field provided the real location of the tablespace. It was filled in during the CREATE or ALTER TABLESPACE command. So it could be wrong: somebody just had to shutdown the cluster, move the tablespace's directory, re-create the symlink in pg_tblspc, and forget to update the spclocation field. The cluster would still run, as the spclocation wasn't used.<br />
<br />
So this field has been removed. To get the tablespace's location, use pg_tablespace_location():<br />
<br />
<pre><br />
=# SELECT *, pg_tablespace_location(oid) AS spclocation FROM pg_tablespace;<br />
spcname | spcowner | spcacl | spcoptions | spclocation <br />
------------+----------+--------+------------+----------------<br />
pg_default | 10 | | | <br />
pg_global | 10 | | | <br />
tmptblspc | 10 | | | /tmp/tmptblspc<br />
</pre><br />
<br />
==Have EXTRACT of a non-timezone-aware value measure the epoch from local midnight, not UTC midnight <!-- (Tom Lane) -->==<br />
<br />
<br />
With PostgreSQL 9.1:<br />
<br />
<pre><br />
=#SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamp);<br />
date_part <br />
------------<br />
1341180000<br />
(1 row)<br />
<br />
=# SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamptz);<br />
date_part <br />
------------<br />
1341180000<br />
(1 row)<br />
</pre><br />
<br />
There is no difference in behaviour between a timstamp with or without timezone.<br />
<br />
With 9.1:<br />
<pre><br />
=#SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamp);<br />
date_part <br />
------------<br />
1341187200<br />
(1 row)<br />
<br />
=# SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamptz);<br />
date_part <br />
------------<br />
1341180000<br />
(1 row)<br />
</pre><br />
When the timestamp has no timezone, the epoch is calculated with the "local midnight", meaning the 1st january of 1970 at midnight, local-time.<br />
<br />
<br />
==Fix to_date() and to_timestamp() to wrap incomplete dates toward 2020 <!-- (Bruce Momjian)-->==<br />
<br />
The wrapping was not consistent between 2 digit dates and 3 digit dates: 2 digit dates always chose the date closest to 2020, 3 digit dates mapped dates from 100 to 999 on 1100 to 1999, and 000 to 099 on 2000 to 2099.<br />
<br />
Now PostgreSQL chooses the date closest to 2020, for 2 and 3 digit dates.<br />
<br />
With 9.1:<br />
<pre><br />
=# SELECT to_date('200-07-02','YYY-MM-DD');<br />
to_date <br />
------------<br />
1200-07-02<br />
</pre><br />
<br />
With 9.2:<br />
<pre><br />
SELECT to_date('200-07-02','YYY-MM-DD');<br />
to_date <br />
------------<br />
2200-07-02<br />
</pre><br />
<br />
<br />
==pg_stat_activity and pg_stat_replication's definitions have changed <!--Magnus Hagander -->==<br />
<br />
The view pg_stat_activity has changed. It's not backward compatible, but let's see what this new definition brings us:<br />
<br />
* current_query disappears and is replaced by two columns:<br />
** state: is the session running a query, waiting<br />
** query: what is the last run (or still running if stat is "active") query<br />
* The column procpid is renamed to pid, to be consistent with other system views<br />
<br />
The benefit is mostly for tracking «idle in transaction» sessions. Up until now, all we could know was that one of these sessions was idle in transaction, meaning it has started a transaction, maybe done some operations, but still not committed. If that session stayed in this state for a while, there was no way of knowing how it got in this state.<br />
<br />
Here is an example:<br />
<pre><br />
-[ RECORD 1 ]----+---------------------------------<br />
datid | 16384<br />
datname | postgres<br />
pid | 20804<br />
usesysid | 10<br />
usename | postgres<br />
application_name | psql<br />
client_addr | <br />
client_hostname | <br />
client_port | -1<br />
backend_start | 2012-07-02 15:02:51.146427+02<br />
xact_start | 2012-07-02 15:15:28.386865+02<br />
query_start | 2012-07-02 15:15:30.410834+02<br />
state_change | 2012-07-02 15:15:30.411287+02<br />
waiting | f<br />
state | idle in transaction<br />
query | DELETE FROM test;<br />
</pre><br />
<br />
With PostgreSQL 9.1, all we would have would be «idle in transaction».<br />
<br />
As this change was backward-incompatible, procpid was also renamed to pid, to be more consistent with other system views.<br />
The view pg_stat_replication has also changed. The column procpid is renamed to pid, to also be consistent with other system views.<br />
<br />
==Change all SQL-level statistics timing values to float8-stored milliseconds <!-- (Tom Lane) -->==<br />
<br />
pg_stat_user_functions.total_time, pg_stat_user_functions.self_time, pg_stat_xact_user_functions.total_time, pg_stat_xact_user_functions.self_time, and pg_stat_statements.total_time (contrib) are now in milliseconds, to be consistent with the rest of the timing values.<br />
<br />
==postgresql.conf parameters changes <!-- (Heikki Linnakangas, Tom Lane, Peter Eisentraut) -->==<br />
<br />
* silent_mode has been removed. Use pg_ctl -l postmaster.log<br />
* wal_sender_delay has been removed. It is no longer needed<br />
* custom_variable_classes has been removed. All «classes» are accepted without declaration now<br />
* ssl_ca_file, ssl_cert_file, ssl_crl_file, ssl_key_file have been added, meaning you can now specify the ssl files<br />
<br />
= Other new features =<br />
<br />
== DROP INDEX CONCURRENTLY ==<br />
<br />
The regular DROP INDEX command takes an exclusive lock on the table. Most of the time, this isn't a problem, because this lock is short-lived. The problem usually occurs when:<br />
<br />
* A long-running transaction is running, and has a (shared) lock on the table<br />
* A DROP INDEX is run on this table in another session, asking for an exclusive lock (and waiting for it, as it won't be granted until the long-running transaction ends)<br />
<br />
At this point, all other transactions needing to take a shared lock on the table (for a simple SELECT for instance) will have to wait too: their lock acquisition is queued after the DROP INDEX's one.<br />
<br />
<br />
DROP INDEX CONCURRENTLY works around this and won't lock normal DML statements, just as CREATE INDEX CONCURRENTLY. The limitations are also the same: Since you can only DROP one index with the CONCURRENTLY option, and the CASCADE option is not supported.<br />
<br />
== NOT VALID CHECK constraints ==<br />
<br />
PostgreSQL 9.1 introduced «NOT VALID» foreign keys. This has been extended to CHECK constraints. Adding a «NOT VALID» constraint on a table means that current data won't be validated, only new and updated rows.<br />
<br />
=# CREATE TABLE test (a int); <br />
CREATE TABLE<br />
=# INSERT INTO test SELECT generate_series(1,100);<br />
INSERT 0 100<br />
=# ALTER TABLE test ADD CHECK (a>100) NOT VALID;<br />
ALTER TABLE<br />
=# INSERT INTO test VALUES (99);<br />
ERROR: new row for relation "test" violates check constraint "test_a_check"<br />
DETAIL: Failing row contains (99).<br />
=# INSERT INTO test VALUES (101);<br />
INSERT 0 1<br />
<br />
Then, later, we can validate the whole table:<br />
<br />
=# ALTER TABLE test VALIDATE CONSTRAINT test_a_check ;<br />
ERROR: check constraint "test_a_check" is violated by some row<br />
<br />
Domains, which are types with added constraints, can also be declared as not valid, and validated later.<br />
<br />
Check constraints can also be renamed now:<br />
<br />
=# ALTER TABLE test RENAME CONSTRAINT test_a_check TO validate_a;<br />
ALTER TABLE<br />
<br />
== NO INHERIT constraints ==<br />
<br />
Here is another improvement about constraints: they can be declared as not inheritable, which will be useful in partitioned environments. Let's take PostgreSQL documentation example, and see how it improves the situation:<br />
<br />
CREATE TABLE measurement (<br />
city_id int not null,<br />
logdate date not null,<br />
peaktemp int,<br />
unitsales int,<br />
CHECK (logdate IS NULL) NO INHERIT<br />
);<br />
<br />
CREATE TABLE measurement_y2006m02 (<br />
CHECK ( logdate >= DATE '2006-02-01' AND logdate < DATE '2006-03-01' )<br />
) INHERITS (measurement);<br />
CREATE TABLE measurement_y2006m03 (<br />
CHECK ( logdate >= DATE '2006-03-01' AND logdate < DATE '2006-04-01' )<br />
) INHERITS (measurement);<br />
<br />
<br />
INSERT INTO measurement VALUES (1,'2006-02-20',1,1);<br />
ERROR: new row for relation "measurement" violates check constraint "measurement_logdate_check"<br />
DETAIL: Failing row contains (1, 2006-02-20, 1, 1).<br />
INSERT INTO measurement_y2006m02 VALUES (1,'2006-02-20',1,1);<br />
INSERT 0 1<br />
<br />
Until now, every check constraint created on measurement would have been inherited by children tables. So adding a constraint forbidding inserts, or allowing only some of them, on the parent table was impossible.<br />
<br />
== Reduce ALTER TABLE rewrites ==<br />
<br />
A table won't get rewritten anymore during an ALTER TABLE when changing the type of a column in the following cases:<br />
<br />
* varchar(x) to varchar(y) when y>=x. It works too if going from varchar(x) to varchar or text (no size limitation)<br />
* numeric(x,z) to numeric(y,z) when y>=x, or to numeric without specifier<br />
* varbit(x) to varbit(y) when y>=x, or to varbit without specifier<br />
* timestamp(x) to timestamp(y) when y>=x or timestamp without specifier<br />
* timestamptz(x) to timestamptz(y) when y>=x or timestamptz without specifier<br />
* interval(x) to interval(y) when y>=x or interval without specifier<br />
<br />
== Security barriers and Leakproof ==<br />
<br />
This new feature has to do with views security. First, let's explain the problem, with a very simplified example:<br />
<br />
=# CREATE TABLE all_data (company_id int, company_data varchar);<br />
CREATE TABLE<br />
# INSERT INTO all_data VALUES (1,'secret_data_for_company_1');<br />
INSERT 0 1<br />
=# INSERT INTO all_data VALUES (2,'secret_data_for_company_2');<br />
INSERT 0 1<br />
=# CREATE VIEW company1_data AS SELECT * FROM all_data WHERE company_id = 1;<br />
CREATE VIEW<br />
<br />
This is a quite classical way of giving access to only a part of a table to a user: we'll create a user for company_id 1, grant to him the right to access company1_data, and deny him the right to access all_data.<br />
<br />
The plan to this query is the following:<br />
<br />
=# explain SELECT * FROM company1_data ;<br />
QUERY PLAN <br />
----------------------------------------------------------<br />
Seq Scan on all_data (cost=0.00..25.38 rows=6 width=36)<br />
Filter: (company_id = 1)<br />
<br />
Even if there was more data, a sequential scan could still be forced: just "SET enable_indexscan to OFF" and the likes.<br />
<br />
So this query reads all the records from all_data, filters them, and returns to the user only the matching rows. There is a way to display scanned records before they are filtered: just create a function with a very low cost, and call it while doing the query:<br />
<br />
CREATE OR REPLACE FUNCTION peek(text) RETURNS boolean LANGUAGE plpgsql AS<br />
$$<br />
BEGIN<br />
RAISE NOTICE '%',$1;<br />
RETURN true;<br />
END<br />
$$<br />
COST 0.1;<br />
<br />
This function just has to cost less than the = operator, which costs 1, to be executed first.<br />
<br />
The result is this:<br />
<br />
<br />
=# SELECT * FROM company1_data WHERE peek(company1_data.company_data);<br />
NOTICE: secret_data_for_company_1<br />
NOTICE: secret_data_for_company_2<br />
company_id | company_data <br />
------------+---------------------------<br />
1 | secret_data_for_company_1<br />
(1 row)<br />
<br />
We got access to the record from the second company (in the NOTICE messages).<br />
<br />
So this is the first new feature: the view can be declared as implementing "security barriers":<br />
<br />
<br />
=# CREATE VIEW company1_data WITH (security_barrier) AS SELECT * FROM all_data WHERE company_id = 1;<br />
CREATE VIEW<br />
=# SELECT * FROM company1_data WHERE peek(company1_data.company_data);<br />
NOTICE: secret_data_for_company_1<br />
company_id | company_data <br />
------------+---------------------------<br />
1 | secret_data_for_company_1<br />
(1 row)<br />
<br />
The view is not leaking anymore. The problem, of course, is that there is a performance impact: maybe the "peek" function could have made the query faster, by filtering lots of rows early in the plan.<br />
<br />
This leads to the complementary feature: some function may be declared as "LEAKPROOF", meaning that they won't leak the data they are passed into error or notice messages.<br />
<br />
Declaring our peek function as LEAKPROOF is a very bad idea, but let's do it just to demonstrate how it's used:<br />
<br />
CREATE OR REPLACE FUNCTION peek(text) RETURNS boolean LEAKPROOF LANGUAGE plpgsql AS<br />
$$<br />
BEGIN<br />
RAISE NOTICE '%',$1;<br />
RETURN true;<br />
END<br />
$$<br />
COST 0.1;<br />
<br />
A LEAKPROOF function is executed «normally»:<br />
<br />
=# SELECT * FROM company1_data WHERE peek(company1_data.company_data);<br />
NOTICE: secret_data_for_company_1<br />
NOTICE: secret_data_for_company_2<br />
company_id | company_data <br />
------------+---------------------------<br />
1 | secret_data_for_company_1<br />
(1 row)<br />
<br />
Of course, in our case, peek isn't LEAKPROOF and shouldn't be declared as such. Only superuser have the permission to declare a LEAKPROOF function.<br />
<br />
== New options for pg_dump ==<br />
<br />
Until now, one could ask pg_dump to dump a table's data, or a table's meta-data (DDL statements for creating the table's structure, indexes, constraints). Some meta-data is better restored before the data (the table's structure, check constraints), some is better after the data (indexes, unique constraints, foreign keys…), for performance reasons mostly.<br />
<br />
So there are now a few more options:<br />
<br />
* --section=pre-data: dump what's needed before restoring the data. Of course, this can be combined with a -t for instance, to specify one table<br />
* --section=post-data : dump what's needed after restoring the data.<br />
* --section=data: dump the data<br />
* --exclude-table-data: dump everything, except THIS table's data. It means pg_dump will still dump other tables' data.<br />
<br />
<br />
<br />
<br />
<br />
[[Category:PostgreSQL 9.2]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=What%27s_new_in_PostgreSQL_9.2&diff=18113What's new in PostgreSQL 9.22012-08-29T18:41:42Z<p>Schmiddy: grammar fixes</p>
<hr />
<div>{{Languages}}<br />
<br />
This document showcases many of the latest developments in PostgreSQL 9.2, compared to the last major release &ndash; PostgreSQL 9.1. There are many improvements in this release, so this wiki page covers many of the more important changes in detail. The full list of changes is itemised in ''Release Notes''.<br />
<br />
<br />
=Major new features=<br />
<br />
==Index-only scans <!-- Robert Haas, Ibrar Ahmed, Heikki Linnakangas, Tom Lane -->==<br />
<br />
In PostgreSQL, indexes have no "visibility" information. It means that when you access a record by its index, PostgreSQL has to visit the real tuple in the table to be sure it is visible to you: the tuple the index points to may simply be an old version of the record you are looking for.<br />
<br />
It can be a very big performance problem: the index is mostly ordered, so accessing its records is quite efficient, while the records may be scattered all over the place (that's a reason why PostgreSQL has a cluster command, but that's another story). In 9.2, PostgreSQL will use an "Index Only Scan" when possible, and not access the record itself if it doesn't need to.<br />
<br />
There is still no visibility information in the index. So in order to do this, PostgreSQL uses the visibility map ([http://www.postgresql.org/docs/devel/static/storage-vm.html visibility map]) , which tells it whether the whole content of a (usually) 8K page is visible to all transactions or not. When the index record points to a tuple contained in an «all visible» page, PostgreSQL won't have to access the tuple, it will be able to build it directly from the index. Of course, all the columns requested by the query must be in the index.<br />
<br />
The visibility map is maintained by VACUUM (it sets the visible bit), and by the backends doing SQL work (they unset the visible bit).<br />
<br />
If the data has been read only since the last VACUUM then the data is All Visible and the index only scan feature can improve performance.<br />
<br />
Here is an example.<br />
<br />
CREATE TABLE demo_ios (col1 float, col2 float, col3 text);<br />
<br />
In this table, we'll put random data, in order to have "scattered" data. We'll put 100 million records, to have a big recordset, and have it not fit in memory (that's a 4GB-ram machine). This is an ideal case, made for this demo. The gains won't be that big in real life.<br />
<br />
INSERT INTO demo_ios SELECT generate_series(1,100000000),random(), 'mynotsolongstring';<br />
<br />
SELECT pg_size_pretty(pg_total_relation_size('demo_ios'));<br />
pg_size_pretty <br />
----------------<br />
6512 MB<br />
<br />
Let's pretend that the query is this:<br />
<br />
SELECT col1,col2 FROM demo_ios where col2 BETWEEN 0.01 AND 0.02<br />
<br />
In order to use an index only scan on this query, we need an index on col2,col1 (col2 first, as it is used in the WHERE clause).<br />
<br />
CREATE index idx_demo_ios on demo_ios(col2,col1);<br />
<br />
We vacuum the visibility map to be up-to-date:<br />
<br />
VACUUM demo_ios;<br />
<br />
All the timing you'll see below are done on a cold OS and PostgreSQL cache (that's where the gains are, as the purpose on Index Only Scans is to reduce I/O).<br />
<br />
Let's first try without Index Only Scans:<br />
<br />
SET enable_indexonlyscan to off;<br />
<br />
EXPLAIN (analyze,buffers) select col1,col2 FROM demo_ios where col2 between 0.01 and 0.02;<br />
QUERY PLAN <br />
----------------------------------------------------------------------------------------------------------------------------------------<br />
Bitmap Heap Scan on demo_ios (cost=25643.01..916484.44 rows=993633 width=16) (actual time=763.391..362963.899 rows=1000392 loops=1)<br />
Recheck Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))<br />
Rows Removed by Index Recheck: 68098621<br />
Buffers: shared hit=2 read=587779<br />
-> Bitmap Index Scan on idx_demo_ios (cost=0.00..25394.60 rows=993633 width=0) (actual time=759.011..759.011 rows=1000392 loops=1)<br />
Index Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))<br />
Buffers: shared hit=2 read=3835<br />
Total runtime: 364390.127 ms<br />
<br />
<br />
With Index Only Scans:<br />
<br />
explain (analyze,buffers) select col1,col2 from demo_ios where col2 between 0.01 and 0.02;<br />
QUERY PLAN <br />
-----------------------------------------------------------------------------------------------------------------------------------------------<br />
Index Only Scan using idx_demo_ios on demo_ios (cost=0.00..35330.93 rows=993633 width=16) (actual time=58.100..3250.589 rows=1000392 loops=1)<br />
Index Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))<br />
Heap Fetches: 0<br />
Buffers: shared hit=923073 read=3848<br />
Total runtime: 4297.405 ms<br />
<br />
<br />
<br />
As nothing is free, there are a few things to keep in mind:<br />
<br />
* Adding indexes for index only scans obviously adds indexes to your table. So updates will be slower.<br />
* You will index columns that weren't indexed before. So there will be less opportunities for HOT updates.<br />
* Gains will probably be smaller in real life situations, especially when data is changed between VACUUMs<br />
<br />
This required making visibility map changes crash-safe, so visibility map bit changes are now WAL-logged.<br />
<br />
==Replication improvements <!-- Fujii Masao, Simon Riggs, Magnus Hagander, Jun Ishizuka -->==<br />
<br />
Streaming Replication becomes more polished with this release. <br />
<br />
One on the main remaining gripes about streaming replication is that all the slaves have to be connected to the same and unique master, consuming its resources. Moreover, in case of a failover, it could be complicated to reconnect all the remaining slaves to the newly promoted master, if not using a tool like repmgr. <br />
<br />
* With 9.2, a standby can also send replication changes, allowing cascading replication.<br />
<br />
Let's build this. We start with an already working 9.2 database.<br />
<br />
We set it up for replication:<br />
<br />
postgresql.conf:<br />
wal_level=hot_standby #(could be archive too)<br />
max_wal_senders=5<br />
hot_standby=on<br />
<br />
You'll probably also want to activate archiving in production, it won't be done here.<br />
<br />
pg_hba.conf (do not use trust in production):<br />
host replication replication_user 0.0.0.0/0 md5<br />
<br />
Create the user:<br />
create user replication_user replication password 'secret';<br />
<br />
Clone the database:<br />
<br />
pg_basebackup -h localhost -U replication_user -D data2<br />
Password:<br />
<br />
We have a brand new cluster in the data2 directory. We'll change the port so that it can start (postgresql.conf):<br />
port=5433<br />
<br />
We add a recovery.conf to tell it how to stream from the master database:<br />
standby_mode = on<br />
primary_conninfo = 'host=localhost port=5432 user=replication_user password=secret' <br />
<br />
pg_ctl -D data2 start<br />
server starting<br />
LOG: database system was interrupted; last known up at 2012-07-03 17:58:09 CEST<br />
LOG: creating missing WAL directory "pg_xlog/archive_status"<br />
LOG: entering standby mode<br />
LOG: streaming replication successfully connected to primary<br />
LOG: redo starts at 0/9D000020<br />
LOG: consistent recovery state reached at 0/9D0000B8<br />
LOG: database system is ready to accept read only connections<br />
<br />
Now, let's add a second slave, which will use this slave:<br />
<br />
<br />
pg_basebackup -h localhost -U replication_user -D data3 -p 5433<br />
Password: <br />
<br />
We edit data3's postgresql.conf to change the port:<br />
port=5434<br />
<br />
We modify the recovery.conf to stream from the slave:<br />
standby_mode = on<br />
primary_conninfo = 'host=localhost port=5433 user=replication_user password=secret' # e.g. 'host=localhost port=5432'<br />
<br />
We start the cluster:<br />
pg_ctl -D data3 start<br />
server starting<br />
LOG: database system was interrupted while in recovery at log time 2012-07-03 17:58:09 CEST<br />
HINT: If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.<br />
LOG: creating missing WAL directory "pg_xlog/archive_status"<br />
LOG: entering standby mode<br />
LOG: streaming replication successfully connected to primary<br />
LOG: redo starts at 0/9D000020<br />
LOG: consistent recovery state reached at 0/9E000000<br />
LOG: database system is ready to accept read only connections<br />
<br />
Now, everything modified on the master cluster get streamed to the first slave, and from there to the second slave. This second replication has to be monitored from the first slave (the master knows nothing about it).<br />
<br />
<br />
* As you may have noticed from the examble, pg_basebackup now works from slaves.<br />
<br />
* There is another use case that wasn't covered: what if a user didn't care for having a full fledged slave, and only wanted to stream the WAL files to another location, to benefit from the reduced data loss without the burden of maintaining a slave ?<br />
<br />
pg_receivexlog is provided just for this purpose: it pretends to be a PostgreSQL slave, but only stores the log files as they are streamed, in a directory:<br />
pg_receivexlog -D /tmp/new_logs -h localhost -U replication_user<br />
<br />
will connect to the master (or a slave), and start creating files: <br />
ls /tmp/new_logs/<br />
00000001000000000000009E.partial<br />
<br />
Files are of the segment size, so they can be used for a normal recovery of the database. It's the same as an archive command, but with a much smaller granularity.<br />
Remember to rename the last segment to remove the .partial suffix before using it with PITR or other.<br />
<br />
* synchronous_commit has a new value: remote_write. It can be used when there is a synchronous slave (synchronous_standby_names is set), meaning that the master doesn't have to wait for the slave to have written the data to disk, only for the slave to have acknowledged the data. With this set, data is protected from a crash on the master, but could still be lost if the slave crashed at the same time (i.e. before having written the in flight data to disk). As this is a quite remote possibility, some people will be interested in this compromise.<br />
<br />
==JSON datatype==<br />
<br />
The JSON datatype is meant for storing JSON-structured data. It will validate that the input JSON string is correct JSON:<br />
<br />
=# SELECT '{"username":"john","posts":121,"emailaddress":"john@nowhere.com"}'::json;<br />
json <br />
-------------------------------------------------------------------<br />
{"username":"john","posts":121,"emailaddress":"john@nowhere.com"}<br />
(1 row)<br />
<br />
=# SELECT '{"username","posts":121,"emailaddress":"john@nowhere.com"}'::json;<br />
ERROR: invalid input syntax for type json at character 8<br />
DETAIL: Expected ":", but found ",".<br />
CONTEXT: JSON data, line 1: {"username",...<br />
STATEMENT: SELECT '{"username","posts":121,"emailaddress":"john@nowhere.com"}'::json;<br />
ERROR: invalid input syntax for type json<br />
LINE 1: SELECT '{"username","posts":121,"emailaddress":"john@nowhere...<br />
^<br />
DETAIL: Expected ":", but found ",".<br />
CONTEXT: JSON data, line 1: {"username",...<br />
<br />
You can also convert a row type to JSON:<br />
<br />
=#SELECT * FROM demo ;<br />
username | posts | emailaddress <br />
----------+-------+---------------------<br />
john | 121 | john@nowhere.com<br />
mickael | 215 | mickael@nowhere.com<br />
(2 rows)<br />
<br />
=# SELECT row_to_json(demo) FROM demo;<br />
row_to_json <br />
-------------------------------------------------------------------------<br />
{"username":"john","posts":121,"emailaddress":"john@nowhere.com"}<br />
{"username":"mickael","posts":215,"emailaddress":"mickael@nowhere.com"}<br />
(2 rows)<br />
<br />
Or an array type:<br />
<br />
<br />
=# select array_to_json(array_agg(demo)) from demo;<br />
array_to_json <br />
---------------------------------------------------------------------------------------------------------------------------------------------<br />
[{"username":"john","posts":121,"emailaddress":"john@nowhere.com"},{"username":"mickael","posts":215,"emailaddress":"mickael@nowhere.com"}]<br />
(1 row)<br />
<br />
== Range Types ==<br />
<br />
Range types are used to store a range of data of a given type. There are a few pre-defined types. They are integer (int4range), bigint (int8range), numeric (numrange), timestamp without timezone (tsrange), timestamp with timezone (tstzrange), and date (daterange).<br />
<br />
Ranges can be made of continuous (numeric, timestamp...) or discrete (integer, date...) data types. They can be open (the bound isn't part of the range) or closed (the bound is part of the range). A bound can also be infinite.<br />
<br />
Without these datatypes, most people solve the range problems by using two columns in a table. These range types are much more powerful, as you can use many operators on them:<br />
<br />
Here is the intersection between then 1000(open)-2000(closed) and 1000(closed)-1200(closed) numeric range:<br />
<br />
SELECT '(1000,2000]'::numrange * '[1000,1200]'::numrange;<br />
?column? <br />
-------------<br />
(1000,1200]<br />
(1 row)<br />
<br />
So you can query on things like: «give me all ranges that intersect this»:<br />
<br />
=# SELECT * from test_range ;<br />
period <br />
-----------------------------------------------------<br />
["2012-01-01 00:00:00+01","2012-01-02 12:00:00+01"]<br />
["2012-01-01 00:00:00+01","2012-03-01 00:00:00+01"]<br />
["2008-01-01 00:00:00+01","2015-01-01 00:00:00+01"]<br />
(3 rows)<br />
<br />
<br />
=# SELECT * FROM test_range WHERE period && '[2012-01-03 00:00:00,2012-01-03 12:00:00]'; <br />
period <br />
-----------------------------------------------------<br />
["2012-01-01 00:00:00+01","2012-03-01 00:00:00+01"]<br />
["2008-01-01 00:00:00+01","2015-01-01 00:00:00+01"]<br />
(2 rows)<br />
<br />
This query could use an index defined like this:<br />
<br />
=# CREATE INDEX idx_test_range on test_range USING gist (period);<br />
<br />
<br />
<br />
You can also use these range data types to define exclusion constraints:<br />
<br />
CREATE EXTENSION btree_gist ;<br />
CREATE TABLE reservation (room_id int, period tstzrange);<br />
ALTER TABLE reservation ADD EXCLUDE USING GIST (room_id WITH =, period WITH &&);<br />
<br />
This means that now it is forbidden to have two records in this table where room_id is equal and period overlaps. The extension btree_gist is required to create a GiST index on room_id (it's an integer, it is usually indexed with a btree index).<br />
<br />
<br />
=# INSERT INTO reservation VALUES (1,'(2012-08-23 14:00:00,2012-08-23 15:00:00)');<br />
INSERT 0 1<br />
=# INSERT INTO reservation VALUES (2,'(2012-08-23 14:00:00,2012-08-23 15:00:00)');<br />
INSERT 0 1<br />
=# INSERT INTO reservation VALUES (1,'(2012-08-23 14:45:00,2012-08-23 15:15:00)');<br />
ERROR: conflicting key value violates exclusion constraint "reservation_room_id_period_excl"<br />
DETAIL: Key (room_id, period)=(1, ("2012-08-23 14:45:00+02","2012-08-23 15:15:00+02")) <br />
conflicts with existing key (room_id, period)=(1, ("2012-08-23 14:00:00+02","2012-08-23 15:00:00+02")).<br />
STATEMENT: INSERT INTO reservation VALUES (1,'(2012-08-23 14:45:00,2012-08-23 15:15:00)');<br />
<br />
One can also declare new range types.<br />
<br />
=Performance improvements=<br />
<br />
This version has performance improvements on a very large range of domains (non-exaustive):<br />
<br />
* The most visible will probably be the Index Only Scans, which has already been introduced in this document.<br />
<br />
* The lock contention of several big locks has been significantly reduced, leading to better multi-processor scalability, for machines with over 32 cores mostly. <!-- Robert Haas --><br />
<br />
* The performance of in-memory sorts has been improved by up to 25% in some situations, with certain specialized sort functions introduced. <!-- Peter Geoghegan --><br />
<br />
* An idle PostgreSQL server now makes less wakeups, leading to lower power consumption<!--Peter Geoghegan-->. This is especially useful on virtualized and embedded environments.<br />
<br />
* COPY has been improved, it will generate less WAL volume and fewer locks of a table's pages. <!-- Heikki Linnakangas --><br />
<br />
* Statistics are collected on array contents<!-- Alexander Korotkov -->, allowing for better estimations of selectivity on array operations.<br />
<br />
* Text-to-anytype concatenation and quote_literal/quote_nullable functions are not volatile any more, enabling better optimization in some cases <!-- Marti Raudsepp --><br />
<br />
* The system can now track IO durations <!--Ants Aasma --><br />
<br />
This one deserves a little explanation, as it can be a little tricky. Tracking IO durations means asking repeatedly the time to the operating system. Depending on the operating system and the hardware, this can be quite cheap, or extremely costly. The most import factor here is where the system gets its time from. It could be directly retrieved from the processor (TSC), dedicated hardware such as HPET, or an ACPI call. What's most important is that the cost of getting time can vary from a factor of thousands.<br />
<br />
If you are interested in this timing data, it's better to first check if your system will support it without to much of a performance hit. PostgreSQL provides you with the pg_test_timing tool:<br />
<br />
<pre><br />
$ pg_test_timing <br />
Testing timing overhead for 3 seconds.<br />
Per loop time including overhead: 28.02 nsec<br />
Histogram of timing durations:<br />
< usec: count percent<br />
32: 41 0.00004%<br />
16: 1405 0.00131%<br />
8: 200 0.00019%<br />
4: 388 0.00036%<br />
2: 2982558 2.78523%<br />
1: 104100166 97.21287%<br />
</pre><br />
<br />
Here, everything is good: getting time costs around 28 nanoseconds, and has a very small variation. Anything under 100 nanoseconds should be good for production. If you get higher values, you may still find a way to tune your system. You'd better check on the [http://www.postgresql.org/docs/9.2/static/pgtesttiming.html documentation].<br />
<br />
Anyway, here is the data you'll be able to collect if your system is ready for this:<br />
<br />
First, you'll get per-database statistics, which will now give accurate informations about which database is doing most I/O:<br />
<br />
<pre><br />
=# SELECT * FROM pg_stat_database WHERE datname = 'mydb';<br />
-[ RECORD 1 ]--+------------------------------<br />
datid | 16384<br />
datname | mydb<br />
numbackends | 1<br />
xact_commit | 270<br />
xact_rollback | 2<br />
blks_read | 1961<br />
blks_hit | 17944<br />
tup_returned | 269035<br />
tup_fetched | 8850<br />
tup_inserted | 16<br />
tup_updated | 4<br />
tup_deleted | 45<br />
conflicts | 0<br />
temp_files | 0<br />
temp_bytes | 0<br />
deadlocks | 0<br />
blk_read_time | 583.774<br />
blk_write_time | 0<br />
stats_reset | 2012-07-03 17:18:54.796817+02<br />
</pre><br />
We see here that mydb has only consumed 583.774 milliseconds of read time.<br />
<br />
Explain will benefit from this too:<br />
<pre><br />
=# EXPLAIN (analyze,buffers) SELECT count(*) FROM mots ;<br />
QUERY PLAN<br />
----------------------------------------------------------------------------------------------------------------<br />
Aggregate (cost=1669.95..1669.96 rows=1 width=0) (actual time=21.943..21.943 rows=1 loops=1)<br />
Buffers: shared read=493<br />
I/O Timings: read=2.578<br />
-> Seq Scan on mots (cost=0.00..1434.56 rows=94156 width=0) (actual time=0.059..12.933 rows=94156 loops=1)<br />
Buffers: shared read=493<br />
I/O Timings: read=2.578<br />
Total runtime: 22.059 ms<br />
</pre><br />
We now have a separate information about the time taken to retrieve data from the operating system. Obviously, here, the data was in the operating system's cache (2 milliseconds to read 493 blocks).<br />
<br />
And last, if you have enabled pg_stat_statements:<br />
<pre><br />
select * from pg_stat_statements where query ~ 'words';<br />
-[ RECORD 1 ]-------+---------------------------<br />
userid | 10<br />
dbid | 16384<br />
query | select count(*) from words;<br />
calls | 2<br />
total_time | 78.332<br />
rows | 2<br />
shared_blks_hit | 0<br />
shared_blks_read | 986<br />
shared_blks_dirtied | 0<br />
shared_blks_written | 0<br />
local_blks_hit | 0<br />
local_blks_read | 0<br />
local_blks_dirtied | 0<br />
local_blks_written | 0<br />
temp_blks_read | 0<br />
temp_blks_written | 0<br />
blk_read_time | 58.427<br />
blk_write_time | 0<br />
</pre><br />
<br />
* As for every version, the optimizer has received its share of improvements <!-- Tom Lane--><br />
** Prepared statements used to be optimized once, without any knowledge of the parameters' values. With 9.2, the planner will use specific plans regarding to the parameters sent (the query will be planned at execution), except if the query is executed several times and the planner decides that the generic plan is not too much more expensive than the specific plans.<br />
** A new feature has been added: parameterized paths. Simply put, it means that a sub-part of a query plan can use parameters it has got from a parent node. It fixes several bad plans that could occur, especially when the optimizer couldn't reorder joins to put nested loops where it would have been efficient.<br />
<br />
This example is straight from the developpers mailing lists <!-- Andres Freund -->:<br />
<br />
<pre><br />
CREATE TABLE a (<br />
a_id serial PRIMARY KEY NOT NULL,<br />
b_id integer<br />
);<br />
CREATE INDEX a__b_id ON a USING btree (b_id);<br />
<br />
<br />
CREATE TABLE b (<br />
b_id serial NOT NULL,<br />
c_id integer<br />
);<br />
CREATE INDEX b__c_id ON b USING btree (c_id);<br />
<br />
<br />
CREATE TABLE c (<br />
c_id serial PRIMARY KEY NOT NULL,<br />
value integer UNIQUE<br />
);<br />
<br />
INSERT INTO b (b_id, c_id)<br />
SELECT g.i, g.i FROM generate_series(1, 50000) g(i);<br />
<br />
INSERT INTO a(b_id)<br />
SELECT g.i FROM generate_series(1, 50000) g(i);<br />
<br />
INSERT INTO c(c_id,value)<br />
VALUES (1,1);<br />
</pre><br />
<br />
So we have a referencing b, b referencing c.<br />
<br />
Here is an example of a query working badly with PostgreSQL 9.1:<br />
<br />
<pre><br />
EXPLAIN ANALYZE SELECT 1<br />
FROM<br />
c<br />
WHERE<br />
EXISTS (<br />
SELECT *<br />
FROM a<br />
JOIN b USING (b_id)<br />
WHERE b.c_id = c.c_id)<br />
AND c.value = 1;<br />
QUERY PLAN<br />
-----------------------------------------------------------------------------------------------------------------------<br />
Nested Loop Semi Join (cost=1347.00..3702.27 rows=1 width=0) (actual time=13.799..13.802 rows=1 loops=1)<br />
Join Filter: (c.c_id = b.c_id)<br />
-> Index Scan using c_value_key on c (cost=0.00..8.27 rows=1 width=4) (actual time=0.006..0.008 rows=1 loops=1)<br />
Index Cond: (value = 1)<br />
-> Hash Join (cost=1347.00..3069.00 rows=50000 width=4) (actual time=13.788..13.788 rows=1 loops=1)<br />
Hash Cond: (a.b_id = b.b_id)<br />
-> Seq Scan on a (cost=0.00..722.00 rows=50000 width=4) (actual time=0.007..0.007 rows=1 loops=1)<br />
-> Hash (cost=722.00..722.00 rows=50000 width=8) (actual time=13.760..13.760 rows=50000 loops=1)<br />
Buckets: 8192 Batches: 1 Memory Usage: 1954kB<br />
-> Seq Scan on b (cost=0.00..722.00 rows=50000 width=8) (actual time=0.008..5.702 rows=50000 loops=1)<br />
Total runtime: 13.842 ms<br />
</pre><br />
<br />
Not that bad, 13 milliseconds. Still, we are doing sequential scans on a and b, when our common sense tells us that c.value=1 should be used to filter rows more aggressively.<br />
<br />
Here's what 9.2 does with this query:<br />
<br />
<pre><br />
QUERY PLAN<br />
----------------------------------------------------------------------------------------------------------------------------<br />
Nested Loop Semi Join (cost=0.00..16.97 rows=1 width=0) (actual time=0.035..0.037 rows=1 loops=1)<br />
-> Index Scan using c_value_key on c (cost=0.00..8.27 rows=1 width=4) (actual time=0.007..0.009 rows=1 loops=1)<br />
Index Cond: (value = 1)<br />
-> Nested Loop (cost=0.00..8.69 rows=1 width=4) (actual time=0.025..0.025 rows=1 loops=1)<br />
-> Index Scan using b__c_id on b (cost=0.00..8.33 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=1)<br />
Index Cond: (c_id = c.c_id)<br />
-> Index Only Scan using a__b_id on a (cost=0.00..0.35 rows=1 width=4) (actual time=0.014..0.014 rows=1 loops=1)<br />
Index Cond: (b_id = b.b_id)<br />
Total runtime: 0.089 ms<br />
</pre><br />
<br />
The «parameterized path» is:<br />
<pre><br />
-> Nested Loop (cost=0.00..8.69 rows=1 width=4) (actual time=0.025..0.025 rows=1 loops=1)<br />
-> Index Scan using b__c_id on b (cost=0.00..8.33 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=1)<br />
Index Cond: (c_id = c.c_id)<br />
-> Index Only Scan using a__b_id on a (cost=0.00..0.35 rows=1 width=4) (actual time=0.014..0.014 rows=1 loops=1)<br />
Index Cond: (b_id = b.b_id)<br />
Total runtime: 0.089 ms<br />
</pre><br />
<br />
This part of the plan depends on a parent node (c_id=c.c_id). This part of the plan is called each time with a different parameter coming from the parent node.<br />
<br />
This plan is of course much faster, as there is no need to fully scan a, and to fully scan AND hash b.<br />
<br />
=SP-GiST=<br />
<br />
SP-GiST stands for Space Partitionned GiST, GiST being Generalized Search Tree. GiST is an index type, and has been available for quite a while in PostgreSQL. GiST is already very efficient at indexing complex data types, but performance tends to suffer when the source data isn't uniformly distributed. SP-GiST tries to fix that.<br />
<br />
As all indexing methods available in PostgreSQL, SP-GiST is a generic indexing method, meaning its purpose is to index whatever you'll throw at it, using operators you'll provide. It means that if you want to create a new datatype, and make it indexable through SP-GiST, you'll have to follow the documented API.<br />
<br />
SP-GiST can be used to implement 3 type of indexes: trie (suffix) indexing, Quadtree (data is divided into quadrants), and k-d tree (k-dimensional tree).<br />
<br />
For now, SP-GiST is provided with operator families called "quad_point_ops", "kd_point_ops" and "text_ops".<br />
<br />
As their names indicate, the first one indexes point types, using a quadtree, the second one indexes point types using a k-d tree, and the third one indexes text, using suffix.<br />
<br />
=pg_stat_statements=<br />
<br />
This contrib module has received a lot of improvements in this version:<br />
<br />
* Queries are normalized: queries that are identical except for their constant values will be considered the same, as long as their post-parse analysis query tree (that is, the internal representation of the query before rule expansion) are the same. This also implies that differences that are not semantically essential to the query, such as variations in whitespace or alias names, or the use of one particular syntax over another equivalent one will not differentiate queries.<br />
<br />
<pre><br />
=#SELECT * FROM words WHERE word= 'foo';<br />
word <br />
------<br />
(0 ligne)<br />
<br />
=# SELECT * FROM words WHERE word= 'bar';<br />
word <br />
------<br />
bar<br />
<br />
=#select * from pg_stat_statements where query like '%words where%';<br />
-[ RECORD 1 ]-------+-----------------------------------<br />
userid | 10<br />
dbid | 16384<br />
query | SELECT * FROM words WHERE word= ?;<br />
calls | 2<br />
total_time | 142.314<br />
rows | 1<br />
shared_blks_hit | 3<br />
shared_blks_read | 5<br />
shared_blks_dirtied | 0<br />
shared_blks_written | 0<br />
local_blks_hit | 0<br />
local_blks_read | 0<br />
local_blks_dirtied | 0<br />
local_blks_written | 0<br />
temp_blks_read | 0<br />
temp_blks_written | 0<br />
blk_read_time | 142.165<br />
blk_write_time | 0<br />
<br />
</pre><br />
<br />
The two queries are shown as one in pg_stat_statements.<br />
<br />
* For prepared statements, the execution part (execute statement) is charged on the prepare statement. That way it is easier to use, and avoids the double-counting there was with PostgreSQL 9.1.<br />
<br />
* pg_stat_statements displays timing in milliseconds, to be consistent with other system views.<br />
<br />
= Explain improvements=<br />
<br />
* Timing can now be disabled with EXPLAIN (analyze on, timing off), leading to lower overhead on platforms where getting the current time is expensive <!--Tomas Vondra--><br />
<br />
=# EXPLAIN (analyze on,timing off) SELECT * FROM reservation ;<br />
QUERY PLAN <br />
----------------------------------------------------------------------------------------<br />
Seq Scan on reservation (cost=0.00..22.30 rows=1230 width=36) (actual rows=2 loops=1)<br />
Total runtime: 0.045 ms<br />
<br />
<br />
<br />
* Have EXPLAIN ANALYZE report the number of rows rejected by filter steps <!--(Marko Tiikkaja)--><br />
<br />
This new feature makes it much easier to know how many rows are removed by a filter (and spot potential places to put indexes):<br />
<br />
=# EXPLAIN ANALYZE SELECT * FROM test WHERE a ~ 'tra';<br />
QUERY PLAN <br />
---------------------------------------------------------------------------------------------------------------<br />
Seq Scan on test (cost=0.00..106876.56 rows=2002 width=11) (actual time=2.914..8538.285 rows=120256 loops=1)<br />
Filter: (a ~ 'tra'::text)<br />
Rows Removed by Filter: 5905600<br />
Total runtime: 8549.539 ms<br />
(4 rows)<br />
<br />
=Backward compatibility=<br />
<br />
These changes may incur regressions in your applications.<br />
<br />
==Ensure that xpath() escapes special characters in string values <!-- (Florian Pflug)--> ==<br />
<br />
Before 9.2:<br />
<pre><br />
SELECT (XPATH('/*/text()', '<root>&amp;lt;</root>'))[1];<br />
xpath <br />
-------<br />
<<br />
<br />
'<' Isn't valid XML.<br />
</pre><br />
With 9.2:<br />
<pre><br />
SELECT (XPATH('/*/text()', '<root>&amp;lt;</root>'))[1];<br />
xpath <br />
-------<br />
&amp;lt;<br />
</pre><br />
<br />
==Remove hstore's => operator <!-- (Robert Haas)-->==<br />
Up to 9.1, one could use the => operator to create a hstore. Hstore is a contrib, used to store key/values pairs in a column.<br />
<br />
In 9.1:<br />
<pre><br />
=# SELECT 'a'=>'b';<br />
?column? <br />
----------<br />
"a"=>"b"<br />
(1 row)<br />
<br />
=# SELECT pg_typeof('a'=>'b');<br />
pg_typeof <br />
-----------<br />
hstore<br />
(1 row)<br />
</pre><br />
<br />
With 9.2:<br />
<pre><br />
SELECT 'a'=>'b';<br />
ERROR: operator does not exist: unknown => unknown at character 11<br />
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.<br />
STATEMENT: SELECT 'a'=>'b';<br />
ERROR: operator does not exist: unknown => unknown<br />
LINE 1: SELECT 'a'=>'b';<br />
^<br />
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.<br />
</pre><br />
<br />
It doesn't mean one cannot use '=>' in hstores, it just isn't an operator anymore:<br />
<br />
<pre><br />
=# select hstore('a=>b');<br />
hstore <br />
----------<br />
"a"=>"b"<br />
(1 row)<br />
<br />
=# select hstore('a','b');<br />
hstore <br />
----------<br />
"a"=>"b"<br />
(1 row)<br />
</pre><br />
are still two valid ways to input a hstore.<br />
<br />
"=>" is removed as an operator as it is a reserved keyword in SQL.<br />
<br />
<br />
==Have pg_relation_size() and friends return NULL if the object does not exist <!-- (Phil Sorber)-->==<br />
<br />
A relation could be dropped by a concurrent session, while one was doing a pg_relation_size on it, leading to a SQL exception. Now, it merely returns NULL for this record.<br />
<br />
<br />
==Remove the spclocation field from pg_tablespace <!-- (Magnus Hagander)-->==<br />
<br />
The spclocation field provided the real location of the tablespace. It was filled in during the CREATE or ALTER TABLESPACE command. So it could be wrong: somebody just had to shutdown the cluster, move the tablespace's directory, re-create the symlink in pg_tblspc, and forget to update the spclocation field. The cluster would still run, as the spclocation wasn't used.<br />
<br />
So this field has been removed. To get the tablespace's location, use pg_tablespace_location():<br />
<br />
<pre><br />
=# SELECT *, pg_tablespace_location(oid) AS spclocation FROM pg_tablespace;<br />
spcname | spcowner | spcacl | spcoptions | spclocation <br />
------------+----------+--------+------------+----------------<br />
pg_default | 10 | | | <br />
pg_global | 10 | | | <br />
tmptblspc | 10 | | | /tmp/tmptblspc<br />
</pre><br />
<br />
==Have EXTRACT of a non-timezone-aware value measure the epoch from local midnight, not UTC midnight <!-- (Tom Lane) -->==<br />
<br />
<br />
With PostgreSQL 9.1:<br />
<br />
<pre><br />
=#SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamp);<br />
date_part <br />
------------<br />
1341180000<br />
(1 row)<br />
<br />
=# SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamptz);<br />
date_part <br />
------------<br />
1341180000<br />
(1 row)<br />
</pre><br />
<br />
There is no difference in behaviour between a timstamp with or without timezone.<br />
<br />
With 9.1:<br />
<pre><br />
=#SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamp);<br />
date_part <br />
------------<br />
1341187200<br />
(1 row)<br />
<br />
=# SELECT extract(epoch FROM '2012-07-02 00:00:00'::timestamptz);<br />
date_part <br />
------------<br />
1341180000<br />
(1 row)<br />
</pre><br />
When the timestamp has no timezone, the epoch is calculated with the "local midnight", meaning the 1st january of 1970 at midnight, local-time.<br />
<br />
<br />
==Fix to_date() and to_timestamp() to wrap incomplete dates toward 2020 <!-- (Bruce Momjian)-->==<br />
<br />
The wrapping was not consistent between 2 digit dates and 3 digit dates: 2 digit dates always chose the date closest to 2020, 3 digit dates mapped dates from 100 to 999 on 1100 to 1999, and 000 to 099 on 2000 to 2099.<br />
<br />
Now PostgreSQL chooses the date closest to 2020, for 2 and 3 digit dates.<br />
<br />
With 9.1:<br />
<pre><br />
=# SELECT to_date('200-07-02','YYY-MM-DD');<br />
to_date <br />
------------<br />
1200-07-02<br />
</pre><br />
<br />
With 9.2:<br />
<pre><br />
SELECT to_date('200-07-02','YYY-MM-DD');<br />
to_date <br />
------------<br />
2200-07-02<br />
</pre><br />
<br />
<br />
==pg_stat_activity and pg_stat_replication's definitions have changed <!--Magnus Hagander -->==<br />
<br />
The view pg_stat_activity has changed. It's not backward compatible, but let's see what this new definition brings us:<br />
<br />
* current_query disappears and is replaced by two columns:<br />
** state: is the session running a query, waiting<br />
** query: what is the last run (or still running if stat is "active") query<br />
* The column procpid is renamed to pid, to be consistent with other system views<br />
<br />
The benefit is mostly for tracking «idle in transaction» sessions. Up until now, all we could know was that one of these sessions was idle in transaction, meaning it has started a transaction, maybe done some operations, but still not committed. If that session stayed in this state for a while, there was no way of knowing how it got in this state.<br />
<br />
Here is an example:<br />
<pre><br />
-[ RECORD 1 ]----+---------------------------------<br />
datid | 16384<br />
datname | postgres<br />
pid | 20804<br />
usesysid | 10<br />
usename | postgres<br />
application_name | psql<br />
client_addr | <br />
client_hostname | <br />
client_port | -1<br />
backend_start | 2012-07-02 15:02:51.146427+02<br />
xact_start | 2012-07-02 15:15:28.386865+02<br />
query_start | 2012-07-02 15:15:30.410834+02<br />
state_change | 2012-07-02 15:15:30.411287+02<br />
waiting | f<br />
state | idle in transaction<br />
query | DELETE FROM test;<br />
</pre><br />
<br />
With PostgreSQL 9.1, all we would have would be «idle in transaction».<br />
<br />
As this change was backward-incompatible, procpid was also renamed to pid, to be more consistent with other system views.<br />
The view pg_stat_replication has also changed. The column procpid is renamed to pid, to also be consistent with other system views.<br />
<br />
==Change all SQL-level statistics timing values to float8-stored milliseconds <!-- (Tom Lane) -->==<br />
<br />
pg_stat_user_functions.total_time, pg_stat_user_functions.self_time, pg_stat_xact_user_functions.total_time, pg_stat_xact_user_functions.self_time, and pg_stat_statements.total_time (contrib) are now in milliseconds, to be consistent with the rest of the timing values.<br />
<br />
==postgresql.conf parameters changes <!-- (Heikki Linnakangas, Tom Lane, Peter Eisentraut) -->==<br />
<br />
* silent_mode has been removed. Use pg_ctl -l postmaster.log<br />
* wal_sender_delay has been removed. It is no longer needed<br />
* custom_variable_classes has been removed. All «classes» are accepted without declaration now<br />
* ssl_ca_file, ssl_cert_file, ssl_crl_file, ssl_key_file have been added, meaning you can now specify the ssl files<br />
<br />
= Other new features =<br />
<br />
== DROP INDEX CONCURRENTLY ==<br />
<br />
The regular DROP INDEX command takes an exclusive lock on the table. Most of the time, this isn't a problem, because this lock is short-lived. The problem usually occurs when:<br />
<br />
* A long-running transaction is running, and has a (shared) lock on the table<br />
* A DROP INDEX is run on this table in another session, asking for an exclusive lock (and waiting for it, as it won't be granted until the long-running transaction ends)<br />
<br />
At this point, all other transactions needing to take a shared lock on the table (for a simple SELECT for instance) will have to wait too: their lock acquisition is queued after the DROP INDEX's one.<br />
<br />
<br />
DROP INDEX CONCURRENTLY works around this and won't lock normal DML statements, just as CREATE INDEX CONCURRENTLY. The limitations are also the same: Since you can only DROP one index with the CONCURRENTLY option, and the CASCADE option is not supported.<br />
<br />
== NOT VALID CHECK constraints ==<br />
<br />
PostgreSQL 9.1 introduced «NOT VALID» foreign keys. This has been extended to CHECK constraints. Adding a «NOT VALID» constraint on a table means that current data won't be validated, only new and updated rows.<br />
<br />
=# CREATE TABLE test (a int); <br />
CREATE TABLE<br />
=# INSERT INTO test SELECT generate_series(1,100);<br />
INSERT 0 100<br />
=# ALTER TABLE test ADD CHECK (a>100) NOT VALID;<br />
ALTER TABLE<br />
=# INSERT INTO test VALUES (99);<br />
ERROR: new row for relation "test" violates check constraint "test_a_check"<br />
DETAIL: Failing row contains (99).<br />
=# INSERT INTO test VALUES (101);<br />
INSERT 0 1<br />
<br />
Then, later, we can validate the whole table:<br />
<br />
=# ALTER TABLE test VALIDATE CONSTRAINT test_a_check ;<br />
ERROR: check constraint "test_a_check" is violated by some row<br />
<br />
Domains, which are types with added constraints, can also be declared as not valid, and validated later.<br />
<br />
Check constraints can also be renamed now:<br />
<br />
=# ALTER TABLE test RENAME CONSTRAINT test_a_check TO validate_a;<br />
ALTER TABLE<br />
<br />
== NO INHERIT constraints ==<br />
<br />
Here is another improvement about constraints: they can be declared as not inheritable, which will be useful in partitioned environments. Let's take PostgreSQL documentation example, and see how it improves the situation:<br />
<br />
CREATE TABLE measurement (<br />
city_id int not null,<br />
logdate date not null,<br />
peaktemp int,<br />
unitsales int,<br />
CHECK (logdate IS NULL) NO INHERIT<br />
);<br />
<br />
CREATE TABLE measurement_y2006m02 (<br />
CHECK ( logdate >= DATE '2006-02-01' AND logdate < DATE '2006-03-01' )<br />
) INHERITS (measurement);<br />
CREATE TABLE measurement_y2006m03 (<br />
CHECK ( logdate >= DATE '2006-03-01' AND logdate < DATE '2006-04-01' )<br />
) INHERITS (measurement);<br />
<br />
<br />
INSERT INTO measurement VALUES (1,'2006-02-20',1,1);<br />
ERROR: new row for relation "measurement" violates check constraint "measurement_logdate_check"<br />
DETAIL: Failing row contains (1, 2006-02-20, 1, 1).<br />
INSERT INTO measurement_y2006m02 VALUES (1,'2006-02-20',1,1);<br />
INSERT 0 1<br />
<br />
Until now, every check constraint created on measurement would have been inherited by children tables. So adding a constraint forbidding inserts, or allowing only some of them, on the parent table was impossible.<br />
<br />
== Reduce ALTER TABLE rewrites ==<br />
<br />
A table won't get rewritten anymore during an ALTER TABLE when changing the type of a column in the following cases:<br />
<br />
* varchar(x) to varchar(y) when y>=x. It works too if going from varchar(x) to varchar or text (no size limitation)<br />
* numeric(x,z) to numeric(y,z) when y>=x, or to numeric without specifier<br />
* varbit(x) to varbit(y) when y>=x, or to varbit without specifier<br />
* timestamp(x) to timestamp(y) when y>=x or timestamp without specifier<br />
* timestamptz(x) to timestamptz(y) when y>=x or timestamptz without specifier<br />
* interval(x) to interval(y) when y>=x or interval without specifier<br />
<br />
== Security barriers and Leakproof ==<br />
<br />
This new feature has to do with views security. First, let's explain the problem, with a very simplified example:<br />
<br />
=# CREATE TABLE all_data (company_id int, company_data varchar);<br />
CREATE TABLE<br />
# INSERT INTO all_data VALUES (1,'secret_data_for_company_1');<br />
INSERT 0 1<br />
=# INSERT INTO all_data VALUES (2,'secret_data_for_company_2');<br />
INSERT 0 1<br />
=# CREATE VIEW company1_data AS SELECT * FROM all_data WHERE company_id = 1;<br />
CREATE VIEW<br />
<br />
This is a quite classical way of giving access to only a part of a table to a user: we'll create a user for company_id 1, grant to him the right to access company1_data, and deny him the right to access all_data.<br />
<br />
The plan to this query is the following:<br />
<br />
=# explain SELECT * FROM company1_data ;<br />
QUERY PLAN <br />
----------------------------------------------------------<br />
Seq Scan on all_data (cost=0.00..25.38 rows=6 width=36)<br />
Filter: (company_id = 1)<br />
<br />
Even if there was more data, a sequential scan could still be forced: just "SET enable_indexscan to OFF" and the likes.<br />
<br />
So this query reads all the records from all_data, filters them, and returns to the user only the matching rows. There is a way to display scanned records before they are filtered: just create a function with a very low cost, and call it while doing the query:<br />
<br />
CREATE OR REPLACE FUNCTION peek(text) RETURNS boolean LANGUAGE plpgsql AS<br />
$$<br />
BEGIN<br />
RAISE NOTICE '%',$1;<br />
RETURN true;<br />
END<br />
$$<br />
COST 0.1;<br />
<br />
This function just has to cost less than the = operator, which costs 1, to be executed first.<br />
<br />
The result is this:<br />
<br />
<br />
=# SELECT * FROM company1_data WHERE peek(company1_data.company_data);<br />
NOTICE: secret_data_for_company_1<br />
NOTICE: secret_data_for_company_2<br />
company_id | company_data <br />
------------+---------------------------<br />
1 | secret_data_for_company_1<br />
(1 row)<br />
<br />
We got access to the record from the second company (in the NOTICE messages).<br />
<br />
So this is the first new feature: the view can be declared as implementing "security barriers":<br />
<br />
<br />
=# CREATE VIEW company1_data WITH (security_barrier) AS SELECT * FROM all_data WHERE company_id = 1;<br />
CREATE VIEW<br />
=# SELECT * FROM company1_data WHERE peek(company1_data.company_data);<br />
NOTICE: secret_data_for_company_1<br />
company_id | company_data <br />
------------+---------------------------<br />
1 | secret_data_for_company_1<br />
(1 row)<br />
<br />
The view is not leaking anymore. The problem, of course, is that there is a performance impact: maybe the "peek" function could have made the query faster, by filtering lots of rows early in the plan.<br />
<br />
This leads to the complementary feature: some function may be declared as "LEAKPROOF", meaning that they won't leak the data they are passed into error or notice messages.<br />
<br />
Declaring our peek function as LEAKPROOF is a very bad idea, but let's do it just to demonstrate how it's used:<br />
<br />
CREATE OR REPLACE FUNCTION peek(text) RETURNS boolean LEAKPROOF LANGUAGE plpgsql AS<br />
$$<br />
BEGIN<br />
RAISE NOTICE '%',$1;<br />
RETURN true;<br />
END<br />
$$<br />
COST 0.1;<br />
<br />
A LEAKPROOF function is executed «normally»:<br />
<br />
=# SELECT * FROM company1_data WHERE peek(company1_data.company_data);<br />
NOTICE: secret_data_for_company_1<br />
NOTICE: secret_data_for_company_2<br />
company_id | company_data <br />
------------+---------------------------<br />
1 | secret_data_for_company_1<br />
(1 row)<br />
<br />
Of course, in our case, peek isn't LEAKPROOF and shouldn't be declared as such. Only superuser have the permission to declare a LEAKPROOF function.<br />
<br />
== New options for pg_dump ==<br />
<br />
Until now, one could ask pg_dump to dump a table's data, or a table's meta-data (DDL statements for creating the table's structure, indexes, constraints). Some meta-data is better restored before the data (the table's structure, check constraints), some is better after the data (indexes, unique constraints, foreign keys…), for performance reasons mostly.<br />
<br />
So there are now a few more options:<br />
<br />
* --section=pre-data: dump what's needed before restoring the data. Of course, this can be combined with a -t for instance, to specify one table<br />
* --section=post-data : dump what's needed after restoring the data.<br />
* --section=data: dump the data<br />
* --exclude-table-data: dump everything, except THIS table's data. It means pg_dump will still dump other tables' data.<br />
<br />
<br />
<br />
<br />
<br />
[[Category:PostgreSQL 9.2]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Timestamp_Average&diff=17908Timestamp Average2012-07-13T01:19:21Z<p>Schmiddy: add category tag</p>
<hr />
<div>{{SnippetInfo|Timestamp Average|version=9.1|lang=PL/pgSQL}}<br />
<br />
Here is the code to efficiently compute an average of a timestamp column. I've only tested this on 9.1, but it will probably work on earlier versions as well. Note, you'll need to cast the column to a plain timestamp (e.g. SELECT avg(tstz_col AT TIME ZONE 'UTC') FROM mytable) in order to use it with columns of type 'timestamp with time zone'. <br />
<br />
Author: Josh Kupershmidt<br />
<br />
<source lang="plsql"><br />
-- In order to have a reasonably efficient accumulator<br />
-- function, we need a state variable keeping a running<br />
-- total of seconds since the epoch, along with the number<br />
-- of elements processed already.<br />
CREATE TYPE ts_accum_typ AS (<br />
running_total numeric,<br />
num_elems bigint<br />
);<br />
<br />
-- Accumulator function. Keep a running total of the<br />
-- number of seconds since the epoch (1970-01-01), as well<br />
-- as the number of elements we have processed already.<br />
CREATE OR REPLACE FUNCTION ts_accum (existing ts_accum_typ, newval timestamp)<br />
RETURNS ts_accum_typ AS $$<br />
DECLARE<br />
retval ts_accum_typ;<br />
BEGIN<br />
<br />
IF newval IS NULL THEN<br />
RETURN existing;<br />
END IF;<br />
<br />
IF existing IS NULL THEN<br />
retval.running_total = EXTRACT(epoch FROM newval);<br />
retval.num_elems = 1;<br />
RETURN retval;<br />
ELSE<br />
existing.running_total = existing.running_total + EXTRACT(epoch FROM newval);<br />
existing.num_elems = existing.num_elems + 1;<br />
RETURN existing;<br />
END IF;<br />
END;<br />
$$<br />
LANGUAGE PLPGSQL IMMUTABLE;<br />
<br />
-- Final function for the timestamp 'avg' aggregate.<br />
CREATE OR REPLACE FUNCTION ts_avg (existing ts_accum_typ) RETURNS timestamp AS $$<br />
DECLARE<br />
since_epoch numeric;<br />
BEGIN<br />
-- Handle the case when avg() is called with no rows: answer should be NULL.<br />
IF existing IS NULL THEN<br />
RETURN NULL;<br />
END IF;<br />
<br />
since_epoch = existing.running_total / existing.num_elems;<br />
RETURN to_timestamp(since_epoch);<br />
END;<br />
$$<br />
LANGUAGE PLPGSQL IMMUTABLE;<br />
<br />
CREATE AGGREGATE avg (timestamp)<br />
(<br />
sfunc = ts_accum,<br />
stype = ts_accum_typ,<br />
finalfunc = ts_avg<br />
);<br />
<br />
</source><br />
<br />
[[Category:PL/pgSQL]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Timestamp_Average&diff=17907Timestamp Average2012-07-13T01:17:11Z<p>Schmiddy: initial code for timestamp average</p>
<hr />
<div>{{SnippetInfo|Timestamp Average|version=9.1|lang=PL/pgSQL}}<br />
<br />
Here is the code to efficiently compute an average of a timestamp column. I've only tested this on 9.1, but it will probably work on earlier versions as well. Note, you'll need to cast the column to a plain timestamp (e.g. SELECT avg(tstz_col AT TIME ZONE 'UTC') FROM mytable) in order to use it with columns of type 'timestamp with time zone'. <br />
<br />
Author: Josh Kupershmidt<br />
<br />
<source lang="plsql"><br />
-- In order to have a reasonably efficient accumulator<br />
-- function, we need a state variable keeping a running<br />
-- total of seconds since the epoch, along with the number<br />
-- of elements processed already.<br />
CREATE TYPE ts_accum_typ AS (<br />
running_total numeric,<br />
num_elems bigint<br />
);<br />
<br />
-- Accumulator function. Keep a running total of the<br />
-- number of seconds since the epoch (1970-01-01), as well<br />
-- as the number of elements we have processed already.<br />
CREATE OR REPLACE FUNCTION ts_accum (existing ts_accum_typ, newval timestamp)<br />
RETURNS ts_accum_typ AS $$<br />
DECLARE<br />
retval ts_accum_typ;<br />
BEGIN<br />
<br />
IF newval IS NULL THEN<br />
RETURN existing;<br />
END IF;<br />
<br />
IF existing IS NULL THEN<br />
retval.running_total = EXTRACT(epoch FROM newval);<br />
retval.num_elems = 1;<br />
RETURN retval;<br />
ELSE<br />
existing.running_total = existing.running_total + EXTRACT(epoch FROM newval);<br />
existing.num_elems = existing.num_elems + 1;<br />
RETURN existing;<br />
END IF;<br />
END;<br />
$$<br />
LANGUAGE PLPGSQL IMMUTABLE;<br />
<br />
-- Final function for the timestamp 'avg' aggregate.<br />
CREATE OR REPLACE FUNCTION ts_avg (existing ts_accum_typ) RETURNS timestamp AS $$<br />
DECLARE<br />
since_epoch numeric;<br />
BEGIN<br />
-- Handle the case when avg() is called with no rows: answer should be NULL.<br />
IF existing IS NULL THEN<br />
RETURN NULL;<br />
END IF;<br />
<br />
since_epoch = existing.running_total / existing.num_elems;<br />
RETURN to_timestamp(since_epoch);<br />
END;<br />
$$<br />
LANGUAGE PLPGSQL IMMUTABLE;<br />
<br />
CREATE AGGREGATE avg (timestamp)<br />
(<br />
sfunc = ts_accum,<br />
stype = ts_accum_typ,<br />
finalfunc = ts_avg<br />
);<br />
<br />
</source></div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Show_database_bloat&diff=17888Show database bloat2012-07-06T18:27:16Z<p>Schmiddy: summary of a few shortcomings of the query</p>
<hr />
<div>{{SnippetInfo|Show database bloat|version=>=8.0|lang=SQL|category=Performance}}<br />
<br />
This snippet displays the estimated amount of bloat in your tables and indices. Snippet is taken from Greg Sabino Mullane's excellent [http://bucardo.org/check_postgres/ check_postgres] script.<br />
<br />
<source lang="sql"><br />
SELECT<br />
current_database(), schemaname, tablename, /*reltuples::bigint, relpages::bigint, otta,*/<br />
ROUND(CASE WHEN otta=0 THEN 0.0 ELSE sml.relpages/otta::numeric END,1) AS tbloat,<br />
CASE WHEN relpages < otta THEN 0 ELSE bs*(sml.relpages-otta)::bigint END AS wastedbytes,<br />
iname, /*ituples::bigint, ipages::bigint, iotta,*/<br />
ROUND(CASE WHEN iotta=0 OR ipages=0 THEN 0.0 ELSE ipages/iotta::numeric END,1) AS ibloat,<br />
CASE WHEN ipages < iotta THEN 0 ELSE bs*(ipages-iotta) END AS wastedibytes<br />
FROM (<br />
SELECT<br />
schemaname, tablename, cc.reltuples, cc.relpages, bs,<br />
CEIL((cc.reltuples*((datahdr+ma-<br />
(CASE WHEN datahdr%ma=0 THEN ma ELSE datahdr%ma END))+nullhdr2+4))/(bs-20::float)) AS otta,<br />
COALESCE(c2.relname,'?') AS iname, COALESCE(c2.reltuples,0) AS ituples, COALESCE(c2.relpages,0) AS ipages,<br />
COALESCE(CEIL((c2.reltuples*(datahdr-12))/(bs-20::float)),0) AS iotta -- very rough approximation, assumes all cols<br />
FROM (<br />
SELECT<br />
ma,bs,schemaname,tablename,<br />
(datawidth+(hdr+ma-(case when hdr%ma=0 THEN ma ELSE hdr%ma END)))::numeric AS datahdr,<br />
(maxfracsum*(nullhdr+ma-(case when nullhdr%ma=0 THEN ma ELSE nullhdr%ma END))) AS nullhdr2<br />
FROM (<br />
SELECT<br />
schemaname, tablename, hdr, ma, bs,<br />
SUM((1-null_frac)*avg_width) AS datawidth,<br />
MAX(null_frac) AS maxfracsum,<br />
hdr+(<br />
SELECT 1+count(*)/8<br />
FROM pg_stats s2<br />
WHERE null_frac<>0 AND s2.schemaname = s.schemaname AND s2.tablename = s.tablename<br />
) AS nullhdr<br />
FROM pg_stats s, (<br />
SELECT<br />
(SELECT current_setting('block_size')::numeric) AS bs,<br />
CASE WHEN substring(v,12,3) IN ('8.0','8.1','8.2') THEN 27 ELSE 23 END AS hdr,<br />
CASE WHEN v ~ 'mingw32' THEN 8 ELSE 4 END AS ma<br />
FROM (SELECT version() AS v) AS foo<br />
) AS constants<br />
GROUP BY 1,2,3,4,5<br />
) AS foo<br />
) AS rs<br />
JOIN pg_class cc ON cc.relname = rs.tablename<br />
JOIN pg_namespace nn ON cc.relnamespace = nn.oid AND nn.nspname = rs.schemaname AND nn.nspname <> 'information_schema'<br />
LEFT JOIN pg_index i ON indrelid = cc.oid<br />
LEFT JOIN pg_class c2 ON c2.oid = i.indexrelid<br />
) AS sml<br />
ORDER BY wastedbytes DESC<br />
</source><br />
<br />
== Notes ==<br />
<br />
This query is for informational purposes only. It provides a [http://archives.postgresql.org/pgsql-admin/2011-06/msg00015.php loose estimate] of table growth activity only, and should not be construed as a 100% accurate portrayal of space consumed by database objects. To obtain more accurate information about database bloat, please refer to the [http://www.postgresql.org/docs/9.1/static/pgstattuple.html pgstattuple] or [http://www.postgresql.org/docs/9.1/static/pgfreespacemap.html pg_freespacemap] contrib modules.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=What%27s_new_in_PostgreSQL_9.0&diff=16808What's new in PostgreSQL 9.02012-05-09T20:43:44Z<p>Schmiddy: /* Use of index to get better statistics on the fly */ I found the 'DO' echoed at the end really confusing</p>
<hr />
<div>This document showcases many of the latest developments in PostgreSQL 9.0, compared to the last major release &ndash; PostgreSQL 8.4. There are more than 200 improvements in this release, so this wiki page covers many of the more important changes in detail. The full list of changes is itemised in [http://www.postgresql.org/docs/9.0/static/release-9-0 Release Notes].<br />
<br />
=The two features you can't ignore=<br />
<br />
Hot Standby and Streaming Replication are the two new features that mark Version 9.0 as a landmark in PostgreSQL's development and the motivation for allocating a full version number to this release &ndash; 9.0 (instead of 8.5). Combined, these two features add built-in, "binary replication" to PostgreSQL.<br />
<br />
There is further documention on how to use [[Hot Standby]] on its page, and an extensive [[Binary Replication Tutorial]] is in progress.<br />
<br />
===Hot Standby===<br />
<br />
This feature allows users to create a 'Standby' database &ndash; that is, a second database instance (normally on a separate server) replaying the primary's binary log, while making that standby server available for read-only queries. It is similar to the standby database features of proprietary databases, such as Oracle's Active DataGuard. Queries execute normally while the standby database continuously replays the stream of binary modifications coming from the primary database. Visibility of new data changes follows the MVCC model, so that new changes do not lock out queries.<br />
<br />
Enabling Hot Standby is a simple process. On the primary database server add this to the <tt>postgresql.conf</tt> file:<br />
wal_level = 'hot_standby' # Adds the required data in the WAL logs<br />
<br />
And on the standby server, add this to its <tt>postgresql.conf</tt> file:<br />
hot_standby = on<br />
<br />
Hot Standby works well with the new Streaming Replication feature, though it can also be used with file-based log shipping as available in previous versions and also to create standalone copies that receive no updates at all.<br />
<br />
In some cases, changes from the primary database can conflict with queries on the standby. A simple example is when DROP TABLE executes on the master, but the standby is still executing a query against that table. The standby cannot process that DROP statement without first canceling the running query, and the longer it delays doing that the further behind current replication the standby will become. The two options here are to pause the replay or cancel the read-only queries and move forward. <br />
<br />
A variety of parameters allow adjusting the conflict resolution mechanism used.<br />
<br />
max_standby_archive_delay = 30s # -1= always wait, 0= never wait, else wait for this<br />
max_standby_streaming_delay = 30s # -1= always wait, 0= never wait, else wait for this<br />
<br />
The two max_standby_{archive,streaming}_delay settings determine the behaviour of the standby database when conflicts between replay and read-only queries occur. In this situation, the standby database will wait at most until it's lagging behind the primary database by that amount before canceling the conflicting read-only queries. The two parameters allow different lag time tolerance levels for files appearing via regular file archive shipping vs. ones that are streamed via the new 9.0 feature for streaming replication.<br />
<br />
On the master it is also possible to avoid conflicts by increasing this parameter<br />
<br />
vacuum_defer_cleanup_age = 10000 # Adjust updwards slowly to reduce conflicts<br />
<br />
This feature is rich and complex, so it's advisable to read the documentation before planning your server deployments.<br />
<br />
===Streaming Replication===<br />
<br />
Complementing Hot Standby, Streaming Replication is the second half of the "great leap forward" for PostgreSQL. While there are several third-party replication solutions available for PostgreSQL that meet a range of specific needs, this new release brings a simple, sturdy and integrated version that will probably be used as a default in most High Availability installations using PostgreSQL.<br />
<br />
This time, the goal is improving the archiving mechanism to make it as continuous as possible and to not rely on log file shipping. Standby servers can now connect to the master/primary and get sent, whenever they want, what they are missing from the Write Ahead Log, not in terms of complete files ('wal segments'), but in terms of records in the WAL (you can think of them as fragments of these files).<br />
<br />
Streaming Replication is an asynchronous mechanism; the standby server lags behind the master. But unlike other replication methods, this lag is very short, and can be as little as a single transaction, depending on network speed, database activity, and Hot Standby settings. Also, the load on the master for each slave is minimal, allowing a single master to support dozens of slaves.<br />
<br />
Primary and standby databases are identical at the binary level (well, almost; but don't worry if your datafiles don't have the same checksum).<br />
<br />
For Streaming Replication, wal_level should be 'archive' or 'hot standby'.<br />
<br />
<tt>postgresql.conf</tt>, Primary:<br />
max_wal_senders = 5 # Maximum 'wal_senders', processes responsible for managing a connection with a standby server<br />
wal_keep_segments # How many WAL segments (=files) should be kept on the primary, whatever may happen (you won't have to copy them manually on the standby if the standby gets too far behind)<br />
<br />
On the standby server:<br />
<br />
<tt>recovery.conf</tt>, Standby:<br />
standby_mode = 'on'<br />
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass' # connection string to reach the primary database<br />
<tt>postgresql.conf</tt>, Secondary:<br />
wal_level # same value as on the primary (you'll need this after a failover, to build a new standby)<br />
hot_standby=on/off # Do you want to use Hot Standby at the same time ?<br />
pg_hba.conf file:<br />
<br />
There must be an entry here for the replication connections. The fake database is 'replication', the designated user should be superuser. Be careful not to give broad access to this account: a lot of privileged data can be extracted from WAL records.<br />
<br />
<tt>pg_hba.conf</tt>, Primary:<br />
host replication foo 192.168.1.100/32 md5<br />
As for Hot Standby, this feature is rich and complex. It's advised to read the documentation. And to perform failover and switchover tests when everything is in place.<br />
<br />
One thing should be stressed about these two features: you can use them together. This means you can have a near-realtime standby database, and run read-only queries on it, such as reporting queries. You can also use them independently; a standby database can be Hot Standby with file shipping only, and a Streaming Replication database can stream without accepting queries.<br />
<br />
=Other New features=<br />
<br />
There are literally hundreds of improvements, updates, and new features in 9.0 ... enough to make it a major release even without binary replication. We'll tour a few of them below, by category, with details on how to use them.<br />
<br />
==Security and Authentication==<br />
<br />
Of course, as the most secure SQL database (according to the SQL Hacker's Handbook) we're always eager to improve our data security. 9.0 adds several new features in this realm.<br />
<br />
===GRANT/REVOKE IN SCHEMA===<br />
<br />
One annoying limitation in PostgreSQL has been the lack of global GRANT/REVOKE capabilities. With 9.0 it's now possible to set privileges on all tables, sequences and functions within a schema using without having to write a script or a stored procedure:<br />
<br />
GRANT SELECT ON ALL TABLES IN SCHEMA public TO toto;<br />
And reverting this:<br />
REVOKE SELECT ON ALL TABLES IN SCHEMA public FROM toto;<br />
<br />
See the [http://www.postgresql.org/docs/9.0/static/sql-grant.html GRANT] documentation page for further details.<br />
<br />
Note that the above only works for existing objects. However, it's now also possible to define default permissions for new objects:<br />
<br />
===ALTER DEFAULT PRIVILEGES===<br />
<br />
This feature also makes permission management more efficient.<br />
ALTER DEFAULT PRIVILEGES FOR ROLE marc GRANT SELECT ON TABLES TO public;<br />
CREATE TABLE test_priv (a int);<br />
\z test_priv<br />
Access privileges<br />
Schema | Name | Type | Access privileges | Column access privileges<br />
--------+------------+-------+-------------------+--------------------------<br />
public | test_priv | table | =r/marc +|<br />
| | | marc=arwdDxt/marc |<br />
<br />
This new information is stored in the pg_default_acl system table.<br />
<br />
===passwordcheck===<br />
<br />
This contrib module can check passwords, and prevent the worst of them from getting in. After having it installed and set up as described in the documentation, here is the result:<br />
marc=# ALTER USER marc password 'marc12';<br />
&lt;marc%marc&gt; ERROR: password is too short<br />
&lt;marc%marc&gt; STATEMENT: ALTER USER marc password 'marc12';<br />
ERROR: password is too short<br />
marc=# ALTER USER marc password 'marc123456';<br />
&lt;marc%marc&gt; ERROR: password must not contain user name<br />
&lt;marc%marc&gt; STATEMENT: ALTER USER marc password 'marc123456';<br />
ERROR: password must not contain user name<br />
This module has limitations, mostly due to PostgreSQL accepting already encrypted passwords to be declared, making correct verification impossible.<br />
<br />
Moreover, its code is well documented, and can be easily adapted to suit specific needs (one can activate cracklib very easily, for instance)<br />
<br />
<br />
<br />
==SQL Features==<br />
<br />
SQL03 has a huge array of functionality, more than any one DBMS currently implements. But we keep adding SQL features, as well as extending SQL in a compatible way with various little things to make writing queries easier and more powerful.<br />
<br />
===Column triggers===<br />
<br />
Column triggers fire only when a specific column is explicitly UPDATED. They allow you to avoid adding lots of conditional logic and value comparisons in your trigger code.<br />
<br />
Example:<br />
CREATE TRIGGER foo BEFORE UPDATE OF some_column ON table1 FOR EACH ROW EXECUTE PROCEDURE my_trigger();<br />
This trigger fires only when '<tt>some_column</tt>' column of '<tt>table1</tt>' table has been updated.<br />
<br />
Column triggers are not executed if columns are set to DEFAULT.<br />
<br />
===WHEN Triggers===<br />
<br />
Completing PostgreSQL's effort to limit IF ... THEN code in triggers, conditional triggers define simple conditions under which the trigger will be executed. This can dramatically decrease the number of trigger executions and reduce CPU load on the database server.<br />
<br />
For example, this trigger would check that an account was correctly balanced only when the balance changes:<br />
<br />
CREATE TRIGGER check_update<br />
BEFORE UPDATE ON accounts<br />
FOR EACH ROW<br />
WHEN (OLD.balance IS DISTINCT FROM NEW.balance)<br />
EXECUTE PROCEDURE check_account_update();<br />
<br />
And this trigger will only log a row update when the row actually changes. It's very helpful with framework or ORM applications, which may attempt to save unchanged rows:<br />
<br />
CREATE TRIGGER log_update<br />
AFTER UPDATE ON accounts<br />
FOR EACH ROW<br />
WHEN (OLD.* IS DISTINCT FROM NEW.*)<br />
EXECUTE PROCEDURE log_account_update();<br />
<br />
You could even go further than this and decide not to save a row at all if it hasn't changed:<br />
<br />
CREATE TRIGGER log_update<br />
BEFORE UPDATE ON accounts<br />
FOR EACH ROW<br />
WHEN (OLD.* IS NOT DISTINCT FROM NEW.*)<br />
EXECUTE PROCEDURE no_op();<br />
<br />
===DEFERRABLE UNIQUE CONSTRAINTS===<br />
<br />
This feature will also be very useful. Here is an example, using a primary key instead of a simple unique key:<br />
marc=# CREATE TABLE test (a int primary key);<br />
marc=# INSERT INTO test values (1), (2);<br />
marc=# UPDATE test set a = a+1;<br />
ERROR: duplicate key value violates unique constraint "test_pkey"<br />
DETAIL: Key (a)=(2) already exists.<br />
That's a pity: at the end of the statement, my data would have been consistent, so far as this constraint is concerned. Even worse, if the table had been physically sorted by descending order, the query would have worked! With 8.4, there was no easy way out; we had to find a trick to update the records in the right order.<br />
<br />
We can now do this:<br />
marc=# CREATE TABLE test (a int primary key deferrable);<br />
marc=# INSERT INTO test values (1), (2);<br />
marc=# UPDATE test set a = a+1;<br />
UPDATE 2<br />
<br />
With a DEFERRABLE unique index, uniqueness is enforced as of the end of the statement, rather than after each row update as with a simple<br />
index. This is a bit slower sometimes but is a lifesaver if you need to do this sort of update.<br />
<br />
It is also possible to have the uniqueness check enforced as of the end of the transaction, rather than after each statement. This helps<br />
if you need to do "conflicting" updates that require more than one SQL statement to complete. For example:<br />
marc=# CREATE TABLE test (a int primary key deferrable, b text);<br />
marc=# INSERT INTO test values (1, 'x'), (2, 'y');<br />
marc=# BEGIN;<br />
marc=# SET CONSTRAINTS ALL DEFERRED;<br />
marc=# UPDATE test SET a = 2 WHERE b = 'x';<br />
marc=# UPDATE test SET a = 1 WHERE b = 'y';<br />
marc=# COMMIT;<br />
<br />
If one doesn't want to perform a SET CONSTRAINTS each time, the constraint can also be declared as INITIALLY DEFERRED:<br />
CREATE TABLE test (a int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED);<br />
<br />
Keep in mind that the list of records to be checked at the end of the statement or transaction has to be stored somewhere. So be careful of not doing this for millions of records at once. This is one of the reasons that unique indexes aren't DEFERRABLE by default, even though a strict reading of the SQL spec would require it.<br />
<br />
===New frame options for window functions===<br />
<br />
If you don't know window functions yet, you'd better learn about them. You can start here : [http://www.depesz.com/index.php/2009/01/21/waiting-for-84-window-functions waiting-for-84-window-functions]. They make writing certain kind of queries much easier.<br />
<br />
New options have been added for declaring frames of windowing functions. Let's use this table (not having a better example…)<br />
marc=# SELECT * FROM salary ;<br />
entity | name | salary | start_date<br />
-----------+-----------+---------+---------------<br />
R&amp;D | marc | 700.00 | 2010-02-15<br />
Accounting | jack | 800.00 | 2010-05-01<br />
R&amp;D | maria | 700.00 | 2009-01-01<br />
R&amp;D | kevin | 500.00 | 2009-05-01<br />
R&amp;D | john | 1000.00 | 2008-07-01<br />
R&amp;D | tom | 1100.00 | 2005-01-01<br />
Accounting | millicent | 850.00 | 2006-01-01<br />
Here is a window function example, without declaring the frame:<br />
marc=# SELECT entity, name, salary, start_date,<br />
avg(salary) OVER (PARTITION BY entity ORDER BY start_date)<br />
FROM salary;<br />
<br />
entity | name | salary | start_date | avg <br />
-----------+-----------+---------+---------------+-----------------------<br />
Accounting | millicent | 850.00 | 2006-01-01 | 850.0000000000000000<br />
Accounting | jack | 800.00 | 2010-05-01 | 825.0000000000000000<br />
R&amp;D | tom | 1100.00 | 2005-01-01 | 1100.0000000000000000<br />
R&amp;D | john | 1000.00 | 2008-07-01 | 1050.0000000000000000<br />
R&amp;D | maria | 700.00 | 2009-01-01 | 933.3333333333333333<br />
R&amp;D | kevin | 500.00 | 2009-05-01 | 825.0000000000000000<br />
R&amp;D | marc | 700.00 | 2010-02-15 | 800.0000000000000000<br />
The frame is the group of records over which the window function is run. Of course, if the frame isn't explicitly declared, there is a default one.<br />
<br />
Here is the same query, with an explicit frame:<br />
marc=# SELECT entity, name, salary, start_date,<br />
avg(salary) OVER (PARTITION BY entity ORDER BY start_date<br />
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)<br />
FROM salary;<br />
<br />
entity | name | salary | start_date | avg <br />
-----------+-----------+---------+---------------+-----------------------<br />
Accounting | millicent | 850.00 | 2006-01-01 | 850.0000000000000000<br />
Accounting | jack | 800.00 | 2010-05-01 | 825.0000000000000000<br />
R&amp;D | tom | 1100.00 | 2005-01-01 | 1100.0000000000000000<br />
R&amp;D | john | 1000.00 | 2008-07-01 | 1050.0000000000000000<br />
R&amp;D | maria | 700.00 | 2009-01-01 | 933.3333333333333333<br />
R&amp;D | kevin | 500.00 | 2009-05-01 | 825.0000000000000000<br />
R&amp;D | marc | 700.00 | 2010-02-15 | 800.0000000000000000<br />
In this example, the frame is a 'range' frame, between the start of the partition (the group of similar rows) and the current row (not exactly the current row, but let's put that aside for now, read the documentation if you want to learn more). One can see, the average (avg) function is evaluated from the frame's first row (grouped together records) and the current row.<br />
<br />
First new feature: as of 9.0, the frame can be declared to be between the current row and the end of the partition:<br />
marc=# SELECT entity, name, salary, start_date,<br />
avg(salary) OVER (PARTITION BY entity ORDER BY start_date<br />
RANGE BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)<br />
FROM salary;<br />
<br />
entity | name | salary | start_date | avg <br />
-----------+-----------+---------+---------------+----------------------<br />
Accounting | millicent | 850.00 | 2006-01-01 | 825.0000000000000000<br />
Accounting | jack | 800.00 | 2010-05-01 | 800.0000000000000000<br />
R&amp;D | tom | 1100.00 | 2005-01-01 | 800.0000000000000000<br />
R&amp;D | john | 1000.00 | 2008-07-01 | 725.0000000000000000<br />
R&amp;D | maria | 700.00 | 2009-01-01 | 633.3333333333333333<br />
R&amp;D | kevin | 500.00 | 2009-05-01 | 600.0000000000000000<br />
R&amp;D | marc | 700.00 | 2010-02-15 | 700.0000000000000000<br />
Second new feature: frames can be declared as 'x previous records to y next records'. There is no point with this example, but let's do it anyway::<br />
marc=# SELECT entity, name, salary, start_date,<br />
avg(salary) OVER (PARTITION BY entity ORDER BY start_date<br />
ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)<br />
FROM salary;<br />
<br />
entity | name | salary | start_date | avg <br />
-----------+-----------+---------+---------------+-----------------------<br />
Accounting | millicent | 850.00 | 2006-01-01 | 825.0000000000000000<br />
Accounting | jack | 800.00 | 2010-05-01 | 825.0000000000000000<br />
R&amp;D | tom | 1100.00 | 2005-01-01 | 1050.0000000000000000<br />
R&amp;D | john | 1000.00 | 2008-07-01 | 933.3333333333333333<br />
R&amp;D | maria | 700.00 | 2009-01-01 | 733.3333333333333333<br />
R&amp;D | kevin | 500.00 | 2009-05-01 | 633.3333333333333333<br />
R&amp;D | marc | 700.00 | 2010-02-15 | 600.0000000000000000<br />
The frame is still limited to the partition (see tom's record, for instance: jack's record isn't use for it's average).<br />
<br />
If one wanted the same query, with a moving average on three rows, not reset on each partition switch (still no practical use):<br />
marc=# SELECT entity, name, salary, start_date,<br />
avg(salary) OVER (ORDER BY entity, start_date<br />
ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)<br />
FROM salary;<br />
<br />
entity | name | salary | start_date | avg <br />
-----------+-----------+---------+---------------+----------------------<br />
Accounting | millicent | 850.00 | 2006-01-01 | 825.0000000000000000<br />
Accounting | jack | 800.00 | 2010-05-01 | 916.6666666666666667<br />
R&amp;D | tom | 1100.00 | 2005-01-01 | 966.6666666666666667<br />
R&amp;D | john | 1000.00 | 2008-07-01 | 933.3333333333333333<br />
R&amp;D | maria | 700.00 | 2009-01-01 | 733.3333333333333333<br />
R&amp;D | kevin | 500.00 | 2009-05-01 | 633.3333333333333333<br />
R&amp;D | marc | 700.00 | 2010-02-15 | 600.0000000000000000<br />
<br />
In short, a powerful tool to be mastered, even if I couldn't provide a good example.<br />
<br />
===Sort in aggregates===<br />
<br />
This feature is a subtle one: the result of an aggregate function may depend on the order it receives the data.<br />
<br />
Of course, we're not talking about count, avg, but of array_agg, string_agg…<br />
<br />
This is nice, as this will showcase string_agg, which is another 9.0 feature, killing two birds with one stone.<br />
<br />
Let's start again with our salary table. We want the list of employees, concatenated as a single value, grouped by entity. It's going into a spreadsheet…<br />
marc=# SELECT entity,string_agg(name,', ') FROM salary GROUP BY entity;<br />
entity | string_agg <br />
-----------+-------------------------------<br />
Accounting | stephanie, etienne<br />
R&amp;D | marc, maria, kevin, john, tom<br />
That's already nice. But I want them sorted in alphabetical order, because I don't know how to write a macro in my spreadsheet to sort this data.<br />
marc=# SELECT entity,string_agg(name,', ' ORDER BY name) FROM salary GROUP BY entity;<br />
entity | string_agg <br />
-----------+-------------------------------<br />
Accounting | etienne, stephanie<br />
R&amp;D | john, kevin, marc, maria, tom<br />
To use this new feature, the sort clause must be inserted inside the aggregate function, without a comma to separate it from the parameters.<br />
<br />
==Database Administration==<br />
<br />
DBA is a hard and often thankless job -- especially if that's not your job title. 9.0 includes new and improved features to make that job a bit easier.<br />
<br />
===Better VACUUM FULL===<br />
<br />
Until now, VACUUM FULL was very slow. This statement can recover free space from a table to reduce its size, mostly when VACUUM itself hasn't been run frequently enough.<br />
<br />
It was slow because of the way it operated: records were read and moved one by one from their source bloc to a bloc closer to the beginning of the table. Once the end of the table was emptied, this empty part was removed.<br />
<br />
This strategy was very inefficient: moving records one by one creates a lot of random IO. Moreover, during this reorganization, indexes had to be maintained, making everything even more costly, and fragmenting indexes. It was therefore advised to reindex a table just after a VACUUM FULL.<br />
<br />
The VACUUM FULL statement, as of version 9.0, creates a new table from the current one, copying all the records sequentially. Once all records are copied, index are created back, and the old table is destroyed and replaced.<br />
<br />
This has the advantage of being much faster. VACUUM FULL still needs an AccessExclusiveLock while running though. The only drawback of this method compared to the old one, is that VACUUM FULL can use as much as two times the size of the table on disk, as it is creating a new version of it.<br />
<br />
Let's now compare the runtimes of the two methods. In both cases, we prepare the test data as follows (for 8.4 and 9.0)<br />
marc=# CREATE TABLE test (a int);<br />
CREATE TABLE<br />
marc=# CREATE INDEX idxtsta on test (a);<br />
CREATE INDEX<br />
marc=# INSERT INTO test SELECT generate_series(1,1000000);<br />
INSERT 0 1000000<br />
marc=# DELETE FROM test where a%3=0; -- making holes everywhere<br />
DELETE 333333<br />
marc=# VACUUM test;<br />
VACUUM<br />
With 8.4:<br />
marc=# \timing<br />
Timing is on.<br />
marc=# VACUUM FULL test;<br />
VACUUM<br />
Time: 6306,603 ms<br />
marc=# REINDEX TABLE test;<br />
REINDEX<br />
Time: 1799,998 ms<br />
So around 8 seconds.<br />
With 9.0:<br />
marc=# \timing<br />
Timing is on.<br />
marc=# VACUUM FULL test;<br />
VACUUM<br />
Time: 2563,467 ms<br />
That still doesn't mean that VACUUM FULL is a good idea in production. If you need it, it's probably because your VACUUM policy isn't appropriate.<br />
<br />
===application_name in pg_stat_activity===<br />
<br />
In a monitoring session:<br />
marc=# SELECT * from pg_stat_activity where procpid= 5991;<br />
datid | datname | procpid | usesysid | usename | application_name | client_addr | client_port | backend_start | xact_start | query_start | waiting | current_query<br />
------+---------+---------+----------+---------+------------------+-------------+-------------+-------------------------------+------------+-------------+---------+----------------<br />
16384 | marc | 5991 | 10 | marc | psql | | -1 | 2010-05-16 13:48:10.154113+02 | | | f | &lt;IDLE&gt;<br />
(1 row)<br />
In the '5991' session:<br />
marc=# SET application_name TO 'my_app';<br />
SET<br />
Back to the monitoring session:<br />
>marc=# SELECT * from pg_stat_activity where procpid= 5991;<br />
datid | datname | procpid | usesysid | usename | application_name | client_addr | client_port | backend_start | xact_start | query_start | waiting | current_query<br />
------+---------+---------+----------+---------+------------------+-------------+-------------+-------------------------------+------------+-------------+---------+-----------------+----------------<br />
16384 | marc | 5991 | 10 | marc | my_app | | -1 | 2010-05-16 13:48:10.154113+02 | | 2010-05-16 13:49:13.107413+02 | f | &lt;IDLE&gt;<br />
(1 row)<br />
It's your job to set this up correctly in your program or your sessions. Your DBA will thank you for this, at last knowing who runs what on the database easily.<br />
<br />
===Per database+role configuration===<br />
Instead of being able to set up configuration variables per database or per user, one can now set them up for a certain user in a certain database:<br />
<br />
marc=# ALTER ROLE marc IN database marc set log_statement to 'all';<br />
ALTER ROLE<br />
To know who has which variables set-up in which user+database, there is a new psql command:<br />
marc=# \drds<br />
List of settings<br />
role | database | settings<br />
-----+----------+-----------------<br />
marc | marc | log_statement=all<br />
(1 row)<br />
There was a catalog change to store this:<br />
Table "pg_catalog.pg_db_role_setting"<br />
Column | Type | Modifier<br />
------------+--------+----------<br />
setdatabase | oid | not null<br />
setrole | oid | not null<br />
setconfig | text |<br />
<br />
===Log all changed parameters on a postgresql.conf reload===<br />
<br />
Here is an example, the log_line_prefix parameter has been changed:<br />
LOG:&nbsp; received SIGHUP, reloading configuration files<br />
&lt;%&gt; LOG:&nbsp; parameter "log_line_prefix" changed to "&lt;%u%%%d&gt; "<br />
<br />
===Better unique constraints error messages===<br />
<br />
With 8.4: <br />
marc=# INSERT INTO test VALUES (1);<br />
ERROR: duplicate key value violates unique constraint "test_a_key"<br />
With 9.0:<br />
marc=# INSERT INTO test VALUES (1);<br />
ERROR: duplicate key value violates unique constraint "test_a_key"<br />
DETAIL: Key (a)=(1) already exists.<br />
This will make diagnosing constraint violation errors much easier.<br />
<br />
===vacuumdb --analyze-only===<br />
<br />
As the parameter indicates, one can now use vacuumdb to run analyze only. It may be useful for cronjobs for instance.<br />
<br />
<br />
<br />
==Performance==<br />
<br />
It wouldn't be a new version of PostgreSQL if it didn't get faster, now would it? While 9.0 is not a "performance release", it does add new features which make some specific operations up to 1000% faster.<br />
<br />
===64 bit binaries for Windows===<br />
<br />
It is now possible to compile PostgreSQL for Windows as a 64-bit binary, and the PostgreSQL project is releasing 64-bit packages.<br />
<br />
This has a number of advantages for Windows users: better performance on 64-bit number operations (like BIGINT and BIGSERIAL), the ability to use over 2GB of work_mem, and enhanced compatibility with 64-bit versions of PostgreSQL running on Linux. This last is particularly important given Hot Standby.<br />
<br />
Note, however, that there is no evidence for now the 500MB shared_buffers size limit before performance degrades seen on the 32 bits version for Windows is solved with this 64 bit version, though. There is also the limitation that many 3rd-party open-source libraries are not available in 64-bit for Windows, so you may not be able to add all PostgreSQL extensions. Test reports welcome!<br />
<br />
===Join Removal===<br />
<br />
This new optimization allows us to remove unnecessary joins from SQL execution plans.<br />
<br />
When using automatically generated SQL, such as from ORM (Object Relation Mapping) tools it is possible for the SQL to be sub-optimal. Removing unnecessary joins can improve query plans by an order of magnitude in some cases.<br />
<br />
This is particularly important for databases that use many joins and nested views.<br />
<br />
marc=# CREATE TABLE t1 (a int);<br />
CREATE TABLE<br />
marc=# CREATE TABLE t2 (b int);<br />
CREATE TABLE<br />
marc=# CREATE TABLE t3 (c int);<br />
CREATE TABLE<br />
We put a little bit of data with a generate_series…<br />
<br />
marc=# EXPLAIN SELECT t1.a,t2.b from t1 join t2 on (t1.a=t2.b) left join t3 on (t1.a=t3.c);<br />
QUERY PLAN <br />
------------------------------------------------------------------------------<br />
Merge Right Join (cost=506.24..6146.24 rows=345600 width=8)<br />
Merge Cond: (t3.c = t1.a)<br />
-&gt; Sort (cost=168.75..174.75 rows=2400 width=4)<br />
Sort Key: t3.c<br />
-&gt; Seq Scan on t3 (cost=0.00..34.00 rows=2400 width=4)<br />
-&gt; Materialize (cost=337.49..853.49 rows=28800 width=8)<br />
-&gt; Merge Join (cost=337.49..781.49 rows=28800 width=8)<br />
Merge Cond: (t1.a = t2.b)<br />
-&gt; Sort (cost=168.75..174.75 rows=2400 width=4)<br />
Sort Key: t1.a<br />
-&gt; Seq Scan on t1 (cost=0.00..34.00 rows=2400 width=4)<br />
-&gt; Sort (cost=168.75..174.75 rows=2400 width=4)<br />
Sort Key: t2.b<br />
-&gt; Seq Scan on t2 (cost=0.00..34.00 rows=2400 width=4)<br />
<br />
For now, everything is normal, and we have the same behavior in 8.4. But let's imagine that on t3, there is a UNIQUE constraint on the 'c' column. In this case, the join on t3 doesn't serve any purpose, theoretically speaking: the number of rows returned won't change, neither will their content. It's because the column is UNIQUE, the join is a LEFT JOIN, and no column of t3 is retrieved. If the column wasn't UNIQUE, the join could bring more rows. If that wasn't a LEFT JOIN, the join could ignore some rows.<br />
<br />
With 9.0:<br />
marc=# ALTER TABLE t3 ADD UNIQUE (c);<br />
NOTICE: ALTER TABLE / ADD UNIQUE will create implicit index "t3_c_key" for table "t3"<br />
ALTER TABLE<br />
marc=# EXPLAIN SELECT t1.a,t2.b from t1 join t2 on (t1.a=t2.b) left join t3 on (t1.a=t3.c);<br />
QUERY PLAN <br />
------------------------------------------------------------------<br />
Merge Join (cost=337.49..781.49 rows=28800 width=8)<br />
Merge Cond: (t1.a = t2.b)<br />
-&gt; Sort (cost=168.75..174.75 rows=2400 width=4)<br />
Sort Key: t1.a<br />
-&gt; Seq Scan on t1 (cost=0.00..34.00 rows=2400 width=4)<br />
-&gt; Sort (cost=168.75..174.75 rows=2400 width=4)<br />
Sort Key: t2.b<br />
-&gt; Seq Scan on t2 (cost=0.00..34.00 rows=2400 width=4)<br />
(8 rows)<br />
<br />
===IS NOT NULL can now use indexes===<br />
<br />
For this demonstration, we will compare the 8.4 and 9.0 versions (the table I created contains mostly nulls):<br />
<br />
With 8.4:<br />
marc=# EXPLAIN ANALYZE SELECT max(a) from test;<br />
QUERY PLAN <br />
------------------------------------------------------------------------------------------------------------------------------------------------<br />
Result (cost=0.03..0.04 rows=1 width=0) (actual time=281.320..281.321 rows=1 loops=1)<br />
InitPlan 1 (returns $0)<br />
-&gt; Limit (cost=0.00..0.03 rows=1 width=4) (actual time=281.311..281.313 rows=1 loops=1)<br />
-&gt; Index Scan Backward using idxa on test (cost=0.00..29447.36 rows=1001000 width=4) (actual time=281.307..281.307 rows=1 loops=1)<br />
Filter: (a IS NOT NULL)<br />
Total runtime: 281.360 ms<br />
(6 rows)<br />
With 9.0:<br />
marc=# EXPLAIN ANALYZE SELECT max(a) from test;<br />
QUERY PLAN <br />
--------------------------------------------------------------------------------------------------------------------------------------------<br />
Result (cost=0.08..0.09 rows=1 width=0) (actual time=0.100..0.102 rows=1 loops=1)<br />
InitPlan 1 (returns $0)<br />
-&gt; Limit (cost=0.00..0.08 rows=1 width=4) (actual time=0.092..0.093 rows=1 loops=1)<br />
-&gt; Index Scan Backward using idxa on test (cost=0.00..84148.06 rows=1001164 width=4) (actual time=0.089..0.089 rows=1 loops=1)<br />
Index Cond: (a IS NOT NULL)<br />
Total runtime: 0.139 ms<br />
(6 rows)<br />
The difference is that 9.0 only scans the not-null keys in the index. 8.4 has to go check in the table (Filter step, when 9.0 uses an index condition). In this precise use case, the gain is really big.<br />
<br />
===Use of index to get better statistics on the fly===<br />
<br />
Before starting to explain this new feature, let's talk about histograms: PostgreSQL, like some other databases, uses a statistical optimizer. This means that when planning a query it has (or should have) an approximately correct idea of how many records each step of the query will bring back. In order to do this, it uses statistics, such as the approximate number of records in a table, its size, most common values, and histograms. PostgreSQL use these to get estimates about the number of records brought back by a WHERE clause on a column, depending on the value or range asked in this WHERE clause.<br />
<br />
In some cases, these histograms are rapidly out of date, and become a problem, for certain SQL queries. For instance, a log table in which timestamped records would be inserted, and from which we would most of the time want to get the records from the last 5 minutes.<br />
<br />
In this specific case, it was impossible before 9.0 to get correct statistics. Now, when PostgreSQL detects while planning that a query asks for a 'range scan' on a value larger than the largest of the histogram (or smaller than the smallest), that is, the largest detected value during the last statistics calculation, and this column has an index, it gets the max (or min) value for this column using the index BEFORE really executing the query, in order to get more realistic statistics. As PostgreSQL uses an index for this, there HAS to be an index, of course.<br />
<br />
Here comes an example. The a column of the test table has already been filled with a lot of dates, all in the past. It's statistics are up to date.<br />
<br />
It's 13:37, and I haven't inserted anything after 13:37 yet.<br />
marc=# EXPLAIN ANALYZE select * from test where a &gt; '2010-06-03 13:37:00';<br />
QUERY PLAN <br />
--------------------------------------------------------------------------------------------------------------<br />
Index Scan using idxtsta on test (cost=0.00..8.30 rows=1 width=8) (actual time=0.007..0.007 rows=0 loops=1)<br />
Index Cond: (a &gt; '2010-06-03 13:37:00'::timestamp without time zone)<br />
Total runtime: 0.027 ms<br />
(3 rows)<br />
Everything's normal. The upper boundary of the histogram is '2010-06-03 13:36:16.830007' (this information comes from pg_stats). There is no way of guessing how many records are larger than 13:37, and with 8.4, PostgreSQL would have continued estimating '1' until the next analyze.<br />
marc=# DO LANGUAGE plpgsql<br />
$$<br />
DECLARE<br />
i int;<br />
BEGIN<br />
FOR i IN 1..10000 LOOP<br />
INSERT INTO test VALUES (clock_timestamp());<br />
END LOOP;<br />
END<br />
$$<br />
;<br />
<br />
(I must say I really like 'DO').<br />
We just inserted 10000 records with a date larger than 13:37.<br />
marc=# EXPLAIN ANALYZE SELECT * FROM test WHERE a &gt; '2010-06-03 13:37:00';<br />
QUERY PLAN <br />
-----------------------------------------------------------------------------------------------------------------------<br />
Index Scan using idxtsta on test (cost=0.00..43.98 rows=1125 width=8) (actual time=0.012..13.590 rows=10000 loops=1)<br />
Index Cond: (a &gt; '2010-06-03 13:37:00'::timestamp without time zone)<br />
Total runtime: 23.567 ms<br />
(3 rows)<br />
The estimated rows isn't 0 or 1 anymore. The statistics haven't been updated, though:<br />
marc=# SELECT last_autoanalyze FROM pg_stat_user_tables WHERE relname = 'test';<br />
last_autoanalyze <br />
-------------------------------<br />
2010-06-03 13:36:21.553477+02<br />
(1 row)<br />
We still have a one magnitude error in the evaluation (10 times). But it's not that bad: without this enhancement, it would be of four magnitudes (10,000). Anyway, a much smaller error makes it more likely we'll get a good plan out of this kind of queries.<br />
<br />
===Explain buffers, hashing statistics, xml, json, yaml, new optional explain syntax===<br />
<br />
Here is EXPLAIN ANALYZE as we all know it:<br />
marc=# EXPLAIN ANALYZE SELECT a, sum(c) FROM pere JOIN fils ON (pere.a = fils.b) WHERE b BETWEEN 1000 AND 300000 GROUP BY a;<br />
QUERY PLAN <br />
---------------------------------------------------------------------------------------------------------------------------------<br />
HashAggregate (cost=905.48..905.86 rows=31 width=8) (actual time=0.444..0.453 rows=6 loops=1)<br />
-&gt; Nested Loop (cost=10.70..905.32 rows=31 width=8) (actual time=0.104..0.423 rows=6 loops=1)<br />
-&gt; Bitmap Heap Scan on fils (cost=10.70..295.78 rows=31 width=8) (actual time=0.040..0.154 rows=30 loops=1)<br />
Recheck Cond: ((b &gt;= 1000) AND (b &lt;= 300000))<br />
-&gt; Bitmap Index Scan on fils_pkey (cost=0.00..10.69 rows=31 width=0) (actual time=0.023..0.023 rows=30 loops=1)<br />
Index Cond: ((b &gt;= 1000) AND (b &lt;= 300000))<br />
-&gt; Index Scan using pere_pkey on pere (cost=0.00..19.65 rows=1 width=4) (actual time=0.005..0.005 rows=0 loops=30)<br />
Index Cond: (pere.a = fils.b)<br />
Total runtime: 0.560 ms<br />
(9 rows)<br />
To get access to the new available information, use the new syntax::<br />
<br />
EXPLAIN [ ( { ANALYZE boolean | VERBOSE boolean | COSTS boolean | BUFFERS boolean | FORMAT { TEXT | XML | JSON | YAML } } [, ...] ) ] instruction<br />
For instance:<br />
marc=# EXPLAIN (ANALYZE true, VERBOSE true, BUFFERS true) SELECT a, sum(c) FROM pere JOIN fils ON (pere.a = fils.b) WHERE b BETWEEN 1000 AND 300000 GROUP BY a;<br />
QUERY PLAN<br />
-------------------------------------------------------------------------------------------------------------------------------------<br />
HashAggregate (cost=905.48..905.86 rows=31 width=8) (actual time=1.326..1.336 rows=6 loops=1)<br />
Output: pere.a, sum(fils.c)<br />
Buffers: shared hit=58 read=40<br />
-&gt; Nested Loop (cost=10.70..905.32 rows=31 width=8) (actual time=0.278..1.288 rows=6 loops=1)<br />
Output: pere.a, fils.c<br />
Buffers: shared hit=58 read=40<br />
-&gt; Bitmap Heap Scan on public.fils (cost=10.70..295.78 rows=31 width=8) (actual time=0.073..0.737 rows=30 loops=1)<br />
Output: fils.b, fils.c<br />
Recheck Cond: ((fils.b &gt;= 1000) AND (fils.b &lt;= 300000))<br />
Buffers: shared hit=4 read=28<br />
-&gt; Bitmap Index Scan on fils_pkey (cost=0.00..10.69 rows=31 width=0) (actual time=0.030..0.030 rows=30 loops=1)<br />
Index Cond: ((fils.b &gt;= 1000) AND (fils.b &lt;= 300000))<br />
Buffers: shared hit=3<br />
-&gt; Index Scan using pere_pkey on public.pere (cost=0.00..19.65 rows=1 width=4) (actual time=0.013..0.014 rows=0 loops=30)<br />
Output: pere.a<br />
Index Cond: (pere.a = fils.b)<br />
Buffers: shared hit=54 read=12<br />
Total runtime: 1.526 ms<br />
(18 rows)<br />
VERBOSE displays the 'Output' lines (it already existed on 8.4).<br />
<br />
BUFFERS displays data about buffers (input-output operations performed by the query): hit is the number of blocks obtained directly from shared_buffers, read is the number of blocs asked to the operating system. Here, there was very little data in shared_buffers.<br />
<br />
One can also ask for another formatting than plain text. For a user, it's not useful. For people developing GUIs over EXPLAIN, it simplifies development as they can get rid of an 'explain' parser (and its potential bugs), and use a more standard one, such as XML.<br />
<br />
Costs display can also be deactivated with COSTS false.<br />
<br />
===Per tablespace seq_page_cost/random_page_cost===<br />
<br />
marc=# ALTER TABLESPACE pg_default SET ( random_page_cost = 10, seq_page_cost=5);<br />
ALTER TABLESPACE<br />
We just changed random_page_cost and seq_page_cost for all the objects contained in pg_default. What for ?<br />
<br />
The use case is when different tablespaces have different performance: for instance, you have some critical data on a SSD drive, or historical data on an older disk array, slower than the brand new array you use for active data. This makes it possible to tell PostgreSQL that all your tablespaces don't always behave the same way, from a performance point of view. This is only useful, of course, for quite big databases.<br />
<br />
===Force distinct statistics on a column===<br />
<br />
This makes it possible to set the number of different values for a column. This mustn't be used lightly, but only when ANALYZE on this column can't get a good value.<br />
<br />
Here's how to do this:<br />
marc=# ALTER TABLE test ALTER COLUMN a SET (n_distinct = 2);<br />
ALTER TABLE<br />
ANALYZE has to be run again for this to be taken into account:<br />
marc=# ANALYZE test;<br />
ANALYZE<br />
Let's try now:<br />
marc=# EXPLAIN SELECT distinct * from test;<br />
QUERY PLAN <br />
------------------------------------------------------------------<br />
HashAggregate (cost=6263.00..6263.02 rows=2 width=8)<br />
-&gt; Seq Scan on test (cost=0.00..5338.00 rows=370000 width=8)<br />
(2 rows)<br />
This is an example of what SHOULDN'T be done : there REALLY is 370 000 distinct values in my table. Now my execution plans may be very bad.<br />
<br />
If n_distinct is positive, it's the number of distinct values.<br />
<br />
If it's negative (between 0 and -1), it's the multiplying factor regarding the number of estimated records in the table: for instance, -0.2 means that there is a distinct value for each 5 records of the table.<br />
<br />
0 brings the behavior back to normal (ANALYZE estimates distinct by itself).<br />
<br />
Don't change this parameter, unless you are completely sure you have correctly diagnosed your problem. Else, be assured performance will be degraded.<br />
<br />
===Statement logged by auto_explain===<br />
<br />
auto_explain contrib module will now print the statement with its plan, which will make it much easier to use.<br />
<br />
===Buffers accounting for pg_stat_statements===<br />
<br />
This already very useful contrib module now also provides data about buffers. pg_stat_statements, as a reminder, collects statistics on the queries run on the database. Until now, it stored the query's code, number of executions, accumulated runtime, accumulated returned records. It now collects buffer operations too.<br />
marc=# SELECT * from pg_stat_statements order by total_time desc limit 2;<br />
-[ RECORD 1 ]-------+---------------------<br />
userid | 10<br />
dbid | 16485<br />
query | SELECT * from table1 ;<br />
calls | 2<br />
total_time | 0.491229<br />
rows | 420000<br />
shared_blks_hit | 61<br />
shared_blks_read | 2251<br />
shared_blks_written | 0<br />
local_blks_hit | 0<br />
local_blks_read | 0<br />
local_blks_written | 0<br />
temp_blks_read | 0<br />
temp_blks_written | 0<br />
-[ RECORD 2 ]-------+---------------------<br />
userid | 10<br />
dbid | 16485<br />
query | SELECT * from table2;<br />
calls | 2<br />
total_time | 0.141445<br />
rows | 200000<br />
shared_blks_hit | 443<br />
shared_blks_read | 443<br />
shared_blks_written | 0<br />
local_blks_hit | 0<br />
local_blks_read | 0<br />
local_blks_written | 0<br />
temp_blks_read | 0<br />
temp_blks_written | 0<br />
When this contrib is installed, one can now answer these questions:<br />
* Which query has the biggest accumulated runtime ?<br />
* Which query generates the most IO operations ? (we still can't know if data has been found in the Operating System's cache)<br />
* Which query uses mostly the cache (and hence won't be faster if we make it bigger) ?<br />
* Which query modifies the most blocks ?<br />
* Who does sorting ?<br />
'local' and 'temp' are the buffer operations relative to temporary tables and other local operations (sorts, hashes) to a database backend. If there are many reads and writes in them, you might be better to increase temp_buffers (for 'local') or work_mem (for 'temp').<br />
<br />
==Stored Procedures==<br />
<br />
PostgreSQL isn't just a database, it's a whole application platform. Many of our users write entire applications using stored procedures and functions. So, it's no surprise that 9.0 brings a number of improvements in database procedural code:<br />
<br />
===PL/pgSQL by default===<br />
<br />
You won't have to add PL/pgSQL in databases, as it will be installed by default. This has been requested for a long time.<br />
<br />
===Many improvements on PL languages.===<br />
<br />
Many languages have been vastly improved, PLPerl for instance. Read the release notes if you want more details, there are too many to detail here.<br />
<br />
===Anonymous Functions (aka Anonymous Blocks)===<br />
<br />
This new feature is for creating run-once functions. Effectively, this allows you to run stored procedure code on the command line or dynamically as you can on SQL Server and Oracle. Unlike those, however, PostgreSQL allows you to run an anonymous function in any procedural language which is installed of the more than a dozen which PostgreSQL supports.<br />
<br />
This feature will be very useful for schema upgrade scripts for instance. Here is a slightly different version of the 'GRANT SELECT ON ALL TABLES' that will be seen later in this document, giving SELECT rights to a bunch of tables, depending on the table owner, and excluding two schemas:<br />
DO language plpgsql $$<br />
DECLARE<br />
vr record;<br />
<br />
BEGIN<br />
<br />
FOR vr IN SELECT tablename FROM pg_tables WHERE tableowner = 'marc' AND schemaname NOT IN ('pg_catalog','information_schema')<br />
LOOP<br />
EXECUTE 'GRANT SELECT ON ' || vr.tablename || ' TO toto';<br />
END LOOP;<br />
END<br />
$$;<br />
As of 8.4, this would have required creating a function (with CREATE FUNCTION), running it, then removing it (with DROP FUNCTION). All of this requiring having rights to do this. 9.0 simplifies performing this kind of procedures.<br />
<br />
Anonymous functions are also called "anonymous code blocks" in the software industry.<br />
<br />
===Named Parameter Calls===<br />
<br />
Combined with the Default Parameters introduced in version 8.4, named parameters allow for dynamic calling of functions with variable numbers of arguments, much as they would be inside a programming language. Named parameters are familiar to users of SQL Server or Sybase, but PostgreSQL does one better by supporting both named parameter calls ''and'' function overloading.<br />
<br />
The chosen syntax to name parameters is the following:<br />
CREATE FUNCTION test (a int, b text) RETURNS text AS $$<br />
DECLARE<br />
value text;<br />
BEGIN<br />
value := 'a is ' || a::text || ' and b is ' || b;<br />
RETURN value;<br />
END;<br />
$$ LANGUAGE plpgsql;<br />
Until now, we wrote:<br />
SELECT test(1,'foo');<br />
test <br />
-------------------------<br />
a is 1 and b is foo<br />
(1 row)<br />
Now this explicit syntax can be used:<br />
SELECT test( b:='foo', a:=1);<br />
test <br />
-------------------------<br />
a is 1 and b is foo<br />
(1 row)<br />
Named parameters should eliminate the need to write many overloaded "wrapper" functions. Note that this does add a backwards compatibility issue; you are no longer able to rename function parameters using a REPLACE command, but must now drop and recreate the function.<br />
<br />
===ALIAS keyword===<br />
<br />
ALIAS can now be used. As its name suggests, it can be used to alias variable names to other names.<br />
<br />
The syntax is <tt>new_name ALIAS FOR old_name</tt>. This is put in the DECLARE section of PL/pgSQL code.<br />
<br />
It has two main use cases:<br />
* to give names to PL function variables:<br />
myparam ALIAS FOR $0<br />
* to rename potentially conflicting variables. In a trigger for instance:<br />
<br />
new_value ALIAS FOR new<br />
: (without this, we might have conflicted with the NEW variable in the trigger function).<br />
<br />
==Advanced Features==<br />
<br />
Some features in PostgreSQL are cutting-edge database features which are pretty much "PostgreSQL only". It's why we're the "most advanced database". These features enable new types of applications.<br />
<br />
===Exclusion constraints===<br />
<br />
Exclusion constraints are very similar to unique constraints. They could be seen as unique constraints using other operators than '=': A unique constraint defines a set of columns for which two records in the table cannot be identical.<br />
<br />
To illustrate this, we will use the example provided by this feature's author, using the temporal data type, that he also developed. This datatype stores time ranges, that is 'the time range from 10:15 to 11:15'.<br />
<br />
First, we need to retrieve the temporal module here: http://pgfoundry.org/projects/temporal/ , then compile and install it as a contrib (run the provided sql script). We may also need to install the btree_gist module as a contrib. From source, one can run 'make install' in contrib/btree_gist directory for the same.<br />
<br />
CREATE TABLE reservation<br />
(<br />
room TEXT,<br />
professor TEXT,<br />
during PERIOD);<br />
<br />
ALTER TABLE reservation ADD CONSTRAINT test_exclude EXCLUDE USING gist (room WITH =,during WITH &amp;&amp;);<br />
Doing this, we declare that a record should be rejected (exclusion constraint) if there already is one verifying the two conditions 'the same room' and 'be in intersection for the time range' (the &amp;&amp; operator).<br />
marc=# INSERT INTO reservation (professor,room,during) VALUES ( 'mark', 'tech room', period('2010-06-16 09:00:00', '2010-06-16 10:00:00'));<br />
INSERT 0 1<br />
marc=# INSERT INTO reservation (professor,room,during) VALUES ( 'john', 'chemistry room', period('2010-06-16 09:00:00', '2010-06-16 11:00:00'));<br />
INSERT 0 1<br />
marc=# INSERT INTO reservation (professor,room,during) VALUES ( 'mark', 'chemistry room', period('2010-06-16 10:00:00', '2010-06-16 11:00:00'));<br />
ERROR: conflicting key value violates exclusion constraint "test_exclude"<br />
DETAIL: Key (room, during)=(chemistry room, [2010-06-16 10:00:00+02, 2010-06-16 11:00:00+02)) conflicts with existing key (room, during)=(chemistry room, [2010-06-16 09:00:00+02, 2010-06-16 11:00:00+02)).<br />
The insert is forbidden, as the chemistry room is already reserved from 9 to 11.<br />
<br />
Exclusion constraints may also be used with arrays, geographic data, or other non-scalar data in order to implement advanced scientific and calendaring applications. No other database system has this feature.<br />
<br />
===Message passing in NOTIFY/pg_notify===<br />
<br />
Messages can now be passed using NOTIFY. Here is how:<br />
* Subscribe in session 1 to the 'instant_messenging' queue.<br />
: Session 1:<br />
marc=# LISTEN instant_messenging;<br />
LISTEN<br />
* Send a notification through 'instant_messenging', from another session<br />
: Session 2:<br />
marc=# NOTIFY instant_messenging, 'You just received a message';<br />
NOTIFY<br />
* Check the content of the queue in the first session<br />
: Session 1:<br />
marc=# LISTEN instant_messenging;<br />
LISTEN<br />
Asynchronous notification "instant_messenging" with payload "You just received a message" received from server process with PID 5943.<br />
<br />
So we can now associate messages (payloads) with notifications, making NOTIFY even more useful.<br />
<br />
Let's also mention the new pg_notify function. With it, the second session's code can also be:<br />
SELECT pg_notify('instant_messenging','You just received a message');<br />
This can simplify some code, in the case of a program managing a lot of different queues.<br />
<br />
===Hstore contrib enhancements===<br />
<br />
This already powerful contrib module has become even more powerful:<br />
* Keys and values size limit has been removed.<br />
* GROUP BY and DISTINCT can now be used.<br />
* New operators and functions have been added.<br />
<br />
An example would take too long, this module has a lot of features. Read the documentation at once !<br />
<br />
===Unaccent filtering dictionary===<br />
<br />
Filtering dictionaries can now be set up. This is about Full Text Search dictionaries.<br />
<br />
These dictionaries' purpose it applying a first filter on words before lexemizing them. The module presented here is the first one to use this mechanism. Filtering can consist in removing words or modifying them.<br />
<br />
Unaccent doesn't remove words, it removes accents (all diacritic signs, as a matter of fact), replacing accentuated characters with non-accentuated ones (many people, at least in French, don't type them). Unaccent is a contrib module.<br />
<br />
Installing it, as all contrib modules, is as easy as<br />
psql mydb &lt; contribs_path/unaccent.sql.<br />
We'll now follow unaccent's documentation, the example being filtering french words.<br />
<br />
Let's create a new 'fr' dictionary (keeping standard 'french' dictionary clean): <br />
marc=# CREATE TEXT SEARCH CONFIGURATION fr ( COPY = french );<br />
CREATE TEXT SEARCH CONFIGURATION<br />
The next statement alters the 'fr' setup for word and alike lexemes. These now have to go through unaccent and french_stem instead of only french_stem.<br />
marc=# ALTER TEXT SEARCH CONFIGURATION fr<br />
>ALTER MAPPING FOR hword, hword_part, word<br />
>WITH unaccent, french_stem;<br />
>ALTER TEXT SEARCH CONFIGURATION<br />
<br />
SELECT to_tsvector('fr','Hôtels de la Mer');<br />
to_tsvector <br />
-------------------<br />
'hotel':1 'mer':4<br />
(1 row)<br />
<br />
marc=# SELECT to_tsvector('fr','Hôtel de la Mer') @@ to_tsquery('fr','Hotels');<br />
?column?<br />
----------<br />
t<br />
(1 row)<br />
It's now easy, without changing even one line of code in the client application, and keeping accentuated characters in the database, to look up words without taking accents into account.<br />
<br />
<br />
<br />
<br />
===get_bit and set_bit for bit strings===<br />
<br />
Here is a very simple example. This tool can manipulate bits in a bit() independently.<br />
marc=# SELECT set_bit('1111'::bit(4),2,0);<br />
set_bit<br />
---------<br />
1101<br />
(1 row)<br />
<br />
<br />
marc=# SELECT get_bit('1101'::bit(4),2);<br />
get_bit<br />
---------<br />
0<br />
(1 row)<br />
<br />
<br />
=Backwards Compatibility and Upgrade Issues=<br />
<br />
The PostgreSQL project has a commitment not to break backwards compatibility when we can possibly avoid doing so. Sometimes, however, we have to break things in order to add new features or fix longstanding problematic behavior. Some of these issues are documented below.<br />
<br />
==PL/pgSQL changes which may cause regressions==<br />
<br />
There are two changes in PL/pgSQL which may break code which works in 8.4 or earlier, meaning PL/pgSQL functions should be audited before before migrating to 9.0 to prevent possible runtime errors. A lot of these come about due to uniting the lexer for SQL and PL/pgSQL, which is an important architectural improvement which has made several new features possible.<br />
<br />
===Removal of column/variable name ambiguity===<br />
<br />
In 8.4 and earlier, PL/PgSQL variables will take preference over a table or view column with the same name. While this behaviour is consistent, it is a potential source of coding errors. 9.0 will throw a runtime error if this situation occurs:<br />
<br />
marc=# DO LANGUAGE plpgsql<br />
$$<br />
DECLARE<br />
a int;<br />
BEGIN<br />
SELECT a FROM test;<br />
END<br />
$$<br />
;<br />
ERROR: column reference "a" is ambiguous<br />
LINE 1: select a from test<br />
DETAIL: It could refer to either a PL/pgSQL variable or a table column.<br />
QUERY: select a from test<br />
CONTEXT: PL/pgSQL function "inline_code_block" line 4 at SQL statement<br />
<br />
This behaviour can be altered globally in postgresql.conf, or on a per function basis by inserting one of these three options in the function declaration:<br />
<br />
#variable_conflict error (default)<br />
#variable_conflict use_variable (variable name name takes precedence - pre-9.0 behaviour)<br />
#variable_conflict use_column (column name takes precedence)<br />
<br />
The [http://www.postgresql.org/docs/9.0/static/plpgsql-implementation.html manual] contains more details.<br />
<br />
===Reserved words===<br />
<br />
From 9.0, use of unquoted reserved words as PL/PgSQL variable names is no longer permitted:<br />
<br />
marc=# DO LANGUAGE plpgsql<br />
$$<br />
DECLARE<br />
table int;<br />
BEGIN<br />
table :=table+1;<br />
END<br />
$$<br />
;<br />
ERROR: syntax error at or near "table"<br />
LINE 6: table :=table+1;<br />
<br />
The correct syntax is:<br />
<br />
marc=# DO LANGUAGE plpgsql<br />
$$<br />
DECLARE<br />
"table" int;<br />
BEGIN<br />
"table" :="table"+1;<br />
END<br />
$$<br />
;<br />
DO<br />
<br />
Best practice is of course to avoid reserved words completely.<br />
<br />
[[Category:PostgreSQL 9.0]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Binary_Replication_Tutorial&diff=16626Binary Replication Tutorial2012-04-26T23:47:22Z<p>Schmiddy: /* Recovery.conf */ typofix for pg_standby command, reported by Glen Robertson</p>
<hr />
<div>Welcome to the new PostgreSQL 9 replication and standby databases guide. This new set of features implements possibly the longest awaited functionality in PostgreSQL's history. As a result, a lot of people are going to be trying to deploy standby databases for the first time, and find the process rather unintuitive. This guide is here to help.<br />
<br />
'''Work in progress: only 40% complete'''<br />
<br />
= 5 Minutes to Simple Replication =<br />
<br />
This is the easiest way to set up replication between a master and standby. It requires shutting down the master; other methods are detailed later in this guide.<br />
<br />
What we're going to do is shut down the master and copy the files we need over to the slave server, creating a cloned copy of the master. Because the master is shut down, we don't have to worry about changes being made to it.<br />
<br />
Note: Both the '5 minutes' instructions and the '10 minutes' version which follows do not deal with the complications that arise with a database that uses tablespaces, specifically what to do about the pg_tblspc directory and its contents.<br />
<br />
== Prerequisites ==<br />
<br />
You must have the right setup to make this work:<br />
<br />
* 2 servers with similar operating systems (e.g both Linux 64-bit).<br />
* The same release of PostgreSQL 9.0 installed on both servers.<br />
* PostgreSQL superuser shell access on both servers.<br />
* Knowledge of how to start, stop and reload Postgres.<br />
* PostgreSQL 9.0 running on Server1.<br />
* A database created and loaded on Server1.<br />
* A postgres user or root user who has network <br />
<br />
See the full documentation for more information:<br />
<br />
* [http://www.postgresql.org/docs/9.0/static/warm-standby.html 9.0 Replication Documentation]<br />
* [http://www.postgresql.org/docs/9.1/static/warm-standby.html 9.1 Replication Documentation]<br />
<br />
== Binary Replication in 6 Steps ==<br />
<br />
This 6-step guide, and all of the examples in this tutorial, assume that you have a master server at 192.168.0.1 and a standby server at 192.168.0.2 and that your database and its configuration files are installed at /var/lib/postgresql/data. Replace those with whatever your actual server addresses and directories are.<br />
<br />
1. Edit postgresql.conf on the master to turn on streaming replication. Change these settings:<br />
<br />
listen_address = '*'<br />
wal_level = hot_standby<br />
max_wal_senders = 3<br />
<br />
2. Edit pg_hba.conf on the master in order to let the standby connect. <br />
<br />
host replication all 192.168.0.2/32 trust<br />
<br />
3. Edit recovery.conf and postgresql.conf on the standby to start up replication and hot standby. First, in postgresql.conf, change this line:<br />
<br />
hot_standby = on<br />
<br />
Then create a file in the standby's '''data directory''' (which is often the same directory as postgresql.conf and pg_hba.conf, except on some Linux distributions such as Debian and Ubuntu), called recovery.conf, with the following lines:<br />
<br />
standby_mode = 'on'<br />
primary_conninfo = 'host=192.168.0.1'<br />
<br />
4. Shutdown the master and copy the files. You want to copy most but not all files between the two servers, excluding the configuration files and the pg_xlog directory. An example rsync script would be:<br />
<br />
rsync -av --exclude pg_xlog --exclude postgresql.conf data/* 192.168.0.2:/var/lib/postgresql/data/<br />
<br />
5. Start the standby first, so that they can't get out of sync. (Messages will be logged about not being able to connect to the primary server, that's OK.)<br />
<br />
6. Start the master.<br />
<br />
== Starting Replication with only a Quick Master Restart ==<br />
<br />
Is taking down the master for long enough to copy the files too long? Then you need the 10-minute version.<br />
<br />
What we're going to do this time is similar to what we did before, cloning the database by copying the files from the master to the slave server. However, because the database is only going to be shut down for a short period of time, long enough to activate the changes in the configuration file, after we've copied the data files we will need to copy additional files so that the slave will be an up-to-date copy of the master. <br />
<br />
So, we will tell the master we're running a backup, copy the data files (not quite the same set of files as before), tell the master the backup is complete, then copy the WAL files in the pg_xlog directory so that when the slave comes up it can make all the changes that were committed to the master database after the backup was started.<br />
<br />
First, start with the same prerequisites as above.<br />
<br />
1. Set the postgresql.conf variables the same in step (1) as above.<br />
<br />
2. Don't close the file yet. You'll need to set two other variables which control the size of your write-ahead-log (WAL). The first is wal_keep_segments, the second is checkpoint_segments. Unless you've already done so, you're going to need to increase these, which is usually a good idea for performance anyway. You want the WAL to be big enough to not get used up in 15 or 20 minutes. If you don't have a clear idea of that, here's some reasonable values, based on how busy and how large your database is. Also, a database with large blob objects may require a much larger setting. Remember, these logs will take up disk space, so make sure that you have enough available - space requirements are below.<br />
<br />
checkpoint_segments = 8 <br />
wal_keep_segments = 8 <br />
# light load 500MB<br />
<br />
checkpoint_segments = 16<br />
wal_keep_segments = 32<br />
# moderately busy 1.5GB <br />
<br />
checkpoint_segments = 64<br />
wal_keep_segments = 128<br />
# busy server 5GB<br />
<br />
You don't ''have'' to increase checkpoint_segments in order to increase wal_keep_segments, but it's generally a good idea. Now save the file. <br />
<br />
3. Edit pg_hba.conf as in (2) in the "Six Steps" above.<br />
<br />
4. Now you need to restart the master. Given the interruption in service, you should probably plan this ahead. <br />
<br />
5. Edit postgresql.conf and recovery.conf on the standby as in (3) above.<br />
<br />
6. Now, we're going to need to copy the files from the master and start the standby. Unlike in the 6-step version, this needs to be done quickly or the standby will fail to sync and you'll need to try again. First step, you need to tell the master you're starting a backup (see below for a more detailed explanation of this). Log in to psql as the database superuser.<br />
<br />
psql -U postgres<br />
# select pg_start_backup('clone',true);<br />
<br />
Note that the string you use as a backup label doesn't matter; use any string you want.<br />
<br />
7. Now, quickly copy all the database files. This rsync is slightly different from the 6-step version:<br />
<br />
rsync -av --exclude pg_xlog --exclude postgresql.conf --exclude postgresql.pid \ <br />
data/* 192.168.0.2:/var/lib/postgresql/data/<br />
<br />
8. As soon as that's done you need to stop the backup on the master:<br />
<br />
# select pg_stop_backup();<br />
<br />
9. As soon as that completes, you need to quickly copy the WAL files from the master to the standby.<br />
<br />
rsync -av data/pg_xlog 192.168.0.2:/var/lib/postgresql/data/<br />
<br />
10. Now, start the standby.<br />
<br />
If you've done this quickly enough, then the standby should catch up with the master and you should be replicating. If not, you'll get this message:<br />
<br />
(Future Revisions note: Message needs to go here)<br />
<br />
... which means you need to try again, possibly with checkpoint_segments and wal_keep_segments higher. If that still doesn't work, you're going to need to use the even more complex archiving method described below.<br />
<br />
Now, the rest of the guide will explain how to deal with more complex situations, such as archive logs, handling security, and maintaining availability, failover and standby promotion.<br />
<br />
= Introduction to Binary Replication =<br />
<br />
Binary replication is also called "Hot Standby" and "Streaming Replication" which are two separate, but complimentary, features of PostgreSQL 9.0 and later. Here's some general information about how they work and what they are for.<br />
<br />
== What Can You Do With Binary Replication? ==<br />
<br />
* Have a simple and complete replica of your production database, preventing all but a couple seconds of data loss even under catastrophic circumstances.<br />
* Load-balance between your read/write master server and multiple read-only servers.<br />
* Run reporting or other long-running queries on a replica server, taking them off your main transaction-processing server.<br />
* Replicate all DDL, including table and index changes, and even creating new databases.<br />
* Replicate a hosted multi-tenant database, making no specific requirements for primary keys or database changes of your users.<br />
<br />
== What Can't You Do With Binary Replication? ==<br />
<br />
* Replicate a specific table, schema, or database. Binary replication is the entire Postgres instance (or "cluster").<br />
* Multi-master replication. Multi-master binary replication is probably technically impossible.<br />
* Replicate between different versions of PostgreSQL, or between different platforms.<br />
* Set up replication without administration rights on the server. Sorry, working on it.<br />
* Replicate data synchronously, guaranteeing zero data loss. But ... this is coming in PostgreSQL 9.1.<br />
<br />
For the reasons above, we expect that Slony-I, Londiste, Bucardo, pgPool2 and other systems will continue to be used.<br />
<br />
== Transaction Logs and Log Shipping ==<br />
<br />
Users who are already familiar with the PostgreSQL transaction log and warm standby can skip this section.<br />
<br />
An individual "instance", "server", or (confusingly) "cluster" of PostgreSQL (hereafter Server) consists of a single postmaster server process connected to a single initialized PostgreSQL data directory (PGDATA), which in turn contains several databases. Each running Server has a transaction log, located in the PGDATA/pg_xlog directory. This transaction log consists of binary snapshots of data, written to record synchronously each change to all databases' data, in case of unexpected shutdown of the database server (such as in a power failure). This ensures that data is not corrupted and no completed transaction is lost.<br />
<br />
You can also use this log to allow a copy of the original database to replicate changes made to a master database. This was first implemented with the PITR feature in PostgreSQL 8.0, and is known as "log shipping". Log shipping is required for most forms of binary replication.<br />
<br />
This log consists of 16MB segments full of new data pages (8K segments) of the database, and not of SQL statements. For this reason there is no before and after auditing possible via this log, as you cannot know exactly what has changed. Also, the log is treated as a buffer, being deleted as it is no longer needed for crash recovery. More importantly, the data page format of the log means that log segments can only be applied to a database which is binary-identical to the database which created the log. <br />
<br />
== PITR, Warm Standby, Hot Standby, and Streaming Replication ==<br />
<br />
For the rest of this tutorial, we will refer to the active read-write instance of the Server which generates transaction logs as the "Master" and the passive, read-only or offline instance (or instances) of the Server which receives transaction logs as the "Standby" (or "Standbys"). The term Master/Standby is equivalent to other terminology which may be used in the database industry, such as Master/Slave, Primary/Secondary or Primary/Replica.<br />
<br />
=== PITR ===<br />
<br />
In Point-In-Time Recovery (PITR), transaction logs are copied and saved to storage until needed. Then, when needed, the Standby server can be "brought up" (made active) and transaction logs applied, either stopping when they run out or at a prior point indicated by the administrator. PITR has been available since PostgreSQL version 8.0, and as such will not be documented here.<br />
<br />
PITR is primarily used for database forensics and recovery. It is also useful when you need to back up a very large database, as it effectively supports incremental backups, which pg_dump does not.<br />
<br />
=== Warm Standby ===<br />
<br />
In Warm Standby, transaction logs are copied from the Master and applied to the Standby immediately after they are received, or at a short delay. The Standby is offline (in "recovery mode") and not available for any query workload. This allows the Standby to be brought up to full operation very quickly. Warm Standby has been available since version 8.3, and will not be fully documented here.<br />
<br />
Warm Standby requires Log Shipping. It is primary used for database failover.<br />
<br />
=== Hot Standby ===<br />
<br />
Hot Standby is identical to Warm Standby, except that the Standby is available to run read-only queries. This offers all of the advantages of Warm Standby, plus the ability to distribute some business workload to the Standby server(s). Hot Standby by itself requires Log Shipping.<br />
<br />
Hot Standby is used both for database failover, and can also be used for load-balancing. In contrast to Streaming Replication, it places no load on the master (except for disk space requirements) and is thus theoretically infinitely scalable. A WAL archive could be distributed to dozens or hundreds of servers via network storage. The WAL files could also easily be copied over a poor quality network connection, or by SFTP.<br />
<br />
However, since Hot Standby replicates by shipping 16MB logs, it is at best minutes behind and sometimes more than that. This can be problematic both from a failover and a load-balancing perspective.<br />
<br />
=== Streaming Replication ===<br />
<br />
Streaming Replication improves either Warm Standby or Hot Standby by opening a network connection between the Standby and the Master database, instead of copying 16MB log files. This allows data changes to be copied over the network almost immediately on completion on the Master. <br />
<br />
In Streaming Replication, the master and the standby have special processes called the walsender and walreceiver which transmit modified data pages over a network port. This requires one fairly busy connection per standby, imposing an incremental load on the master for each additional standby. Still, the load is quite low and a single master should be able to support multiple standbys easily.<br />
<br />
Streaming replication does not require log shipping in normal operation. It may, however, require log shipping to start replication, and can utilize log shipping in order to catch up standbys which fall behind.<br />
<br />
= How to Replicate =<br />
<br />
== Cloning a Live Database ==<br />
<br />
If your workload doesn't allow you to take the master down (and whose does?), things get a bit more complicated. You need to somehow take a "coherent snapshot" of the master, so that you don't have an inconsistent or corrupt database on the standby. Now, in some cases this can be done via filesystem snapshotting tools or similar tricks, but as that approach is tricky and platform-dependant, we're not going to cover it here.<br />
<br />
Instead, we're going to cover the built-in method, which involves keeping a log of all changes applied to the database which happen during the copying process. The steps are essentially the same, regardless of whether you're planning to use just hot standby, streaming replication, or both. There are two parts:<br />
<br />
* Cloning the database files<br />
* Copying the archive logs<br />
<br />
Unintuitive as it is, the latter needs to be set up first, so we're going to start with that.<br />
<br />
== Setting Up Archiving On The Master ==<br />
<br />
Archiving is the process of making an extra copy of each WAL file as it is completed. These log files then need to somehow be accessed by the standby. There are three basic ways to handle this, and you should decide in advance what method you're going to use:<br />
<br />
# Manually<br />
# Automatic file copying from master to standby using rsync or simiar<br />
# Writing them to a common shared network file location<br />
<br />
The first method is only appropriate if you're archiving logs only to jump-start streaming replication, and you have a fairly low-traffic database or the ability to stop all writes. The third method is probably the easiest to manage if you have an appropriate network share; it can even be used to support multiple standbys with some extra thought and scripting. All of these methods will be explained below.<br />
<br />
This needs to be turned on on the master, which if it's never been done before may require a restart (sorry, working on it), and will certainly require a reload. You'll need to set the following parameters:<br />
<br />
wal_level = hot_standby<br />
archive_mode = on<br />
archive_command = 'some command'<br />
<br />
What archive command you use depends on which archiving approach you are taking, of course. Here are three examples of commands you might use. Note that you will need to create the "archive" directories.<br />
<br />
# Manual: cp -f %p /var/lib/postgresql/data/archive/%f </dev/null<br />
# Automatic Copy: rsync -a %p 192.168.0.2:/var/lib/pgsql/data/archive/%f<br />
# Network Share: cp -f %p /shares/walarchive/archive/%f </dev/null<br />
<br />
In these commands, %p is replace by postgres at invocation time with the full path and name of the WAL file, and %f with the name of the file alone. There are more escapes and parameters dealing with WAL archiving which will be detailed later in the tutorial. Note that, in real production, you are unlikely to want to use any commands as simple as the above. In general, you will want to have archive_command call an executable script which traps errors and can be disabled. Examples of such scripts are available in this tutorial.<br />
<br />
Now, if archive_mode was originally "off" or if you had to change wal_level, you're going to need to restart the master (sorry, this will be fixed in a later version). If you just needed to change the archive_command, however, only a reload is required.<br />
<br />
Once you've restarted or reloaded, check the master's logs to make sure archiving is working. If it's failing, the master will complain extensively. You might also check that archive log files are being created; run the command "SELECT pg_switch_xlog();" as the superuser to force a new log to be written.<br />
<br />
== Setting Up Archiving on the Standby ==<br />
<br />
The standby needs to be configured to consume logs. This is simpler than the master's setup, and doesn't really change no matter what archive copying strategy you're using.<br />
<br />
== Recovery.conf ==<br />
<br />
On the standby, replication configuration is controlled through a file called, for historical reasons, recovery.conf. If this file is present in PostgreSQL's data directory when PostgreSQL is started, that server will assume it is a standby and attempt to obey it. Generally, there is an example file installed with the other PostgreSQL shared docs. However, that example file covers all of the various replication options at once, so it's often simpler to write your own file, from scratch. Any change to recovery.conf requires a restart of the standby.<br />
<br />
In recovery.conf, you need to add a command to copy the archived WAL files to the standby's on pg_xlog directory. This is the mirror image of the archive_command on the master. Generally, a simple cp command is sufficient:<br />
<br />
restore_command = 'cp -f /var/lib/postgresql/data/archive/%f %p </dev/null'<br />
restore_command = 'cp -f /shares/walarchive/%f %p </dev/null'<br />
<br />
Again, you might want to use a simple shell script which traps error messages, and, importantly, deletes archive files which are no longer needed. If you will be doing only hot standby and not using streaming replication, you probably want to compile the pg_standby binary provided in PostgreSQL's additional modules or "contrib", and use it instead:<br />
<br />
restore_command = 'pg_standby /shares/walarchive %f %p %r'<br />
<br />
More detail on pg_standby is in its documentation.<br />
<br />
== Cloning a Snapshot of the Master ==<br />
<br />
Once you have archiving working, you're ready to clone the master database. At this point, it's a simple process:<br />
<br />
# As superuser, issue the command "SELECT pg_start_backup('backup');" on the master.<br />
# Copy all of the database files to the standby.<br />
# Start the standby database.<br />
# Issue the command "SELECT pg_stop_backup();" on the master.<br />
<br />
Of course, each of those steps deserves a little more elaboration. pg_start_backup and pg_stop_backup are special commands you issue on the master in order to create, hold open, and close, a "snapshot" which is how we make sure your copy of the database is not inconsistent. They also write special files to the archive log which tell the standby when it has a complete snapshot.<br />
<br />
If you are using the "manual" method of synching the archive logs, immediately after step 4 you need to do one last rsync or copy of the archive logs to the standby.<br />
<br />
When you're done with the cloning, you should see output similar to the below:<br />
<br />
This means that you're up and replicating, and should now be able to run queries on the standby.<br />
<br />
== Failing Over To The Standby ==<br />
<br />
Of course, one of the major reasons to have a standby is in case something (planned or unplanned) causes the master server to shut down. Then you want to "fail over", or stop replication and change the standby to a full read-write master.<br />
<br />
The recommended method is the same regardless of the type of replication or standby: via "trigger file". First, you need to set a configuration option in recovery.conf on the standby:<br />
<br />
trigger_file = '/var/lib/postgresql/data/failover'<br />
<br />
Then, when it's time to fail over, you just create an empty file with that name, such as by using the "touch" command. The standby will notice the file, attempt to apply any remaining WAL records or files it has received, and then switch to read-write or "master" mode. When this happens, you will see a message like this in the Postgres log:<br />
<br />
PostgreSQL will also rename the recovery.conf file to recovery.done in order to prevent having the new master fail on restart. For this reason, the recovery.conf file should be owned by the same user which the server runs as (usually "postgres").<br />
<br />
The alternative to using a trigger file is to failover manually, by deleting or renaming the recovery.conf file and restarting the standby. This method is inferior because it requires a restart which would interrupt any read-only connections to the standby currently in use.<br />
<br />
In a high-availability system, the above activity should be managed automatically in order to avoid downtime. PostgreSQL itself supplies no tools to do this, but numerous third-party utilities such as "Linux heartbeat" are compatible with PostgreSQL replication.<br />
<br />
It's important to prevent the original master from restarting after failover, lest you end up with a "split brain" problem and data loss. There is a substantial body of literature on this, and third-party tools, so we won't discuss them here at this time.<br />
<br />
== Load Balancing ==<br />
<br />
== Managing Archive Logs ==<br />
<br />
== Tuning and Configuration of Binary Replication ==<br />
<br />
== Monitoring Replication ==<br />
<br />
<br />
[[Category:Replication]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Priorities&diff=16502Priorities2012-04-07T21:14:11Z<p>Schmiddy: /* Prioritizing CPU */ Remove text about using "nice" (should have been "renice") only manually, and added link and description of "prioritize" module</p>
<hr />
<div>== Prioritizing users, queries, or databases ==<br />
<br />
PostgreSQL has no facilities to limit what resources a particular user, query, or database consumes, or correspondingly to set priorities such that one user/query/database gets more resources than others. It's necessary to use operating system facilities to achieve what limited prioritization is possible.<br />
<br />
There are three main resources that PostgreSQL users, queries, and databases will contend for:<br />
<br />
* Memory<br />
* CPU<br />
* Disk I/O<br />
<br />
Of these, disk I/O is commonly a bottleneck for database applications, but that's not always the case. Some schema designs and queries are particularly CPU heavy. Others really benefit from having lots of memory to work with, typically for sorting.<br />
<br />
== Are priorities really the problem? ==<br />
<br />
Before struggling too much with prioritizing your queries/users/databases, it's worthwhile to optimize your queries and [[Tuning_Your_PostgreSQL_Server|tune your database]]. You may find that you can get perfectly acceptable performance without playing with priorities or taking extreme measures, using techniques such as:<br />
<br />
* [[Using_EXPLAIN|Improving your queries]]<br />
* Tune autovacuum to reduce bloat<br />
* [[Performance_Optimization|Generally polishing your cluster's performance]]<br />
* Avoiding use of [[VACUUM FULL]]. That can lead to bloated indexes that eat lots of memory and take forever to scan, wasting disk I/O bandwidth. See the wiki page on [[VACUUM FULL]] for more information.<br />
<br />
=== Is CPU really the bottleneck? ===<br />
<br />
People often complain of pegged (100%) CPU and assume that's the cause of database slowdowns. That's not necessarily the case - a system may show an apparent 100% CPU use, but in fact be mainly limited by I/O bandwidth. Consider the following test, which starts 20 `dd' processes, each reading a different 1Gb block from the hard disk at 1Gb offsets.<br />
<br />
<pre><br />
for i in `seq 1 20`; do <br />
( dd if=/dev/md0 bs=1M count=1000 skip=$(($i * 1000)) of=/dev/null &)<br />
done<br />
</pre><br />
<br />
results in `top' output of:<br />
<br />
<pre> <br />
top - 14:51:55 up 3 days, 2:09, 5 users, load average: 10.92, 4.94, 2.93<br />
Tasks: 259 total, 3 running, 256 sleeping, 0 stopped, 0 zombie<br />
Cpu(s): 1.6%us, 15.0%sy, 0.0%ni, 0.0%id, 78.6%wa, 0.8%hi, 4.0%si, 0.0%st<br />
Mem: 4055728k total, 3843408k used, 212320k free, 749448k buffers<br />
Swap: 2120544k total, 4144k used, 2116400k free, 2303356k cached<br />
<br />
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br />
33 root 15 -5 0 0 0 R 5 0.0 0:26.67 kswapd0 <br />
904 root 20 0 4152 1772 628 D 5 0.0 0:00.62 dd <br />
874 root 20 0 4152 1768 628 D 3 0.0 0:00.74 dd <br />
908 root 20 0 4152 1768 628 D 3 0.0 0:00.80 dd <br />
888 root 20 0 4152 1772 628 D 3 0.0 0:00.44 dd <br />
906 root 20 0 4152 1772 628 D 3 0.0 0:00.56 dd <br />
894 root 20 0 4152 1768 628 D 2 0.0 0:00.49 dd <br />
902 root 20 0 4152 1772 628 D 2 0.0 0:00.46 dd <br />
.... etc .... <br />
</pre><br />
<br />
... which could be confused for a busy CPU, but is really load caused by disk I/O. The key warning sign here is the presence of a high iowait cpu percentage ("%wa"), indicating that much of the apparent load is actually caused by delays in the I/O subsystem. Most of the `dd' processes are in 'D' state - ie uninterruptable sleep in a system call - and if you check "wchan" with "ps" you'll see that they're sleeping waiting for I/O. <br />
<br />
Rather than assuming that CPU contention is the issue, it's a good idea to use the available [[Performance Analysis Tools]] to get a better idea of where your system bottlenecks really are.<br />
<br />
== Prioritizing CPU ==<br />
<br />
For adjusting the CPU priority of PostgreSQL processes, you can use "renice" (on UNIX systems), but it's a bit clumsy to do since you need to "renice" the backend of interest, not the client program connected to that backend. You can get the backend process id using the SQL query "SELECT pg_backend_pid()" or by looking at the pg_stat_activities view.<br />
<br />
One significant limitation of "renice", or any approach based on the setpriority() call, is that on most UNIX-like platforms one must be root to lower the numerical priority value (i.e. schedule the process to run more urgently) of a process.<br />
<br />
Increasing the priority of important backends, via a root user's call to "renice", instead of lowering the priority of unimportant ones, may be more effective.<br />
<br />
== prioritize module ==<br />
The [http://pgxn.org/dist/prioritize/ prioritize] extension lets users adjust the CPU priority, in the same way that "renice" does, via the SQL function set_backend_priority(). Normal users may increase the priority value of any backend process running under the same username. Superusers may increase the priority value of any backend process. Just like with using "renice" manually, it is not possible to lower a backend's priority value, since PostgreSQL will not be running as the "root" user.<br />
<br />
If you know your application will be running an unimportant CPU-heavy query, you could have it call set_backend_priority(pg_backend_pid(), 20) after installing the "prioritize" module, so that the process is scheduled for the lowest possible urgency.<br />
<br />
== Prioritizing I/O ==<br />
<br />
I/O is harder. Some operating systems offer I/O priorities for<br />
processes, like Linux's ionice, and you'd think you could use these in a<br />
similar way to how you use 'nice'. Unfortunately, that won't work particularly well,<br />
because a lot of the work PostgreSQL does - especially disk writes - are<br />
done via a separate background writer process working from memory shared<br />
by all backends. Similarly, the write-ahead logs are managed by their<br />
own process via shared memory. Because of those two, it's very hard to effectively give one<br />
user priority over another for writes. ionice should be moderately<br />
effective for reads, though.<br />
<br />
As with "nice", effective control on a per-connection level will require the addition of appropriate helper<br />
functions, and user co-operation is required to achieve per-user priorities.<br />
<br />
Better separation of I/O workloads will require [[Prioritizing_databases_by_separating_into_multiple_clusters|cluster separation]], which has its own costs and is only effective on the per-database level.<br />
<br />
== Prioritizing memory ==<br />
<br />
PostgreSQL does have some [[Tuning_Your_PostgreSQL_Server|tunable parameters]] for memory use that are per-client, particularly <code>[http://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-WORK-MEM work_mem]</code> and <code>[http://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM maintenance_work_mem]</code>. These may be set within a given connection to allow that backend to use more than the usual amount of memory for things like sorts and index creation. You can set these to conservative, low values in <code>postgresql.conf</code> then use the <code>SET</code> command to assign higher values to them for a particular backend, eg <code>SET work_mem = '100MB';</code>.<br />
<br />
You can set different values for <code>work_mem</code> and <code>maintenance_work_mem</code> using per-user GUC variables. For example:<br />
<br />
<pre><br />
ALTER USER myuser SET work_mem = '50MB';<br />
</pre><br />
<br />
You cannot affect the shared memory allocation done with settings like shared_buffers this way, that value is fixed at database startup time and can't be changed without restarting it.<br />
<br />
There's no easy way in most operating systems to prioritize memory allocations, so that for example the OS would prefer to swap one backend's memory out instead of another's.<br />
<br />
== External links ==<br />
<br />
* [http://www.cs.cmu.edu/~harchol/Papers/actual-icde-submission.pdf CMU article studying CPU priorities on Postgres and DB2 on TPC-C and TPC-W workloads]<br />
<br />
== Credits ==<br />
Page initially by [[User:Ringerc|Ringerc]] 02:34, 26 November 2009 (UTC)<br />
<br />
[[Category:FAQ]] [[Category:Performance]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Index_Maintenance&diff=14839Index Maintenance2011-07-06T21:09:47Z<p>Schmiddy: Per suggestion from John Pierce, the second query will not work without explict casts to text</p>
<hr />
<div>One day, you will probably need to cope with [http://www.postgresql.org/docs/current/static/routine-reindex.html routine reindexing] on your database, particularly if you don't use VACUUM aggressively enough. A particularly handy command in this area is [http://www.postgresql.org/docs/8.3/static/sql-cluster.html CLUSTER], which can help with other types of cleanup.<br />
<br />
Avoid using [[VACUUM FULL]].<br />
<br />
== Index summary ==<br />
<br />
Here's a sample query to pull the number of rows, indexes, and some info about those indexes for each table. (Only works on 8.3; ditch the pg_size_pretty if you’re on an earlier version)<br />
<br />
{{SnippetInfo|Index summary|lang=SQL|version=>=8.1|category=Performance}}<br />
<source lang="sql"><br />
SELECT<br />
pg_class.relname,<br />
pg_size_pretty(pg_class.reltuples::bigint) AS rows_in_bytes,<br />
pg_class.reltuples AS num_rows,<br />
count(indexname) AS number_of_indexes,<br />
CASE WHEN x.is_unique = 1 THEN 'Y'<br />
ELSE 'N'<br />
END AS UNIQUE,<br />
SUM(case WHEN number_of_columns = 1 THEN 1<br />
ELSE 0<br />
END) AS single_column,<br />
SUM(case WHEN number_of_columns IS NULL THEN 0<br />
WHEN number_of_columns = 1 THEN 0<br />
ELSE 1<br />
END) AS multi_column<br />
FROM pg_namespace <br />
LEFT OUTER JOIN pg_class ON pg_namespace.oid = pg_class.relnamespace<br />
LEFT OUTER JOIN<br />
(SELECT indrelid,<br />
max(CAST(indisunique AS integer)) AS is_unique<br />
FROM pg_index<br />
GROUP BY indrelid) x<br />
ON pg_class.oid = x.indrelid<br />
LEFT OUTER JOIN<br />
( SELECT c.relname AS ctablename, ipg.relname AS indexname, x.indnatts AS number_of_columns FROM pg_index x<br />
JOIN pg_class c ON c.oid = x.indrelid<br />
JOIN pg_class ipg ON ipg.oid = x.indexrelid )<br />
AS foo<br />
ON pg_class.relname = foo.ctablename<br />
WHERE <br />
pg_namespace.nspname='public'<br />
AND pg_class.relkind = 'r'<br />
GROUP BY pg_class.relname, pg_class.reltuples, x.is_unique<br />
ORDER BY 2;<br />
</source><br />
<br />
== Index size/usage statistics ==<br />
<br />
Table & index sizes along which indexes are being scanned and how many tuples are fetched. See [[Disk Usage]] for another view that includes both table and index sizes.<br />
<br />
{{SnippetInfo|Index statistics|lang=SQL|version=>=8.1|category=Performance}}<br />
<source lang="sql"><br />
SELECT<br />
t.tablename,<br />
indexname,<br />
c.reltuples AS num_rows,<br />
pg_size_pretty(pg_relation_size(t.tablename::text)) AS table_size,<br />
pg_size_pretty(pg_relation_size(indexrelname::text)) AS index_size,<br />
CASE WHEN x.is_unique = 1 THEN 'Y'<br />
ELSE 'N'<br />
END AS unique,<br />
idx_scan AS number_of_scans,<br />
idx_tup_read AS tuples_read,<br />
idx_tup_fetch AS tuples_fetched<br />
FROM pg_tables t<br />
LEFT OUTER JOIN pg_class c ON t.tablename=c.relname<br />
LEFT OUTER JOIN<br />
(SELECT indrelid,<br />
max(CAST(indisunique AS integer)) AS is_unique<br />
FROM pg_index<br />
GROUP BY indrelid) x<br />
ON c.oid = x.indrelid<br />
LEFT OUTER JOIN<br />
( SELECT c.relname as ctablename, ipg.relname as indexname, x.indnatts as number_of_columns, idx_scan, idx_tup_read, idx_tup_fetch,indexrelname FROM pg_index x<br />
JOIN pg_class c ON c.oid = x.indrelid<br />
JOIN pg_class ipg ON ipg.oid = x.indexrelid<br />
JOIN pg_stat_all_indexes psai ON x.indexrelid = psai.indexrelid )<br />
as foo<br />
ON t.tablename = foo.ctablename<br />
WHERE t.schemaname='public'<br />
order by 1,2;<br />
</source><br />
<br />
== Index Bloat ==<br />
<br />
One of the common needs for a REINDEX is when indexes become bloated due to either sparse deletions or use of VACUUM FULL. An estimator for the amount of bloat in a table has been included in the [http://bucardo.org/wiki/Check_postgres check_postgres] script, which you can call directly or incorporate into a larger monitoring system. Scripts based on this code and/or its concepts from other sources include:<br />
* [http://pgsql.tapoueh.org/site/html/news/20080131.bloat.html bloat view] (Dimitri Fontaine)<br />
* [http://www.pgcon.org/2009/schedule/events/153.en.html Visualizing Postgres] - index_byte_sizes view (Michael Glaesemann, myYearbook)<br />
* [http://labs.omniti.com/trac/pgtreats/browser/trunk/tools OmniTI Tasty Treats for PostgreSQL] - shell and Perl pg_bloat_report scripts<br />
<br />
== Unused Indexes ==<br />
<br />
Since indexes add significant overhead to any table change operation, they should be removed if they are not being used for either queries or constraint enforcement (such as making sure a value is unique). How to find such indexes:<br />
<br />
* [http://www.xzilla.net/blog/2008/Jul/Index-pruning-techniques.html Index pruning techniques]<br />
* [http://hype-free.blogspot.com/2008/09/finding-unused-indexes-in-postgresql.html Finding unused indexes]<br />
* [http://it.toolbox.com/blogs/database-soup/finding-useless-indexes-28796 Finding useless indexes]<br />
* [http://radek.cc/2009/09/05/psqlrc-tricks-indexes/ Missing and unused indexes]<br />
<br />
== References ==<br />
<br />
* Index statistics queries from [http://www.baconandtech.com/2009/06/06/book-review-part-i-refactoring-sql-applications-with-bonus-queries/ "Refactoring SQL Applications" review]<br />
<br />
[[Category:Administration]][[Category:Performance]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Todo&diff=14508Todo2011-05-25T01:36:12Z<p>Schmiddy: /* psql */ The \dd command was missing more than just constraint comments, add links to relevant discussions</p>
<hr />
<div><div style="margin: 1ex 1em; float: right;"><br />
__TOC__<br />
</div><br />
<br />
This list contains '''all known PostgreSQL bugs and feature requests'''. If you would like to work on an item, please read the [[Developer FAQ]] first. There is also a [[Development_information|development information page]].<br />
<br />
* {{TodoPending}} - marks ordinary, incomplete items<br />
* {{TodoEasy}} - marks items that are easier to implement<br />
* {{TodoDone}} - marks changes that are done, and will appear in the PostgreSQL 9.1 release.<br />
<br />
For help on editing this list, please see [[Talk:Todo]]. <b>Please do not add items here without discussion on the mailing list.</b><br />
<br />
<div style="padding: 1ex 4em;"><br />
== Administration ==<br />
<br />
{{TodoItem<br />
|Allow administrators to cancel multi-statement idle transactions<br />
|This allows locks to be released, but it is complex to report the cancellation back to the client.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01340.php <nowiki>Cancelling idle in transaction state</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00441.php <nowiki>Re: Cancelling idle in transaction state</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Check for unreferenced table files created by transactions that were in-progress when the server terminated abruptly<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00096.php <nowiki>Removing unreferenced files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Set proper permissions on non-system schemas during db creation<br />
|Currently all schemas are owned by the super-user because they are copied from the template1 database. However, since all objects are inherited from the template database, it is not clear that setting schemas to the db owner is correct.}}<br />
<br />
{{TodoItem<br />
|Allow log_min_messages to be specified on a per-module basis<br />
|This would allow administrators to see more detailed information from specific sections of the backend, e.g. checkpoints, autovacuum, etc. Another idea is to allow separate configuration files for each module, or allow arbitrary SET commands to be passed to them. See also [[Logging Brainstorm]].}}<br />
<br />
{{TodoItem<br />
|Simplify creation of partitioned tables<br />
|This would allow creation of partitioned tables without requiring creation of triggers or rules for INSERT/UPDATE/DELETE, and constraints for rapid partition selection. Options could include range and hash partition selection. See also [[Table partitioning]]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow auto-selection of partitioned tables for min/max() operations<br />
|There was a patch on -hackers from July 2009, but it has not been merged: [http://archives.postgresql.org/pgsql-hackers/2009-07/msg01115.php <nowiki>MIN/MAX optimization for partitioned table</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Allow custom variables to appear in pg_settings()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00850.php <nowiki>Re: count(*) performance improvement ideas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have custom variables be transaction-safe<br />
* {{MessageLink|4B577E9F.8000505@dunslane.net|Custom GUCs still a bit broken}}<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the SQL-standard mechanism whereby REVOKE ROLE revokes only the privilege granted by the invoking role, and not those granted by other roles<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00010.php <nowiki>Re: Grantor name gets lost when grantor role dropped</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Improve server security options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01875.php <nowiki>Re: [0/4] Proposal of SE-PostgreSQL patches</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00000.php <nowiki>Re: [0/4] Proposal of SE-PostgreSQL patches</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent query cancel packets from being replayed by an attacker, especially when using SSL<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00345.php <nowiki>Replay attack of query cancel</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a way to query the log collector subprocess to determine the name of the currently active log file<br />
* [http://archives.postgresql.org/pgsql-general/2008-11/msg00418.php <nowiki>Current log files when rotating?</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow the client to authenticate the server in a Unix-domain socket connection, e.g., using SO_PEERCRED<br />
* http://archives.postgresql.org/message-id/20090401173756.GB21229@svana.org<br />
}}<br />
<br />
{{TodoItem<br />
|Allow simpler reporting of the unix domain socket directory and allow easier configuration of its default location<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg01555.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow custom daemons to be automatically stopped/started along with the postmaster<br />
|This allows easier administration of daemons like user job schedulers or replication-related daemons.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01701.php <nowiki>Re: scheduler in core</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Increase maximum values for max_standby_streaming_delay and log_min_duration_statement<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg01517.php<br />
* Committed: http://archives.postgresql.org/pgsql-committers/2011-03/msg00210.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve logging of prepared transactions recovered during startup<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00092.php <nowiki>&quot;recovering prepared transaction&quot; after server restart message</nowiki>]<br />
}}<br />
<br />
=== Configuration files ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Allow pg_hba.conf to specify host names along with IP addresses<br />
|Host name lookup could occur when the postmaster reads the pg_hba.conf file, or when the backend starts. Another solution would be to reverse lookup the connection IP and check that hostname against the host names in pg_hba.conf. We could also then check that the host name maps to the IP address. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00569.php <nowiki>TODO Item: Allow pg_hba.conf to specify host names along with IP addresses</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00613.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf file values to be changed via an SQL API, perhaps using SET GLOBAL<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00764.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider normalizing fractions in postgresql.conf, perhaps using '%'<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00550.php <nowiki>Fractions in GUC variables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow Kerberos to disable stripping of realms so we can check the username@realm against multiple realms<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00009.php <nowiki>krb_match_realm patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve LDAP authentication configuration options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01745.php <nowiki>Proposed Patch - LDAPS support for servers on port 636 w/o TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add external tool to auto-tune some postgresql.conf parameters<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00000.php <nowiki>Re: Overhauling GUCS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00033.php <nowiki>Simple postgresql.conf wizard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add 'hostgss' pg_hba.conf option to allow GSS link-level encryption<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01454.php <nowiki>Re: Plans for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Process pg_hba.conf keywords as case-insensitive<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00432.php <nowiki>More robust pg_hba.conf parsing/error logging</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Have pg_hba.conf consider "replication" special only in the database field<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00632.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Rename unix domain socket 'ident' connections to 'peer', to avoid confusion with TCP 'ident'<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01053.php<br />
}}<br />
<br />
{{TodoItem<br />
|Create utility to compute accurate random_page_cost value}}<br />
<br />
{{TodoItem<br />
|Allow configuration files to be independently validated<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01831.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf settings to be accepted by backends even if some settings are invalid for those backends<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00330.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00375.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow all backends to receive postgresql.conf setting changes at the same time<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00330.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00375.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Tablespaces ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow a database in tablespace t1 with tables created in tablespace t2 to be used as a template for a new database created with default tablespace t2<br />
|Currently all objects in the default database tablespace must have default tablespace specifications. This is because new databases are created by copying directories. If you mix default tablespace tables and tablespace-specified tables in the same directory, creating a new database from such a mixed directory would create a new database with tables that had incorrect explicit tablespaces. To fix this would require modifying pg_class in the newly copied database, which we don't currently do.}}<br />
<br />
{{TodoItem<br />
|Allow reporting of which objects are in which tablespaces<br />
|This item is difficult because a tablespace can contain objects from multiple databases. There is a server-side function that returns the databases which use a specific tablespace, so this requires a tool that will call that function and connect to each database to find the objects in each database for that tablespace.}}<br />
<br />
{{TodoItem<br />
|Allow WAL replay of CREATE TABLESPACE to work when the directory structure on the recovery computer is different from the original}}<br />
<br />
{{TodoItem<br />
|Allow per-tablespace quotas}}<br />
<br />
{{TodoItem<br />
|Allow tablespaces on RAM-based partitions for unlogged tables<br />
* http://archives.postgresql.org/pgsql-advocacy/2011-05/msg00033.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow toast tables to be moved to a different tablespace<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00980.php<br />
}}<br />
<br />
<br />
<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Statistics Collector ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow statistics last vacuum/analyze execution times to be displayed without requiring track_counts to be enabled<br />
* [http://archives.postgresql.org/pgsql-docs/2007-04/msg00028.php <nowiki>row-level stats and last analyze time</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Clear table counters on TRUNCATE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00169.php <nowiki>Small TRUNCATE glitch</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Allow the clearing of cluster-level statistics<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-03/msg00917.php <nowiki>Resetting cluster-wide statistics</nowiki>]<br />
* ''pg_stat_reset_shared('bgwriter')'' (9.0) now handles the ''pg_stat_bgwriter'' subset of this<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SSL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SSL authentication/encryption over unix domain sockets<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00924.php <nowiki>Re: Spoofing as the postmaster</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL key file permission checks to be optionally disabled when sharing SSL keys with other applications<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php <nowiki>BUG #3809: SSL &quot;unsafe&quot; private key permissions bug</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL CRL files to be re-read during configuration file reload, rather than requiring a server restart<br />
|Unlike SSL CRT files, CRL (Certificate Revocation List) files are updated frequently<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00832.php <nowiki>Automatic CRL reload</nowiki>]<br />
Alternatively or additionally supporting OCSP (online certificate security protocol) would provide real-time revocation discovery without reloading<br />
}}<br />
<br />
{{TodoItem<br />
| Allow automatic selection of SSL client certificates from a certificate store<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00406.php <nowiki>Allow multiple certificates or keys in the postgresql.crt/.key files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Send the full certificate server chain to the client<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-12/msg00145.php BUG #5245: Full Server Certificate Chain Not Sent to client]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Point-In-Time Recovery (PITR) ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Create dump tool for write-ahead logs for use in determining transaction id for point-in-time recovery<br />
|This is useful for checking PITR recovery.}}<br />
<br />
{{TodoItemDone<br />
|Allow recovery.conf to support the same syntax as postgresql.conf, including quoting<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00497.php <nowiki>recovery.conf parsing problems</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg00684.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow archive_mode to be changed without server restart?<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01655.php <nowiki>Enabling archive_mode without restart</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider avoiding WAL switching via archive_timeout if there has been no database activity<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01469.php <nowiki>archive_timeout behavior for no activity</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00395.php <nowiki>Re: archive_timeout behavior for no activity</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Expose pg_controldata via an SQL interface<br />
|Helpful for monitoring replicated databases<br />
* http://archives.postgresql.org/message-id/4B901D73.8030003@agliodbs.com<br />
* [http://archives.postgresql.org/message-id/4B959D7A.6010907@joeconway.com initial patch]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Standby server mode ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
| Allow pg_xlogfile_name() to be used in recovery mode<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1001190135vd9f62f1sa7868abc1ea61d12@mail.gmail.com <nowiki>Streaming replication and pg_xlogfile_name()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Prevent variables inherited from the server environment from begin used for making streaming replication connections.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01011.php <nowiki>Re: Parameter name standby_mode</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Add a new privilege for connecting for streaming replication<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1003040247p6b092241of91784a505e9abd8@mail.gmail.com <nowiki>Streaming replication and privilege</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Add support for synchronous replication.<br />
}}<br />
<br />
{{TodoItemDone<br />
| Add capability to take and send a base backup over the streaming replication connection, making it possible to initialize a new standby server from a running primary server without a WAL archive or other access to the primary server. <br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00136.php<br />
}}<br />
<br />
{{TodoItem<br />
| Allow hot file system backups on standby servers<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01727.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php<br />
}}<br />
<br />
{{TodoItemDone<br />
| Allow the automatic removal of old directories when streaming base backups<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00558.php<br />
}}<br />
<br />
{{TodoItem<br />
| Change walsender so that it applies per-role settings<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00642.php<br />
}}<br />
<br />
{{TodoItem<br />
| Add more control over waiting for synchronous commit<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01611.php<br />
}}<br />
<br />
{{TodoItem<br />
| Restructure configuration parameters for standby mode<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01820.php<br />
}}<br />
<br />
{{TodoItem<br />
| Allow time-delayed application of logs on the standby<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00992.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Data Types ==<br />
<br />
{{TodoItemDone<br />
|Reduce storage space for small NUMERICs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01331.php <nowiki>Saving space for common kinds of numeric values</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-02/msg00505.php <nowiki>Numeric patch to add special-case representations for &lt; 8 bytes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php <nowiki>Re: Reducing NUMERIC size for 8.3</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix data types where equality comparison is not intuitive, e.g. box}}<br />
<br />
{{TodoItem<br />
|Add support for public SYNONYMs<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00519.php <nowiki>Proposal for SYNONYMS</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php<br />
* http://archives.postgresql.org/pgsql-general/2010-12/msg00139.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for SQL-standard GENERATED/IDENTITY columns<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg00543.php <nowiki>Re: Three weeks left until feature freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00038.php <nowiki>GENERATED ... AS IDENTITY, Was: Re: Feature Freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00344.php <nowiki>Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00076.php <nowiki>Re: [HACKERS] Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00604.php <nowiki>IDENTITY/GENERATED patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider placing all sequences in a single table, or create a system view<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a special data type for regular expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg01067.php <nowiki>Why is there a tsquery data type?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce BIT data type overhead using short varlena headers<br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00273.php <nowiki>storage size of &quot;bit&quot; data type..</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow adding enumerated values to an existing enumerated data type<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01718.php <nowiki>Re: [COMMITTERS] pgsql: Update: &lt; * Allow adding enumerated values to an existing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow renaming and deleting enumerated values from an existing enumerated data type<br />
}}<br />
<br />
{{TodoItem<br />
|Support scoped IPv6 addresses in the inet type<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00111.php <nowiki>strange problem with ip6</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a JSON (JavaScript Object Notation) data type<br />
|This would behave similar to the XML data type, which is stored as text, but allows element lookup and conversion functions.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01494.php <nowiki>PATCH: Add hstore_to_json()</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg00001.php <nowiki>Re: PATCH: Add hstore_to_json()</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg01092.php <nowiki>Proposal: Add JSON support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00057.php <nowiki>Re: Proposal: Add JSON support</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00481.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01694.php<br />
}}<br />
<br />
{{TodoItem<br />
|Considering improving performance of computing CHAR() value lengths<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00900.php <nowiki>char() overhead on read-only workloads not so insignifcant as the docs claim it is...</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01787.php <nowiki>Re: [PATCH] backend: compare word-at-a-time in bcTruelen</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add overlaps geometric operators that ignore point overlaps<br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00861.php<br />
}}<br />
<br />
=== Domains ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow functions defined as casts to domains to be called during casting<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-05/msg00072.php <nowiki>bug? non working casts for domain</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg01681.php <nowiki>TODO: Fix CREATE CAST on DOMAINs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow values to be cast to domain types<br />
* [http://archives.postgresql.org/pgsql-hackers/2003-06/msg01206.php <nowiki>Domain casting still doesn't work right</nowiki>] <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00289.php <nowiki>domain casting?</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00812.php<br />
}}<br />
<br />
{{TodoItem<br />
|Make domains work better with polymorphic functions<br />
* [http://archives.postgresql.org/message-id/4887.1228700773@sss.pgh.pa.us Polymorphic types vs. domains]<br />
* [http://archives.postgresql.org/message-id/15535.1238774571@sss.pgh.pa.us some difficulties with fixing it]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Dates and Times ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow infinite intervals just like infinite timestamps}}<br />
<br />
{{TodoItem<br />
|Allow TIMESTAMP WITH TIME ZONE to store the original timezone information, either zone name or offset from UTC<br />
|If the TIMESTAMP value is stored with a time zone name, interval computations should adjust based on the time zone rules. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-10/msg00705.php <nowiki>timestamp with time zone a la sql99</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have timestamp subtraction not call justify_hours()?<br />
* [http://archives.postgresql.org/pgsql-sql/2006-10/msg00059.php <nowiki>timestamp subtraction (was Re: formatting intervals with to_char)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve TIMESTAMP WITH TIME ZONE subtraction to be DST-aware<br />
|Currently subtracting one date from another that crosses a daylight savings time adjustment can return '1 day 1 hour', but adding that back to the first date returns a time one hour in the future. This is caused by the adjustment of '25 hours' to '1 day 1 hour', and '1 day' is the same time the next day, even if daylight savings adjustments are involved.}}<br />
<br />
{{TodoItem<br />
|Fix interval display to support values exceeding 2^31 hours}}<br />
<br />
{{TodoItem<br />
|Add overflow checking to timestamp and interval arithmetic}}<br />
<br />
{{TodoItem<br />
|Add function to allow the creation of timestamps using parameters<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00232.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Arrays ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add support for arrays of domains<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00114.php <nowiki>Re: updated WIP: arrays of composites</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single-byte header storage for array elements}}<br />
<br />
{{TodoItem<br />
|Add function to detect if an array is empty<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00475.php <nowiki>Re: array_length()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of empty arrays<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01033.php <nowiki>So what's an &quot;empty&quot; array anyway?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULLs in arrays<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-11/msg00009.php <nowiki>BUG #4509: array_cat's null behaviour is inconsistent</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01040.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Binary Data ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve vacuum of large objects, like contrib/vacuumlo?}}<br />
<br />
{{TodoItem<br />
|Auto-delete large objects when referencing row is deleted<br />
|contrib/lo offers this functionality.}}<br />
<br />
{{TodoItem<br />
|Allow read/write into TOAST values like large objects<br />
|This requires the TOAST column to be stored EXTERNAL.}}<br />
<br />
{{TodoItem<br />
|Add API for 64-bit large object access<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00781.php <nowiki>64-bit API for large objects</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01790.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== MONEY Data Type ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add locale-aware MONEY type, and support multiple currencies<br />
* [http://archives.postgresql.org/pgsql-general/2005-08/msg01432.php <nowiki>A real currency type</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01181.php <nowiki>Money type todos?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|MONEY dumps in a locale-specific format making it difficult to restore to a system with a different locale}}<br />
<br />
{{TodoItemDone<br />
|Allow MONEY to be easily cast to/from other numeric data types}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Text Search ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dictionaries to change the token that is passed on to later dictionaries<br />
* [http://archives.postgresql.org/pgsql-patches/2007-11/msg00081.php <nowiki>a tsearch2 (8.2.4) dictionary that only filters out stopwords</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a function-based API for '@@' searches<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00511.php <nowiki>Simplifying Text Search</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve text search error messages<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00966.php <nowiki>Poorly designed tsearch NOTICEs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01146.php <nowiki>Re: Poorly designed tsearch NOTICEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing error to warning for strings larger than one megabyte<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00190.php <nowiki>BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00062.php <nowiki>Re: [BUGS] BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|tsearch and tsdicts regression tests fail in Turkish locale on glibc<br />
* [http://archives.postgresql.org/message-id/49749645.5070801@gmx.net tsearch with Turkish locale]<br />
}}<br />
<br />
{{TodoItem<br />
|tsquery negator operator treated as part of lexeme<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-06/msg00346.php BUG #4887: inclusion operator (@>) on tsqeries behaves not conforming to documentation]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of plus signs in email address user names, and perhaps improve URL parsing<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00772.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve default parser, to more easily allow adding new tokens<br />
* http://archives.postgresql.org/message-id/23485.1297727826@sss.pgh.pa.us<br />
}}<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== XML ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow XML arrays to be cast to other data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00981.php <nowiki>proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00231.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00471.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add XML Schema validation and xmlvalidate functions (SQL:2008)}}<br />
<br />
{{TodoItem<br />
|Add xmlvalidatedtd variant to support validating against a DTD?}}<br />
<br />
{{TodoItem<br />
|Relax-NG validation; libxml2 supports this already}}<br />
<br />
{{TodoItem<br />
|Allow reliable XML operation non-UTF8 server encodings (xpath(), in particular, is known to not work)<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-01/msg00135.php <nowiki>BUG #4622: xpath only work in utf-8 server encoding</nowiki>] <br />
* http://archives.postgresql.org/message-id/4110.1238973350@sss.pgh.pa.us}}<br />
<br />
{{TodoItem<br />
|Add functions from SQL:2006: XMLDOCUMENT, XMLCAST, XMLTEXT}}<br />
<br />
{{TodoItem<br />
|Add XMLNAMESPACES support in XMLELEMENT and elsewhere}}<br />
<br />
{{TodoItem<br />
|Move XSLT from contrib/xml2 to a more reasonable location<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00539.php<br />
}}<br />
<br />
{{TodoItem<br />
|Report errors returned by the XSLT library<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00562.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve the XSLT parameter passing API<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00416.php<br />
}}<br />
<br />
{{TodoItem<br />
|XML Canonical: Convert XML documents to canonical form to compare them. libxml2 has support for this.}}<br />
<br />
{{TodoItem<br />
|Add pretty-printed XML output option<br />
|Parse a document and serialize it back in some indented form. libxml2 might support this.}}<br />
<br />
{{TodoItem<br />
|Add XMLQUERY (from the SQL/XML standard)}}<br />
<br />
{{TodoItem<br />
|Allow XML sthredding<br />
|In some cases shredding could be better option (if there is no need to keep XML docs entirely, e.g. if we have already developed tools that understand only relational data. This would be a separate module that implements annotated schema decomposition technique, similar to DB2 and SQL Server functionality.}}<br />
<br />
{{TodoItem<br />
|Fix Nested or repeated xpath() that apparently mess up namespaces [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00097.php] [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00144.php] [http://archives.postgresql.org/pgsql-general/2008-03/msg00295.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/message-id/004f01c90e91$138e9d10$3aabd730$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItem<br />
|XPath: Adding the <x> at the root causes problems [http://archives.postgresql.org/pgsql-bugs/2008-05/msg00184.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/pgsql-general/2008-07/msg00613.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table needs to be implemented/implementable to get rid of contrib/xml2 [http://archives.postgresql.org/pgsql-general/2008-05/msg00823.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table is pretty broken anyway [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02424.php]}}<br />
<br />
{{TodoItem<br />
|better handling of XPath data types [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00616.php] [http://archives.postgresql.org/message-id/004a01c90e90$4b986d90$e2c948b0$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItemDone<br />
|xpath_exists() is needed<br />
|This checks whether or not the path specified exists in the XML value. Without this function we need to use the weird "array_dims(xpath(...)) IS NOT NULL" syntax.}}<br />
<br />
{{TodoItem<br />
|Improve handling of PIs and DTDs in xmlconcat() [http://archives.postgresql.org/message-id/200904211211.n3LCB09p008988@wwwmaster.postgresql.org]}}<br />
<br />
{{TodoItem<br />
|Restructure XML and /contrib/xml2 functionality<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02314.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00017.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Functions ==<br />
<br />
{{TodoItem<br />
|Allow INET subnet comparisons using non-constants to be indexed}}<br />
<br />
{{TodoItem<br />
|Add an INET overlaps operator, for use by exclusion constraints <br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00845.php<br />
}}<br />
<br />
{{TodoItem<br />
|Enforce typmod for function inputs, function results and parameters for spi_prepare'd statements called from PLs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01403.php <nowiki>Re: BUG #2917: spi_prepare doesn't accept typename aliases</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01160.php <nowiki>RFC for adding typmods to functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix IS OF so it matches the ISO specification, and add documentation<br />
* [http://archives.postgresql.org/pgsql-patches/2003-08/msg00060.php <nowiki>Re: [HACKERS] IS OF</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00060.php <nowiki>ToDo: add documentation for operator IS OF</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement Boyer-Moore searching in LIKE queries<br />
* {{messageLink|27645.1220635769@sss.pgh.pa.us|TODO item: Implement Boyer-Moore searching (First time hacker)}}<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent malicious functions from being executed with the permissions of unsuspecting users<br />
|Index functions are safe, so VACUUM and ANALYZE are safe too. Triggers, CHECK and DEFAULT expressions, and rules are still vulnerable. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00268.php <nowiki>Some notes about the index-functions security vulnerability</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce memory usage of aggregates in set returning functions<br />
* [http://archives.postgresql.org/pgsql-performance/2008-01/msg00031.php <nowiki>Re: Performance of aggregates over set-returning functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/ltree operator<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-11/msg00044.php <nowiki>BUG #3720: wrong results at using ltree</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/btree_gist's implementation of inet indexing<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-10/msg00099.php <nowiki>BUG #5705: btree_gist: Index on inet changes query result</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|<nowiki>Fix inconsistent precedence of =, &gt;, and &lt; compared to &lt;&gt;, &gt;=, and &lt;=</nowiki><br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00145.php <nowiki>BUG #3822: Nonstandard precedence for comparison operators</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix regular expression bug when using complex back-references<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00000.php <nowiki>BUG #3645: regular expression back references seem broken</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have /contrib/dblink reuse unnamed connections<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00895.php <nowiki>dblink un-named connection doesn't get re-used</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve formatting of pg_get_viewdef() output<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01648.php <nowiki>pg_get_viewdef formattiing</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01885.php <nowiki>Re: pretty print viewdefs</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add printf()-like functionality<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add function to dump pg_depend information cleanly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00226.php <nowiki>Elementary dependency look-up</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve relation size functions such as pg_relation_size() to avoid producing an error when called against a no longer visible relation<br />
* [http://archives.postgresql.org/message-id/28488.1286461610@sss.pgh.pa.us pg_relation_size / could not open relation with OID #]<br />
}}<br />
<br />
=== Character Formatting ===<br />
<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Allow to_date() and to_timestamp() to accept localized month names}}<br />
<br />
{{TodoItem<br />
|Add missing parameter handling in to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg00948.php <nowiki>Re: to_char and i18n</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Throw an error from to_char() instead of printing a string of "#" when a number doesn't fit in the desired output format.<br />
* discussed in [http://archives.postgresql.org/message-id/37ed240d0907290836w42187222n18664dfcbcb445b1@mail.gmail.com "to_char, support for EEEE format"]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow to_char() on interval values to accumulate the highest unit requested<br />
|2= Some special format flag would be required to request such accumulation. Such functionality could also be added to EXTRACT. Prevent accumulation that crosses the month/day boundary because of the uneven number of days in a month.<br />
* to_char(INTERVAL '1 hour 5 minutes', 'MI') =&gt; 65<br />
* to_char(INTERVAL '43 hours 20 minutes', 'MI' ) =&gt; 2600<br />
* to_char(INTERVAL '43 hours 20 minutes', 'WK:DD:HR:MI') =&gt; 0:1:19:20<br />
* to_char(INTERVAL '3 years 5 months','MM') =&gt; 41<br />
}}<br />
<br />
{{TodoItem<br />
|Fix to_number() handling for values not matching the format string<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01447.php <nowiki>Re: numeric_to_number() function skipping some digits</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Multi-Language Support ==<br />
<br />
{{TodoItem<br />
|Add NCHAR (as distinguished from ordinary varchar),}}<br />
<br />
{{TodoItemDone<br />
|Allow more fine-grained collation selection<br />
|Right now the collation is fixed at database creation time.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-03/msg00932.php <nowiki>Re: Patch for collation using ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-08/msg00039.php <nowiki>FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-08/msg00309.php <nowiki>Re: FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00110.php <nowiki>Proof of concept COLLATE support with patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-09/msg00020.php <nowiki>For review: Initial support for COLLATE</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01121.php <nowiki>Proposed COLLATE implementation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-01/msg00767.php <nowiki>TODO item: locale per database patch (new iteration)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-03/msg00233.php <nowiki>Re: FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg00662.php <nowiki>Re: Fixed length data types issue</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00557.php <nowiki>[WIP] collation support revisited (phase 1)</nowiki>]<br />
* [[Todo:Collate]]<br />
* [[Todo:ICU]]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01362.php <nowiki>WIP patch: Collation support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00012.php <nowiki>Re: WIP patch: Collation support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00868.php <nowiki>PGDay.it collation discussion notes</nowiki>]<br />
* [http://www.unicode.org/unicode/reports/tr10/ Unicode collation algorithm]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a cares-about-collation column to pg_proc, so that unresolved-collation errors can be thrown at parse time<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-03/msg01520.php <nowiki>Open issues for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Integrate collations with text search configurations<br />
* [http://archives.postgresql.org/message-id/28887.1303579034@sss.pgh.pa.us <nowiki>Some TODO items for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Integrate collations with to_char() and related functions<br />
* [http://archives.postgresql.org/message-id/28887.1303579034@sss.pgh.pa.us <nowiki>Some TODO items for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a LOCALE option to CREATE DATABASE, as a shorthand<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00119.php <nowiki> Re: 8.4 open items list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support multiple simultaneous character sets, per SQL:2008}}<br />
<br />
{{TodoItem<br />
|Improve UTF8 combined character handling?}}<br />
<br />
{{TodoItem<br />
|Add octet_length_server() and octet_length_client()}}<br />
<br />
{{TodoItem<br />
|Make octet_length_client() the same as octet_length()?}}<br />
<br />
{{TodoItem<br />
|Fix problems with wrong runtime encoding conversion for NLS message files}}<br />
<br />
{{TodoItem<br />
|Add URL to more complete multi-byte regression tests<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-07/msg00272.php <nowiki>Multi-byte and client side character encoding tests for copy command..</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix contrib/fuzzystrmatch to work with multibyte encodings<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00047.php <nowiki> soundex function returns UTF-16 characters</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00138.php <nowiki> dmetaphone woes</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Set client encoding based on the client operating system encoding<br />
|Currently client_encoding is set in postgresql.conf, which defaults to the server encoding. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg01696.php <nowiki>Re: [GENERAL] invalid byte sequence ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Change memory allocation for multi-byte functions so memory is allocated inside conversion functions<br />
|Currently we preallocate memory based on worst-case usage.}}<br />
<br />
{{TodoItem<br />
|Add ability to use case-insensitive regular expressions on multi-byte characters<br />
|Currently it works for UTF-8, but not other multi-byte encodings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00433.php <nowiki>Regexps vs. locale</nowiki>]<br />
* {{MessageLink|20091201210024.B1393753FB7@cvs.postgresql.org|A partial solution for UTF-8}}<br />
}}<br />
<br />
{{TodoItem<br />
|Improve encoding of connection startup messages sent to the client<br />
|Currently some authentication error messages are sent in the server encoding<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00801.php <nowiki>encoding of PostgreSQL messages</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-01/msg00005.php <nowiki>Re: encoding of PostgreSQL messages</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have pg_stat_activity display query strings in the correct client encoding<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00131.php <nowiki>pg_stats queries versus per-database encodings</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|More sensible support for Unicode combining characters, normal forms<br />
* http://archives.postgresql.org/message-id/200904141532.44618.peter_e@gmx.net<br />
}}<br />
<br />
== Views / Rules ==<br />
<br />
{{TodoItem<br />
|Automatically create rules on views so they are updateable, per SQL:2008<br />
|We can only auto-create rules for simple views. For more complex cases users will still have to write rules manually.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00586.php <nowiki>Proposal for updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-08/msg00255.php <nowiki>Updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01746.php <nowiki>Re: [COMMITTERS] pgsql: Automatic view update rules Bernd Helmle</nowiki>]<br />
* http://wiki.postgresql.org/wiki/Updatable_views<br />
}}<br />
<br />
{{TodoItem<br />
|Add the functionality of the WITH CHECK OPTION clause to CREATE VIEW}}<br />
<br />
{{TodoItem<br />
|Allow VIEW/RULE recompilation when the underlying tables change<br />
|This is both difficult and controversial.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01723.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01724.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
}}<br />
<br />
{{TodoItem<br />
|Make it possible to use RETURNING together with conditional DO INSTEAD rules, such as for partitioning setups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00577.php <nowiki>RETURNING and DO INSTEAD ... Intentional or not?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add the ability to automatically create materialized views<br />
|Right now materialized views require the user to create triggers on the main table to keep the summary table current. SQL syntax should be able to manage the triggers and summary table automatically. A more sophisticated implementation would automatically retrieve from the summary table when the main table is referenced, if possible. See [[Materialized Views]] for implementation details<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00479.php <nowiki>GSoC - proposal - Materialized Views in PostgreSQL</nowiki>] <br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to modify views via ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00691.php <nowiki>Re: idea: storing view source in system catalogs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01410.php <nowiki>modifying views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00300.php <nowiki>Re: patch: Add columns via CREATE OR REPLACE VIEW</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent low-cost functions from seeing unauthorized view rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg01346.php <nowiki>Using views for row-level access control is leaky</nowiki>]<br />
}}<br />
<br />
== SQL Commands ==<br />
<br />
{{TodoItem<br />
|Add CORRESPONDING BY to UNION/INTERSECT/EXCEPT}}<br />
<br />
{{TodoItem<br />
|Improve type determination of unknown (NULL or quoted literal) result columns for UNION/INTERSECT/EXCEPT<br />
* [http://archives.postgresql.org/message-id/9799.1302719551@sss.pgh.pa.us <nowiki>UNION construct type cast gives poor error message</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ROLLUP, CUBE, GROUPING SETS options to GROUP BY<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00838.php <nowiki>WIP: grouping sets support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00466.php <nowiki>Implementation of GROUPING SETS (T431: Extended grouping capabilities)</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Fix TRUNCATE ... RESTART IDENTITY so its effect on sequences is rolled back on transaction abort<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00550.php <nowiki>Re: [PATCHES] TRUNCATE TABLE with IDENTITY</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow prepared transactions with temporary tables created and dropped in the same transaction, and when an ON COMMIT DELETE ROWS temporary table is accessed<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00047.php <nowiki>Re: &quot;could not open relation 1663/16384/16584: No such file or directory&quot; in a specific combination of transactions with temp tables</nowiki>]<br />
* [http://archives.postgresql.org/message-id/492543D5.9050904@enterprisedb.com A suggestion on how to implement this]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a GUC variable to warn about non-standard SQL usage in queries}}<br />
<br />
{{TodoItem<br />
|Add SQL-standard MERGE/REPLACE/UPSERT command<br />
|MERGE is typically used to merge two tables. REPLACE or UPSERT command does UPDATE, or on failure, INSERT. See [[SQL MERGE]] for notes on the implementation details.<br />
}}<br />
<br />
{{TodoItem<br />
|Add NOVICE output level for helpful messages<br />
|For example, have it warn about unjoined tables. This could also control automatic sequence/index creation messages.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow NOTIFY in rules involving conditionals}}<br />
<br />
{{TodoItem<br />
|Allow EXPLAIN to identify tables that were skipped because of constraint_exclusion<br />
}}<br />
<br />
{{TodoItemDone<br />
|Enable standard_conforming_strings by default<br />
|When this is done, backslash-quote should be prohibited in non-E<nowiki>''</nowiki> strings because of possible confusion over how such strings treat backslashes. Basically, <nowiki>''</nowiki> is always safe for a literal single quote, while \' might or might not be based on the backslash handling rules.}}<br />
<br />
{{TodoItem<br />
|Simplify dropping roles that have objects in several databases}}<br />
<br />
{{TodoItem<br />
|Allow the count returned by SELECT, etc to be represented as an int64 to allow a higher range of values}}<br />
<br />
{{TodoItem<br />
|Add support for WITH RECURSIVE ... CYCLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00291.php <nowiki>WITH RECURSIVE ... CYCLE in vanilla SQL: issues with arrays of rows</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Add DEFAULT .. AS OWNER so permission checks are done as the table owner<br />
|This would be useful for SERIAL nextval() calls and CHECK constraints.}}<br />
<br />
{{TodoItem<br />
|Allow DISTINCT to work in multiple-argument aggregate calls}}<br />
<br />
{{TodoItem<br />
|Add column to pg_stat_activity that shows the progress of long-running commands like CREATE INDEX and VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00203.php <nowiki>EXPLAIN progress info</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow INSERT/UPDATE/DELETE ... RETURNING in common table expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00472.php <nowiki>Writeable CTEs and side effects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add comments on system tables/columns using the information in catalogs.sgml<br />
|Ideally the information would be pulled from the SGML file automatically.}}<br />
<br />
{{TodoItem<br />
|Prevent the specification of conflicting transaction read/write options<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00684.php <nowiki>Re: SET TRANSACTION and SQL Standard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support LATERAL subqueries<br />
|Lateral subqueries can reference columns of tables defined outside the subquery at the same level, i.e. ''laterally''.<br />
For example, a LATERAL subquery in a FROM clause could reference tables defined in the same FROM clause.<br />
Currently only the columns of tables defined ''above'' subqueries are recognized.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00292.php <nowiki>LATERAL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00991.php <nowiki>Re: LATERAL</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add support for functional dependencies<br />
|This would allow omitting GROUP BY columns when grouping by the primary key.<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent temporary tables created with ON COMMIT DELETE ROWS from repeatedly truncating the table on every commit if the table is already empty<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00842.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-03/msg00392.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-04/msg00046.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow DELETE and UPDATE to be used with LIMIT and ORDER BY<br />
* http://archives.postgresql.org/pgadmin-hackers/2010-04/msg00078.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01997.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00021.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow finer control over the caching of prepared query plans<br />
|Currently anonymous (un-named) queries prepared via the wire protocol are replanned every time bind parameters are supplied --- allow SQL PREPARE to do the same. Also, allow control over replanning prepared queries either manually or automatically when statistics for execute parameters differ dramatically from those used during planning.<br />
* http://archives.postgresql.org/message-id/201002151911.o1FJBYh22763@momjian.us<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00597.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow PREPARE of cursors}}<br />
<br />
{{TodoItem<br />
|Have DISCARD PLANS discard plans cached by functions<br />
|DISCARD all should do the same.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00431.php<br />
}}<br />
<br />
=== CREATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow CREATE TABLE AS to determine column lengths for complex expressions like SELECT col1 || col2}}<br />
<br />
{{TodoItem<br />
|Have WITH CONSTRAINTS also create constraint indexes<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00149.php <nowiki>Re: CREATE TABLE LIKE INCLUDING INDEXES support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move NOT NULL constraint information to pg_constraint<br />
|Currently NOT NULL constraints are stored in pg_attribute without any designation of their origins, e.g. primary keys. One manifest problem is that dropping a PRIMARY KEY constraint does not remove the NOT NULL constraint designation. Another issue is that we should probably force NOT NULL to be propagated from parent tables to children, just as CHECK constraints are. (But then does dropping PRIMARY KEY affect children?)<br />
* http://archives.postgresql.org/message-id/19768.1238680878@sss.pgh.pa.us<br />
* http://archives.postgresql.org/message-id/200909181005.n8IA5Ris061239@wwwmaster.postgresql.org<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent concurrent CREATE TABLE from sometimes returning a cryptic error message<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00169.php <nowiki>BUG #3692: Conflicting create table statements throw unexpected error</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow CREATE TABLE to optionally create a table if it does not already exist, without throwing an error<br />
|The fact that tables contain data makes this more complex than other CREATE OR REPLACE operations.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01300.php <nowiki>Add column if not exists (CINE)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add CREATE SCHEMA ... LIKE that copies a schema}}<br />
<br />
{{TodoItem<br />
|Fix CREATE OR REPLACE FUNCTION to not leave objects depending on the function in inconsistent state<br />
* [http://archives.postgresql.org/pgsql-general/2008-08/msg00985.php indexes on functions and create or replace function]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow temporary tables to exist as empty by default in all sessions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00006.php <nowiki>what is difference between LOCAL and GLOBAL TEMP TABLES in PostgreSQL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg01329.php <nowiki>idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-05/msg00016.php <nowiki>Re: idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01098.php <nowiki>global temporary tables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow the creation of "distinct" types<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01647.php <nowiki>Distinct types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider analyzing temporary tables when they are first used in a query<br />
|Autovacuum cannot analyze or vacuum temporary tables.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00416.php <nowiki>autovacuum and temp tables support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow an unlogged table to be changed to logged<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00315.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== UPDATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow UPDATE tab SET ROW (col, ...) = (SELECT...)</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg01308.php <nowiki>Re: [PATCHES] extension for sql update</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00865.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00315.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00237.php <nowiki>Re: UPDATE using sub selects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Research self-referential UPDATEs that see inconsistent row versions in read-committed mode<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00507.php <nowiki>Concurrently updating an updatable view</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00016.php <nowiki>Re: Do we need a TODO? (was Re: Concurrently updating anupdatable view)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve performance of EvalPlanQual mechanism that rechecks already-updated rows<br />
|This is related to the previous item, which questions whether it even has the right semantics<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-09/msg00045.php <nowiki>BUG #4401: concurrent updates to a table blocks one update indefinitely</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-07/msg00302.php <nowiki>BUG #4945: Parallel update(s) gone wild</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ALTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have ALTER TABLE RENAME of a SERIAL column rename the sequence<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have ALTER SEQUENCE RENAME rename the sequence name stored in the sequence table<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-09/msg00092.php <nowiki>BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00007.php <nowiki>Re: BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Allow ALTER TABLE ... ALTER CONSTRAINT ... RENAME or ALTER TABLE RENAME CONSTRAINT<br />
* [http://archives.postgresql.org/pgsql-patches/2006-02/msg00168.php <nowiki>ALTER CONSTRAINT RENAME patch reverted</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ALTER DOMAIN to modify the underlying data type}}<br />
<br />
{{TodoItemDone<br />
|Allow ALTER TABLE to change constraint deferrability}}<br />
<br />
{{TodoItemDone<br />
|Add missing object types for ALTER ... SET SCHEMA}}<br />
<br />
{{TodoItem<br />
|Allow ALTER TABLESPACE to move the tablespace to different directories}}<br />
<br />
{{TodoItem<br />
|Allow moving system tables to other tablespaces, where possible<br />
|Currently non-global system tables must be in the default database tablespace. Global system tables can never be moved.}}<br />
<br />
{{TodoItem<br />
|Have ALTER INDEX update the name of a constraint using that index}}<br />
<br />
{{TodoItem<br />
|Allow column display reordering by recording a display, storage, and permanent id for every column?<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00782.php <nowiki>Re: column ordering, was Re: [PATCHES] Enums patch v2</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01029.php <nowiki>Column reordering in pg_dump</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow an existing index to be marked as a table's primary key<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00500.php <nowiki>Setting a pre-existing index as a primary key</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00642.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00265.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow ALTER TYPE on composite types to perform operations similar to ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00245.php <nowiki>ALTER composite type does not work, but ALTER TABLE which ROWTYPE is used as a type - works fine</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Don't require table rewrite on ALTER TABLE ... ALTER COLUMN TYPE, when the old and new data types are binary compatible<br />
* http://archives.postgresql.org/message-id/200903040137.n241bAUV035002@wwwmaster.postgresql.org<br />
* [http://archives.postgresql.org/pgsql-patches/2006-10/msg00154.php <nowiki>Eliminating phase 3 requirement for varlen increases via ALTER COLUMN</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02360.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow deactivating (and reactivating) indexes via ALTER TABLE<br />
|{{messageLink|<87hbegz5ir.fsf@cbbrowne.afilias-int.info>|In discussion on FK activation/deactivation}} <br />
}}<br />
<br />
{{TodoItemDone<br />
|Reduce locking required for ALTER commands<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00533.php <nowiki>ALTER TABLE SET STATISTICS requires AccessExclusiveLock</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg01083.php <nowiki>Re: ALTER TABLE SET STATISTICS requires AccessExclusiveLock</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2008-10/msg01248.php<br />
* http://archives.postgresql.org/pgsql-hackers/2008-10/msg00242.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Fix removal of NULL constraints in inherited tables<br />
* http://archives.postgresql.org/pgsql-hackers/2010-06/msg00919.php <br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01773.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00329.php<br />
}}<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== CLUSTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Automatically maintain clustering on a table<br />
|This might require some background daemon to maintain clustering during periods of low usage. It might also require tables to be only partially filled for easier reorganization. Another idea would be to create a merged heap/index data file so an index lookup would automatically access the heap data too. A third idea would be to store heap rows in hashed groups, perhaps using a user-supplied hash function.<br />
* [http://archives.postgresql.org/pgsql-performance/2004-08/msg00350.php <nowiki>Equivalent praxis to CLUSTERED INDEX?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00155.php <nowiki>Re: Grouped Index Tuples</nowiki>]<br />
* http://community.enterprisedb.com/git/<br />
* [http://archives.postgresql.org/pgsql-performance/2009-10/msg00346.php <nowiki>Re: maintain_cluster_order_v5.patch</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Improve CLUSTER performance by sorting to reduce random I/O<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01371.php <nowiki>Our CLUSTER implementation is pessimal</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Make CLUSTER VERBOSE more verbose.<br />
|It is also used by new VACUUM FULL VERBOSE.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== COPY ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report error lines and continue<br />
|This requires the use of a savepoint before each COPY line is processed, with ROLLBACK on COPY failure. <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00572.php <nowiki>Re: VLDB Features</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY on a newly-created table to skip WAL logging<br />
|On crash recovery, the table involved in the COPY would be removed or have its heap and index files truncated. One issue is that no other backend should be able to add to the table at the same time, which is something that is currently allowed. This currently is done if the table is created inside the same transaction block as the COPY because no other backends can see the table.}}<br />
<br />
{{TodoItem<br />
|Allow COPY FROM to create index entries in bulk<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00811.php <nowiki>Batch update of indexes on data loading</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY in CSV mode to control whether a quoted zero-length string is treated as NULL<br />
|Currently this is always treated as a zero-length string, which generates an error when loading into an integer column <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00905.php <nowiki>Re: [PATCHES] allow CSV quote in NULL</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve COPY performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00954.php <nowiki>Re: 8.3 / 8.2.6 restore comparison</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01882.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report errors sooner<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01169.php <nowiki>Timely reporting of COPY errors</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to handle other number formats<br />
|E.g. the German notation. Best would be something like WITH DECIMAL ','.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow a stalled COPY to exit if the backend is terminated<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00067.php <nowiki>Re: possible bug not in open items</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GRANT/REVOKE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SERIAL sequences to inherit permissions from the base table?}}<br />
<br />
{{TodoItem<br />
|Allow dropping of a role that has connection rights<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00736.php <nowiki>DROP ROLE dependency tracking ...</nowiki>]<br />
}}<br />
{{TodoEndSubsection}}<br />
<br />
=== DECLARE CURSOR ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent DROP TABLE from dropping a table referenced by its own open cursor?}}<br />
<br />
{{TodoItem<br />
|Provide some guarantees about the behavior of cursors that invoke volatile functions<br />
* [http://archives.postgresql.org/message-id/20997.1244563664@sss.pgh.pa.us Re: Cursor with hold emits the same row more than once across commits in 8.3.7]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== INSERT ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow INSERT/UPDATE of the system-generated oid value for a row}}<br />
<br />
{{TodoItem<br />
|In rules, allow VALUES() to contain a mixture of 'old' and 'new' references}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SHOW/SET ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM ANALYZE, and CLUSTER}}<br />
<br />
{{TodoItem<br />
|Rationalize the discrepancy between settings that use values in bytes and SHOW that returns the object count<br />
* [http://archives.postgresql.org/pgsql-docs/2008-07/msg00007.php <nowiki>Re: [ADMIN] shared_buffers and shmmax</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ANALYZE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE issue NOTICE messages when the estimated and actual row counts differ by a specified percentage}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE report rows as floating-point numbers<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01363.php <nowiki>explain analyze rows=%.0f</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00108.php <nowiki>Re: explain analyze rows=%.0f</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve how ANALYZE computes in-doubt tuples<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00771.php <nowiki>VACUUM/ANALYZE counting of in-doubt tuples</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Window Functions ===<br />
See {{messageLink|357.1230492361@sss.pgh.pa.us|TODO items for window functions}}.<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Support creation of user-defined window functions<br />
|We have the ability to create new window functions written in C. Is it<br />
worth the effort to create an API that would let them be written in PL/pgsql, etc?}}<br />
<br />
{{TodoItem<br />
|Implement full support for window framing clauses<br />
|In addition to done clauses described in the [http://developer.postgresql.org/pgdocs/postgres/sql-expressions.html#SYNTAX-WINDOW-FUNCTIONS latest doc], these clauses are not implemented yet.<br />
* RANGE BETWEEN ... PRECEDING/FOLLOWING<br />
* EXCLUDE<br />
}}<br />
<br />
{{TodoItem<br />
|Investigate tuplestore performance issues<br />
|The tuplestore_in_memory() thing is just a band-aid, we ought to try to solve it properly. tuplestore_advance seems like a weak spot as well.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00152.php <nowiki>tuplestore potential performance problem</nowiki>]<br />
}}<br />
<br />
{{TodoItem|Do we really need so much duplicated code between Agg and WindowAgg?}}<br />
<br />
{{TodoItem<br />
|Teach planner to evaluate multiple windows in the optimal order<br />
|Currently windows are always evaluated in the query-specified order.<br />
* http://archives.postgresql.org/message-id/3CDAD71E9D70417290FCF66F0178D1E1@amd64<br />
}}<br />
<br />
{{TodoItem<br />
|Implement DISTINCT clause in window aggregates<br />
|Some proprietary RDBMSs have implemented it already, so it helps with porting from those.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Integrity Constraints ==<br />
=== Keys ===<br />
<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve deferrable unique constraints for cases with many conflicts<br />
|The current implementation fires a trigger for each potentially conflicting row. This might not scale well for an update that changes many key values at once.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Referential Integrity ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add MATCH PARTIAL referential integrity}}<br />
<br />
{{TodoItem<br />
|Change foreign key constraint for array -&gt; element to mean element in array?<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01814.php <nowiki>foreign keys for array/period contains relationships</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when cascading referential triggers make changes on cascaded tables, seeing the tables in an intermediate state<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00174.php <nowiki>Re: [PATCHES] Work-in-progress referential action trigger timing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Optimize referential integrity checks<br />
* [http://archives.postgresql.org/pgsql-performance/2005-10/msg00458.php <nowiki>Re: Effects of cascading references in foreign keys</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00744.php <nowiki>Can't ri_KeysEqual() consider two nulls as equal?</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Server-Side Languages ==<br />
<br />
{{TodoItem<br />
|Add support for polymorphic arguments and return types to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add support for OUT and INOUT parameters to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add more fine-grained specification of functions taking arbitrary data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement stored procedures<br />
|This might involve the control of transaction state and the return of multiple result sets<br />
* [http://archives.postgresql.org/pgsql-general/2008-10/msg00454.php <nowiki>PL/pgSQL stored procedure returning multiple result sets (SELECTs)?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01375.php <nowiki>Proposal: real procedures again (8.4)</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00542.php<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-04/msg01149.php <nowiki>Gathering specs and discussion on feature (post 9.1)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow holdable cursors in SPI}}<br />
<br />
{{TodoItemEasy<br />
|Add SPI_gettypmod() to return a field's typemod from a TupleDesc<br />
* http://archives.postgresql.org/pgsql-hackers/2005-11/msg00250.php<br />
}}<br />
<br />
=== SQL-Language Functions ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SQL-language functions to reference parameters by parameter name<br />
|Currently SQL-language functions can only refer to dollar parameters, e.g. $1<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01479.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01519.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00221.php<br />
}}<br />
<br />
{{TodoItem<br />
|Rethink query plan caching and timing of parse analysis within SQL-language functions<br />
|They should work more like plpgsql functions do ...<br />
* [http://archives.postgresql.org/pgsql-bugs/2011-05/msg00078.php <nowiki>Re: BUG #6019: invalid cached plan on inherited table</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/pgSQL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow handling of %TYPE arrays, e.g. tab.col%TYPE[]}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow listing of record column names, and access to record columns via variables, e.g. columns := r.(*), tval2 := r.(colname)</nowiki><br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00458.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00302.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00031.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow row and record variables to be set to NULL constants, and allow NULL tests on such variables<br />
|Because a row is not scalar, do not allow assignment from NULL-valued scalars.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00070.php <nowiki>NULL and plpgsql rows</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider keeping separate cached copies when search_path changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01009.php <nowiki>pl/pgsql Plan Invalidation and search_path</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULL row values vs. NULL rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg01758.php <nowiki>Null row vs. row of nulls in plpgsql</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg01973.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Perl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow regex operations in plperl using UTF8 characters in non-UTF8 encoded databases}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Python ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Add table function support}}<br />
<br />
{{TodoItemDone<br />
|Add tracebacks<br />
* [http://archives.postgresql.org/pgsql-patches/2006-02/msg00288.php <nowiki>Re: plpython tracebacks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Develop a trusted variant of PL/Python.}}<br />
<br />
{{TodoItem<br />
|Create a new restricted execution class that will allow passing function arguments in as locals. Passing them as globals means functions cannot be called recursively.<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-02/msg01468.php <nowiki>Re: pl/python do not delete function arguments</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve documentation}}<br />
<br />
{{TodoItem<br />
|Add a DB-API compliant interface on top of the SPI interface}}<br />
<br />
{{TodoItem<br />
|Improve behaviour of exception functions and types<br />
* http://archives.postgresql.org/pgsql-docs/2010-11/msg00022.php<br />
* http://archives.postgresql.org/pgsql-docs/2010-11/msg00031.php<br />
}}<br />
{{TodoItem<br />
|For functions returning a setof record with a composite type, cache the I/O functions for the composite type<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02007.php<br />
}}<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Tcl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add table function support}}<br />
<br />
{{TodoItem<br />
|Check encoding validity of values passed back to Postgres in function returns, trigger tuple changes, and SPI calls.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Clients ==<br />
<br />
{{TodoItem<br />
|Add a function like pg_get_indexdef() that report more detailed index information<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00166.php <nowiki>BUG #3829: Wrong index reporting from pgAdmin III (v1.8.0 rev 6766-6767)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Split out pg_resetxlog output into pre- and post-sections<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg02040.php<br />
}}<br />
<br />
=== pg_ctl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow pg_ctl to work properly with configuration files located outside the PGDATA directory<br />
|pg_ctl can not read the pid file because it isn't located in the config directory but in the PGDATA directory. The solution is to allow pg_ctl to read and understand postgresql.conf to find the data_directory value.<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-10/msg00024.php <nowiki>BUG #5103: &quot;pg_ctl -w (re)start&quot; fails with custom unix_socket_directory</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Modify pg_ctl behavior and exit codes to make it easier to write an LSB conforming init script<br />
|It may be desirable to condition some of the changes on a command-line switch, to avoid breaking existing scripts. A Linux shell (sh) script is referenced which has been tested and seems to provide a high degree of conformance in multiple environments. Study of this script might suggest areas where pg_ctl could be modified to make writing an LSB conforming script easier; however, some aspects of that script would be unnecessary with other suggested changes to pg_ctl, and discussion on the lists did not reach consensus on support for all aspects of this script. Further discussion of particular changes is needed before beginning any work.<br />
* [[Lsb_conforming_init_script|LSB conforming init script]]<br />
These threads should be studied for other ideas on improvements:<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01390.php <nowiki>We should Axe /contrib/start-scripts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01843.php <nowiki>Linux LSB init script</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00008.php <nowiki>Re: Linux LSB init script</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== psql ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have psql \ds show all sequences and their settings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00916.php <nowiki>Re: TODO item: Have psql show current values for a sequence</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00401.php <nowiki>Quick patch: Display sequence owner</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have \d on a sequence indicate if the sequences is owned by a table}}<br />
<br />
{{TodoItem<br />
|Move psql backslash database information into the backend, use mnemonic commands?<br />
|This would allow non-psql clients to pull the same information out of the database as psql. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-01/msg00191.php <nowiki>Re: psql \d option list overloaded</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make psql's \d commands more consistent in its handling of schemas<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00014.php <nowiki>Re: psql and schemas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consistently display privilege information for all objects in psql}}<br />
<br />
{{TodoItem<br />
|Add &quot;auto&quot; expanded mode that outputs in expanded format if &quot;wrapped&quot; mode can't wrap the output to the screen width<br />
|Consider using auto-expanded mode for backslash commands like \df+.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00417.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg01638.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent tab completion of SET TRANSACTION from querying the database and therefore preventing the transaction isolation level from being set.<br />
|Currently SET &lt;tab&gt; causes a database lookup to check all supported session variables. This query causes problems because setting the transaction isolation level must be the first statement of a transaction.}}<br />
<br />
{{TodoItem<br />
|Add a \set variable to control whether \s displays line numbers<br />
|Another option is to add \# which lists line numbers, and allows command execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00255.php <nowiki>Re: psql possible TODO</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have \d show child tables that inherit from the specified parent}}<br />
<br />
{{TodoItem<br />
|Include the symbolic SQLSTATE name in verbose error reports<br />
* [http://archives.postgresql.org/pgsql-general/2007-09/msg00438.php <nowiki>Re: Checking is TSearch2 query is valid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add prompt escape to display the client and server versions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00310.php <nowiki>WIP patch for TODO Item: Add prompt escape to display the client and server versions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to wrap column values at whitespace boundaries, rather than chopping them at a fixed width.<br />
|Currently, &quot;wrapped&quot; format chops values into fixed widths. Perhaps the word wrapping could use the same algorithm documented in the W3C specification. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00404.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
* http://www.w3.org/TR/CSS21/tables.html#auto-table-layout}}<br />
<br />
{{TodoItem<br />
|Support the ReST table output format<br />
|Details about the ReST format: http://docutils.sourceforge.net/rst.html#reference-documentation<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01007.php <nowiki>Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00518.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00609.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to print advice for people familiar with other databases<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01845.php <nowiki>MySQL-ism help patch for psql</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Consider showing TOAST and index sizes in \dt+<br />
* [http://archives.postgresql.org/pgsql-general/2010-01/msg00912.php <nowiki>\dt+ sizes don't include TOAST data</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-04/msg00485.php <nowiki>Re: psql \dt and table size</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|\dd is missing comments for several types of objects<br />
|Comments are not handled at all for some object types, and are handled by both \dd and the individual backslash command for others. Consider a system view like pg_comments to manage this mess.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00436.php <nowiki>Re: More robust pg_hba.conf parsing/error logging</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-09/msg00199.php <nowiki>comment on constraint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-09/msg01080.php <nowiki>pg_comments</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-05/msg00885.php <nowiki>patch: Allow \dd to show constraint comments</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ability to edit views with \ev<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00023.php <nowiki>Adding \ev view editor?</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add \dL to show languages<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-07/msg00915.php <nowiki>Re: [PATCH] Psql List Languages</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Distinguish between unique indexes and unique constraints in \d+<br />
* http://archives.postgresql.org/message-id/8780.1271187360@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Fix FETCH_COUNT to handle SELECT ... INTO and WITH queries<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01565.php<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00192.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent psql from sending remaining single-line multi-statement queries after reconnecting<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00159.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01283.php<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Add \i option to bring in the specified file as a quoted literal. This would be useful for creating functions and other areas. Details still need to be worked out.<br />
* http://archives.postgresql.org/pgsql-bugs/2011-02/msg00016.php<br />
* http://archives.postgresql.org/pgsql-bugs/2011-02/msg00020.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having psql -c read .psqlrc, for consistency<br />
|psql -f already reads .psqlrc<br />
}}<br />
<br />
{{TodoItem<br />
|Allow processing of multiple -f (file) options<br />
}}<br />
<br />
{{TodoItem<br />
|Improve line drawing characters<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00386.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== pg_dump / pg_restore ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|<nowiki>Add full object name to the tag field. eg. for operators we need '=(integer, integer)', instead of just '='.</nowiki>}}<br />
<br />
{{TodoItem<br />
|Add pg_dumpall custom format dumps?<br />
* [http://archives.postgresql.org/pgsql-general/2010-05/msg00509.php pg_dumpall custom format]<br />
|}}<br />
<br />
{{TodoItem<br />
|Avoid using platform-dependent locale names in pg_dumpall output<br />
|Using native locale names puts roadblocks in the way of porting a dump to another platform. One possible solution is to get<br />
CREATE DATABASE to accept some agreed-on set of locale names and fix them up to meet the platform's requirements.<br />
* http://archives.postgresql.org/message-id/21396.1241716688@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Allow selection of individual object(s) of all types, not just tables}}<br />
<br />
{{TodoItem<br />
|In a selective dump, allow dumping of an object and all its dependencies}}<br />
<br />
{{TodoItem<br />
|Add options like pg_restore -l and -L to pg_dump}}<br />
<br />
{{TodoItem<br />
|Add support for multiple pg_restore -t options, like pg_dump<br />
|pg_restore's -t switch is less useful than pg_dump's in quite a few ways: no multiple switches, no pattern matching, no ability to pick up indexes and other dependent items for a selected table. It should be made to handle this switch just like pg_dump does.}}<br />
<br />
{{TodoItem<br />
|Stop dumping CASCADE on DROP TYPE commands in clean mode}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump --clean to drop roles that own objects or have privileges<br />
|tgl says: if this is about pg_dumpall, it's done as of 8.4. If it's really about pg_dump, what does it mean? pg_dump has no business dropping roles.}}<br />
<br />
{{TodoItem<br />
|Remove unnecessary function pointer abstractions in pg_dump source code}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump to utilize multiple CPUs and I/O channels by dumping multiple objects simultaneously<br />
|The difficulty with this is getting multiple dump processes to produce a single dump output file. It also would require several sessions to share the same snapshot. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php <nowiki>pg_dump additional options for performance</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00135.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00040.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02454.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow pg_restore to load different parts of the COPY data for a single table simultaneously}}<br />
<br />
{{TodoItem<br />
|Remove support for dumping from pre-7.3 servers<br />
|In 7.3 and later, we can get accurate dependency information from the server. pg_dump still contains a lot of crufty code<br />
to try to deal with the lack of dependency info in older servers, but the usefulness of maintaining that code grows small.}}<br />
<br />
{{TodoItem<br />
|Allow pre/data/post files when schema and data are dumped separately, for performance reasons<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php <nowiki>pg_dump additional options for performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00185.php <nowiki>Re: pg_dump additional options for performance</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00821.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00135.php<br />
}}<br />
<br />
{{TodoItem<br />
|Refactor handling of database attributes between pg_dump and pg_dumpall<br />
|Currently only pg_dumpall emits database attributes, such as ALTER DATABASE SET commands and database-level GRANTs.<br />
Many people wish that pg_dump would do that. One proposal is to let pg_dump issue such commands if the -C switch was used,<br />
but it's unclear whether that will satisfy the demand.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01031.php <nowiki>ALTER DATABASE vs pg_dump</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-05/msg00010.php summary of the issues]<br />
}}<br />
<br />
{{TodoItem<br />
|Change pg_dump so that a comment on the dumped database is applied to the loaded database, even if the database has a different name.<br />
|This will require new backend syntax, perhaps COMMENT ON CURRENT DATABASE. This is related to the previous item.}}<br />
<br />
{{TodoItem<br />
|Allow parallel restore of tar dumps<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-02/msg01154.php <nowiki>Re: parallel restore</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow pg_dumpall to output restorable ALTER USER/DATABASE SET settings<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00916.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00394.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02359.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ecpg ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Docs<br />
|Document differences between ecpg and the SQL standard and information about the Informix-compatibility module.}}<br />
<br />
{{TodoItem<br />
|Solve cardinality &gt; 1 for input descriptors / variables?}}<br />
<br />
{{TodoItem<br />
|Add a semantic check level, e.g. check if a table really exists}}<br />
<br />
{{TodoItem<br />
|fix handling of DB attributes that are arrays}}<br />
<br />
{{TodoItem<br />
|Fix nested C comments}}<br />
<br />
{{TodoItemEasy<br />
|sqlwarn[6] should be 'W' if the PRECISION or SCALE value specified}}<br />
<br />
{{TodoItem<br />
|Make SET CONNECTION thread-aware, non-standard?}}<br />
<br />
{{TodoItem<br />
|Allow multidimensional arrays}}<br />
<br />
{{TodoItem<br />
|Implement COPY FROM STDIN}} <br />
<br />
{{TodoItem<br />
|Provide a way to specify size of a bytea parameter<br />
* [http://archives.postgresql.org/message-id/200906192131.n5JLVoMo044178@wwwmaster.postgresql.org <nowiki>BUG #4866: ECPG and BYTEA</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Fix small memory leaks in ecpg<br />
|Memory leaks in a short running application like ecpg are not really a problem, but make debugging more complicated}} <br />
<br />
{{TodoItem<br />
|Allow reuse of cursor name variables<br />
* [http://archives.postgresql.org/message-id/20100329113435.GA3430@feivel.credativ.lan <nowiki>Problems with variable cursorname in ecpg</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== libpq ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent PQfnumber() from lowercasing unquoted column names<br />
|PQfnumber() should never have been doing lowercasing, but historically it has so we need a way to prevent it}}<br />
<br />
{{TodoItem<br />
|Allow statement results to be automatically batched to the client<br />
|Currently all statement results are transferred to the libpq client before libpq makes the results available to the application. This feature would allow the application to make use of the first result rows while the rest are transferred, or held on the server waiting for them to be requested by libpq. One complexity is that a statement like SELECT 1/col could error out mid-way through the result set.}}<br />
<br />
{{TodoItem<br />
|Consider disallowing multiple queries in PQexec() as an additional barrier to SQL injection attacks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00184.php <nowiki>Re: InitPostgres and flatfiles question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add PQexecf() that allows complex parameter substitution<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01803.php <nowiki>Last minute mini-proposal (I know, know) for PQexecf()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add SQLSTATE and severity to errors generated within libpq itself<br />
* [http://archives.postgresql.org/pgsql-interfaces/2007-11/msg00015.php <nowiki>v8.1: Error severity on libpq PGconn*</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01425.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add code to detect client encoding and locale from the operating system environment<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01040.php <nowiki>Determining client_encoding from client locale</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for interface/ipaddress binding to libpq<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01811.php <nowiki>SR/libpq - outbound interface/ipaddress binding</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Triggers ==<br />
<br />
{{TodoItem<br />
|Improve storage of deferred trigger queue<br />
|Right now all deferred trigger information is stored in backend memory. This could exhaust memory for very large trigger queues. This item involves dumping large queues into files, or doing some kind of join to process all the triggers, some bulk operation, or a bitmap. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00876.php <nowiki>Re: BUG #4204: COPY to table with FK has memory leak</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00464.php <nowiki>Scaling up deferred unique checks and the after trigger queue</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow triggers to be disabled in only the current session.<br />
|This is currently possible by starting a multi-statement transaction, modifying the system tables, performing the desired SQL, restoring the system tables, and committing the transaction. ALTER TABLE ... TRIGGER requires a table lock so it is not ideal for this usage.}}<br />
<br />
{{TodoItem<br />
|With disabled triggers, allow pg_dump to use ALTER TABLE ADD FOREIGN KEY<br />
|If the dump is known to be valid, allow foreign keys to be added without revalidating the data.}}<br />
<br />
{{TodoItem<br />
|Allow statement-level triggers to access modified rows}}<br />
<br />
{{TodoItem<br />
|When statement-level triggers are defined on a parent table, have them fire only on the parent table, and fire child table triggers only where appropriate<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01883.php <nowiki>Statement-level triggers and inheritance</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow AFTER triggers on system tables<br />
|System tables are modified in many places in the backend without going through the executor and therefore not causing triggers to fire. To complete this item, the functions that modify system tables will have to fire triggers.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01665.php<br />
* http://wiki.postgresql.org/wiki/DDL_Triggers<br />
}}<br />
<br />
{{TodoItem<br />
|Tighten trigger permission checks<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00564.php <nowiki>Security leak with trigger functions?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow BEFORE INSERT triggers on views<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg01466.php <nowiki>Re: Why can't I put a BEFORE EACH ROW trigger on a view?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add database and transaction-level triggers<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00451.php <nowiki>Proposal for db level triggers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00620.php <nowiki>triggers on prepare, commit, rollback... ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce locking requirements for creating a trigger<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00635.php <nowiki>Re: Change lock requirements for adding a trigger</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid requirement for "AFTER" trigger functions to return a value<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02384.php<br />
}}<br />
<br />
== Inheritance ==<br />
<br />
{{TodoItem<br />
|Allow inherited tables to inherit indexes, UNIQUE constraints, and primary/foreign keys<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-05/msg00285.php <nowiki>Partitioning/inherited tables vs FKs</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00039.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00305.php<br />
}}<br />
<br />
{{TodoItem<br />
|Honor UNIQUE INDEX on base column in INSERTs/UPDATEs on inherited table, e.g. INSERT INTO inherit_table (unique_index_col) VALUES (dup) should fail<br />
|The main difficulty with this item is the problem of creating an index that can span multiple tables.}}<br />
<br />
{{TodoItem<br />
|Determine whether ALTER TABLE / SET SCHEMA should work on inheritance hierarchies (and thus support ONLY). If yes, implement it.}}<br />
<br />
{{TodoItem<br />
|ALTER TABLE variants sometimes support recursion and sometimes not, but this is poorly/not documented, and the ONLY marker would then be silently ignored. Clarify the documentation, and reject ONLY if it is not supported.}}<br />
<br />
== Indexes ==<br />
<br />
{{TodoItem<br />
|Prevent index uniqueness checks when UPDATE does not modify the column<br />
|Uniqueness (index) checks are done when updating a column even if the column is not modified by the UPDATE.<br />
However, HOT already short-circuits this in common cases, so more work might not be helpful.}}<br />
<br />
{{TodoItem<br />
|Allow the creation of on-disk bitmap indexes which can be quickly combined with other bitmap indexes<br />
|Such indexes could be more compact if there are only a few distinct values. Such indexes can also be compressed. Keeping such indexes updated can be costly.<br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00512.php <nowiki>Re: Bitmap index AM</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01107.php <nowiki>Bitmap index thoughts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00265.php <nowiki>Stream bitmaps</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01214.php <nowiki>Re: Bitmapscan changes - Requesting further feedback</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00013.php <nowiki>Updated bitmap index patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00741.php <nowiki>Reviewing new index types (was Re: [PATCHES] Updated bitmap indexpatch)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01023.php <nowiki>Bitmap Indexes: request for feedback</nowiki>]<br />
* http://archives.postgresql.org/message-id/800923.27831.qm@web29010.mail.ird.yahoo.com <br />
}}<br />
<br />
{{TodoItem<br />
|Allow accurate statistics to be collected on indexes with more than one column or expression indexes, perhaps using per-index statistics<br />
* [http://archives.postgresql.org/pgsql-performance/2006-10/msg00222.php <nowiki>Re: Simple join optimized badly?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01131.php <nowiki>Stats for multi-column indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00741.php <nowiki>Cross-column statistics revisited</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01431.php <nowiki>Multi-Dimensional Histograms</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00913.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02179.php <br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00459.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02054.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01731.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having a larger statistics target for indexed columns and expression indexes. <br />
}}<br />
<br />
{{TodoItem<br />
|Consider smaller indexes that record a range of values per heap page, rather than having one index entry for every heap row<br />
|This is useful if the heap is clustered by the indexed values. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00341.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01264.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00465.php <nowiki>Grouped Index Tuples / Clustered Indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-03/msg00163.php <nowiki>Bitmapscan changes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00014.php <nowiki>Re: GIT patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00487.php <nowiki>Re: Index Tuple Compression Approach?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01589.php <nowiki>Re: Index AM change proposals, redux</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add REINDEX CONCURRENTLY, like CREATE INDEX CONCURRENTLY<br />
|This is difficult because you must upgrade to an exclusive table lock to replace the existing index file. CREATE INDEX CONCURRENTLY does not have this complication. This would allow index compaction without downtime. <br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00289.php <nowiki>Re: When/if to Reindex</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multiple indexes to be created concurrently, ideally via a single heap scan<br />
|pg_restore allows parallel index builds, but it is done via subprocesses, and there is no SQL interface for this.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider sorting entries before inserting into btree index<br />
* [http://archives.postgresql.org/pgsql-general/2008-01/msg01010.php <nowiki>Re: ATTN: Clodaldo was Performance problem. Could it be related to 8.3-beta4?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow index scans to return matching index keys, not just the matching heap locations<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01657.php <nowiki>Re: Is this TODO item done?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01477.php <nowiki>Index-only quals</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow creation of an index that can do comparisons to test if a value is between two column values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00757.php <nowiki>Proposal: temporal extension &quot;period&quot; data type</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using "effective_io_concurrency" for index scans<br />
* Currently only bitmap scans use this, which might be fine because most multi-row index scans use bitmap scans.<br />
}}<br />
<br />
=== GIST ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add more GIST index support for geometric data types}}<br />
<br />
{{TodoItem<br />
|Allow GIST indexes to create certain complex index types, like digital trees (see Aoki)}}<br />
<br />
{{TodoItem<br />
|Fix performance issues in contrib/seg and contrib/cube GiST support<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904161633160.4053@aragorn.flymine.org GiST index performance]<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904221704470.22330@aragorn.flymine.org draft patch]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-05/msg00069.php <nowiki>Re: GiST index performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-06/msg00068.php <nowiki>GiST index performance</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GIN ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Support empty indexed values (such as zero-element arrays) properly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00237.php contrib/intarray vs empty arrays]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-05/msg00118.php BUG #4806: Bug with GiST index and empty integer array]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Behave correctly for cases where some elements of an indexed value are NULL<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-03/msg01003.php <nowiki>GIN versus zero-key queries</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Support queries that require a full scan<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg00402.php Issue report]<br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01132.php Older issue report]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-10/msg00521.php Still another complaint]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01581.php Previous partial fix]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Improve GIN's handling of NULL array values<br />
* http://archives.postgresql.org/pgsql-bugs/2010-12/msg00032.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Hash ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add UNIQUE capability to hash indexes}}<br />
<br />
{{TodoItem<br />
|Add hash WAL logging for crash recovery}}<br />
<br />
{{TodoItem<br />
|Allow multi-column hash indexes}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Sorting ==<br />
<br />
{{TodoItem<br />
|Consider whether duplicate keys should be sorted by block/offset<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00558.php <nowiki>Remove hacks for old bad qsort() implementations?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider being smarter about memory and external files used during sorts<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01101.php <nowiki>Sorting Improvements for 8.4</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00045.php <nowiki>Re: Sorting Improvements for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider detoasting keys before sorting}}<br />
<br />
{{TodoItem<br />
|Allow sorts to use more available memory<br />
* http://archives.postgresql.org/pgsql-hackers/2007-11/msg01026.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01123.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg01957.php<br />
}}<br />
<br />
== Fsync ==<br />
<br />
{{TodoItem<br />
|Determine optimal fdatasync/fsync, O_SYNC/O_DSYNC options and whether fsync does anything<br />
|Ideally this requires a separate test program like /contrib/pg_test_fsync that can be run at initdb time or optionally later.}}<br />
<br />
{{TodoItem<br />
|Consider sorting writes during checkpoint<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php <nowiki>Sorted writes in checkpoint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00050.php <nowiki>Re: Sorting writes during checkpoint</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg02012.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00278.php<br />
}}<br />
<br />
== Cache Usage ==<br />
<br />
{{TodoItem<br />
|Speed up COUNT(*)<br />
|We could use a fixed row count and a +/- count to follow MVCC visibility rules, or a single cached value could be used and invalidated if anyone modifies the table. Another idea is to get a count directly from a unique index, but for this to be faster than a sequential scan it must avoid access to the heap to obtain tuple visibility information.}}<br />
<br />
{{TodoItem<br />
|Provide a way to calculate an &quot;estimated COUNT(*)&quot;<br />
|Perhaps by using the optimizer's cardinality estimates or random sampling.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-11/msg00943.php <nowiki>Re: Improving count(*)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow data to be pulled directly from indexes<br />
|Currently indexes do not have enough tuple visibility information to allow data to be pulled from the index without also accessing the heap. The idea is to use the visibility map used for vacuum to avoid heap lookups on pages where all tuples are visible.<br />
* [http://wiki.postgresql.org/wiki/Index-only_scans Index-Only Scans wiki]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider automatic caching of statements at various levels:<br />
* Parsed query tree<br />
* Query execute plan<br />
* Query results <br />
<br />
:<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00823.php <nowiki>Cached Query Plans (was: global prepared statements)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing internal areas (NUM_CLOG_BUFFERS) when shared buffers is increased<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-10/msg01419.php <nowiki>Re: slru.c race condition (was Re: TRAP: FailedAssertion(&quot;!((itemid)-&gt;lp_flags &amp; 0x01)&quot;,)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00030.php <nowiki>clog_buffers to 64 in 8.3?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00024.php <nowiki>CLOG Patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the amount of memory used by PrivateRefCount<br />
|<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00797.php <nowiki>PrivateRefCount (for 8.3)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00752.php <nowiki>Re: PrivateRefCount (for 8.3)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing higher priority queries to have referenced buffer cache pages stay in memory longer<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00562.php <nowiki>Re: How to keep a table in memory?</nowiki>]<br />
}}<br />
<br />
== Vacuum ==<br />
<br />
{{TodoItem<br />
|Auto-fill the free space map by scanning the buffer cache or by checking pages written by the background writer<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-02/msg01125.php <nowiki>Dead Space Map</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00011.php <nowiki>Re: Automatic free space map filling</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow concurrent inserts to use recently created pages rather than creating new ones<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg00853.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having single-page pruning update the visibility map<br />
* <nowiki>https://commitfest.postgresql.org/action/patch_view?id=75</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02344.php <nowiki>Re: visibility maps and heap_prune</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve tracking of total relation tuple counts now that vacuum doesn't always scan the whole heap<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00531.php Partial vacuum versus pg_class.reltuples]<br />
}}<br />
<br />
{{TodoItem<br />
|Bias FSM towards returning free space near the beginning of the heap file, in hopes that empty pages at the end can be truncated by VACUUM<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01124.php <nowiki>FSM search modes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a more compact data representation for dead tuple locations within VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00143.php <nowiki>Re: Have vacuum emit a warning when it runs out of maintenance_work_mem</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more information in order to improve user-side estimates of dead space bloat in relations<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg01039.php <nowiki>Re: Bloated Table</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve locking behaviour of vacuum during trailing page truncation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00319.php<br />
* http://archives.postgresql.org/message-id/4D8DF88E.7080205@Yahoo.com<br />
}}<br />
<br />
=== Auto-vacuum ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Issue log message to suggest VACUUM FULL if a table is nearly empty?}}<br />
<br />
{{TodoItem<br />
|Prevent long-lived temporary tables from causing frozen-xid advancement starvation<br />
|The problem is that autovacuum cannot vacuum them to set frozen xids; only the session that created them can do that. <br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01645.php <nowiki>Re: AutoVacuum Behaviour Question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent autovacuum from running if an old transaction is still running from the last vacuum<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00899.php <nowiki>Re: Autovacuum and OldestXmin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have autoanalyze of parent tables occur when child tables are modified<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00137.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-10/msg00271.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Locking ==<br />
<br />
{{TodoItem<br />
|Fix priority ordering of read and write light-weight locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00893.php <nowiki>lwlocks and starvation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00905.php <nowiki>Re: lwlocks and starvation</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when multiple subtransactions of the same outer transaction hold different types of locks, and one subtransaction aborts<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg01011.php <nowiki>FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00001.php <nowiki>Re: FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00435.php <nowiki>Re: [PATCHES] [pgsql-patches] Phantom Command IDs, updated patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00773.php <nowiki>Re: savepoints and upgrading locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow UPDATEs on only non-referential integrity columns not to conflict with referential integrity locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00073.php <nowiki>Referential Integrity and SHARE locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add idle_in_transaction_timeout GUC so locks are not held for long periods of time}}<br />
<br />
{{TodoItem<br />
|Improve deadlock detection when a page cleaning lock conflicts with a shared buffer that is pinned<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-01/msg00138.php <nowiki>BUG #3883: Autovacuum deadlock with truncate?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-committers/2008-01/msg00365.php <nowiki>Re: pgsql: Add checks to TRUNCATE, CLUSTER, and REINDEX to prevent</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Detect deadlocks involving LockBufferForCleanup()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow finer control over who is cancelled in a deadlock<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01727.php<br />
}}<br />
<br />
<br />
{{TodoItem<br />
|Consider a lock timeout parameter<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00485.php <nowiki>SELECT ... FOR UPDATE [WAIT integer | NOWAIT] for 8.5</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Consider improving serialized transaction behavior to avoid anomalies<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00217.php <nowiki>Serializable Isolation without blocking</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01136.php <nowiki>User-facing aspects of serializable transactions</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00035.php <nowiki>Re: User-facing aspects of serializable transactions</nowiki>]<br />
}}<br />
<br />
== Startup Time Improvements ==<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for backend creation<br />
|This would prevent the overhead associated with process creation. Most operating systems have trivial process creation time compared to database startup overhead, but a few operating systems (Win32, Solaris) might benefit from threading. Also explore the idea of a single session using multiple threads to execute a statement faster.}}<br />
<br />
{{TodoItem<br />
|Allow backends to change their database without restart<br />
|This allows for faster server startup.<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00843.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00336.php<br />
}}<br />
<br />
== Write-Ahead Log ==<br />
<br />
{{TodoItem<br />
|Eliminate need to write full pages to WAL before page modification<br />
|Currently, to protect against partial disk page writes, we write full page images to WAL before they are modified so we can correct any partial page writes during recovery. These pages can also be eliminated from point-in-time archive files. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-06/msg00655.php <nowiki>Re: Index Scans become Seq Scans after VACUUM ANALYSE</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|When full page writes are off, write CRC to WAL and check file system blocks on recovery<br />
|If CRC check fails during recovery, remember the page in case a later CRC for that page properly matches.}}<br />
<br />
{{TodoItem<br />
|Write full pages during file system write and not when the page is modified in the buffer cache<br />
|This allows most full page writes to happen in the background writer. It might cause problems for applying WAL on recovery into a partially-written page, but later the full page will be replaced from WAL.}}<br />
<br />
{{TodoItem<br />
|Reduce WAL traffic so only modified values are written rather than entire rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01589.php <nowiki>Reduction in WAL for UPDATEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow WAL information to recover corrupted pg_controldata<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00025.php <nowiki>Re: [HACKERS] pg_resetxlog -r flag</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a way to reduce rotational delay when repeatedly writing last WAL page<br />
|Currently fsync of WAL requires the disk platter to perform a full rotation to fsync again. One idea is to write the WAL to different offsets that might reduce the rotational delay. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-11/msg00483.php <nowiki>500 tpsQL + WAL log implementation</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow WAL logging to be turned off for a table, but the table might be dropped or truncated during crash recovery<br />
|Allow tables to bypass WAL writes and just fsync() dirty pages on commit. This should be implemented using ALTER TABLE, e.g. <nowiki>ALTER TABLE PERSISTENCE [ DROP | TRUNCATE | DEFAULT ]</nowiki>. Tables using non-default logging should not use referential integrity with default-logging tables. A table without dirty buffers during a crash could perhaps avoid the drop/truncate. <br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01016.php <nowiki>Re: [Bizgres-general] WAL bypass for INSERT, UPDATE and</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Speed WAL recovery by allowing more than one page to be prefetched<br />
|This should be done utilizing the same infrastructure used for prefetching in general to avoid introducing complex error-prone code in WAL replay. <br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00683.php <nowiki>Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00497.php <nowiki>Re: [GENERAL] Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg01279.php <nowiki>Read-ahead and parallelism in redo recovery</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve WAL concurrency by increasing lock granularity<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00556.php <nowiki>Reworking WAL locking</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Be more aggressive about creating WAL files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01325.php <nowiki>Re: PANIC caused by open_sync on Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-07/msg01075.php <nowiki>PreallocXlogFiles</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-04/msg00556.php <nowiki>WAL/PITR additional items</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have resource managers report the duration of their status changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01468.php <nowiki>Recovery of Multi-stage WAL actions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move pgfoundry's xlogdump to /contrib and have it rely more closely on the WAL backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00035.php <nowiki>xlogdump</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Close deleted WAL files held open in *nix by long-lived read-only backends<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01754.php <nowiki>Deleted WAL files held open by backends in Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00060.php <nowiki>Re: Deleted WAL files held open by backends in Linux</nowiki>]<br />
}}<br />
<br />
== Optimizer / Executor ==<br />
<br />
{{TodoItem<br />
|Improve selectivity functions for geometric operators}}<br />
<br />
{{TodoItem<br />
|Consider increasing the default values of from_collapse_limit, join_collapse_limit, and/or geqo_threshold<br />
* [http://archives.postgresql.org/message-id/4136ffa0905210551u22eeb31bn5655dbe7c9a3aed5@mail.gmail.com from_collapse_limit vs. geqo_threshold]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to display optimizer analysis using OPTIMIZER_DEBUG}}<br />
<br />
{{TodoItem<br />
|Log statements where the optimizer row estimates were dramatically different from the number of rows actually found?}}<br />
<br />
{{TodoItem<br />
|Consider compressed annealing to search for query plans<br />
|This might replace GEQO.<br />
* http://archives.postgresql.org/message-id/15658.1241278636%40sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Improve use of expression indexes for ORDER BY <br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01553.php <nowiki>Resjunk sort columns, Heikki's index-only quals patch, and bug #5000</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Modify the planner to better estimate caching effects<br />
* http://archives.postgresql.org/pgsql-performance/2010-11/msg00117.php<br />
}}<br />
<br />
=== Hashing ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Consider using a hash for joining to a large IN (VALUES ...) list<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00450.php <nowiki>Planning large IN lists</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single batch hash joins to preserve outer pathkeys<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00806.php Re: Potential Join Performance Issue]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|"lazy" hash tables - look up only the tuples that are actually requested<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid building the same hash table more than once during the same query<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid hashing for distinct and then re-hashing for hash join<br />
* [http://archives.postgresql.org/message-id/4136ffa0902191346g62081081v8607f0b92c206f0a@mail.gmail.com Re: Fixing Grittner's planner issues]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow hashing to be used on arrays, if the element type is hashable<br />
* http://archives.postgresql.org/message-id/11087.1244905821@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Background Writer ==<br />
<br />
{{TodoItem<br />
|Consider having the background writer update the transaction status hint bits before writing out the page<br />
|Implementing this requires the background writer to have access to system catalogs and the transaction status log.}}<br />
<br />
{{TodoItem<br />
|Consider adding buffers the background writer finds reusable to the free list <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Automatically tune bgwriter_delay based on activity rather then using a fixed interval<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider whether increasing BM_MAX_USAGE_COUNT improves performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg01007.php <nowiki>Bgwriter LRU cleaning: we've been going at this all wrong</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Test to see if calling PreallocXlogFiles() from the background writer will help with WAL segment creation latency<br />
* [http://archives.postgresql.org/pgsql-patches/2007-06/msg00340.php <nowiki>Re: Load Distributed Checkpoints, final patch</nowiki>]<br />
}}<br />
<br />
== Concurrent Use of Resources ==<br />
<br />
{{TodoItem<br />
|Do async I/O for faster random read-ahead of data<br />
|Async I/O allows multiple I/O requests to be sent to the disk with results coming back asynchronously.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00820.php <nowiki>Asynchronous I/O Support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-09/msg00255.php <nowiki>Re: random_page_costs - are defaults of 4.0 realistic for SCSI RAID 1</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00027.php <nowiki>There's random access and then there's random access</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-01/msg00170.php <nowiki>Bitmap index scan preread using posix_fadvise (Was: There's random access and then there's random access)</nowiki>]<br />
The above patch is already applied as of 8.4, but it still remains to figure out how to handle plain indexscans effectively.<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-01/msg00806.php Problems with the patch submitted for posix_fadvise in index scans]<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better I/O utilization<br />
|This would allow a single query to make use of multiple I/O channels simultaneously. One idea is to create a background reader that can pre-fetch sequential and index scan pages needed by other backends. This could be expanded to allow concurrent reads from multiple devices in a partitioned table.<br />
* http://archives.postgresql.org/pgsql-performance/2011-02/msg00123.php<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better CPU utilization<br />
|This would allow several CPUs to be used for a single query, such as for sorting or query execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00945.php <nowiki>Multi CPU Queries - Feedback and/or suggestions wanted!</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|SMP scalability improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00439.php <nowiki>Straightforward changes for increased SMP scalability</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00206.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
== TOAST ==<br />
<br />
{{TodoItem<br />
|Allow user configuration of TOAST thresholds<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00213.php <nowiki>Re: Proposed adjustments in MaxTupleSize and toastthresholds</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00082.php <nowiki>pg_lzcompress strategy parameters</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce unnecessary cases of deTOASTing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00895.php <nowiki>Re: [PATCHES] Eliminate more detoast copies for packed varlenas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce costs of repeat de-TOASTing of values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01096.php <nowiki>WIP patch: reducing overhead for repeat de-TOASTing</nowiki>]<br />
}}<br />
<br />
== Miscellaneous Performance ==<br />
<br />
{{TodoItem<br />
|Use mmap() rather than SYSV for shared buffers?<br />
|This would remove the requirement for SYSV SHM but would introduce portability issues. Anonymous mmap (or mmap to /dev/zero) is required to prevent I/O overhead. We could also consider mmap() for writing WAL.<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00750.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00756.php<br />
}}<br />
<br />
{{TodoItem<br />
|Rather than consider mmap()-ing in 8k pages, consider mmap()'ing entire files into a backend?<br />
|Doing I/O to large tables would consume a lot of address space or require frequent mapping/unmapping. Extending the file also causes mapping problems that might require mapping only individual pages, leading to thousands of mappings. Another problem is that there is no way to _prevent_ I/O to disk from the dirty shared buffers so changes could hit disk before WAL is written.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01239.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider ways of storing rows more compactly on disk:<br />
* Reduce the row header size?<br />
* Consider reducing on-disk varlena length from four bytes to two because a heap row cannot be more than 64k in length}}<br />
<br />
{{TodoItem<br />
|Consider transaction start/end performance improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00948.php <nowiki>Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow configuration of backend priorities via the operating system<br />
|Though backend priorities make priority inversion during lock waits possible, research shows that this is not a huge problem.<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg00493.php <nowiki>Priorities for users or queries?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing the minimum allowed number of shared buffers<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00157.php <nowiki>Re: [PATCH] Don't bail with legitimate -N/-B options</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider if CommandCounterIncrement() can avoid its AcceptInvalidationMessages() call<br />
* [http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php <nowiki>pgsql: Avoid incrementing the CommandCounter when</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider Cartesian joins when both relations are needed to form an indexscan qualification for a third relation<br />
* [http://archives.postgresql.org/pgsql-performance/2007-12/msg00090.php <nowiki>Re: TB-sized databases</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider not storing a NULL bitmap on disk if all the NULLs are trailing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00624.php <nowiki>Proposal for Null Bitmap Optimization(for Trailing NULLs)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-12/msg00109.php <nowiki>Re: [HACKERS] Proposal for Null Bitmap Optimization(for TrailingNULLs)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Sort large UPDATE/DELETEs so it is done in heap order<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01119.php <nowiki>Possible future performance improvement: sort updates/deletes by ctid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow one transaction to see tuples using the snapshot of another transaction<br />
|This would assist multiple backends in working together. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00400.php <nowiki>Transaction Snapshot Cloning</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00135.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00260.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00466.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the I/O caused by updating tuple hint bits<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00847.php <nowiki>Hint Bits and Write I/O</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00199.php <nowiki>Re: [HACKERS] Hint Bits and Write I/O</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00695.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00792.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg01063.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01408.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01453.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid the requirement of freezing pages that are infrequently modified <br />
|If all rows on a page are visible, it is possible to set a bit in the visibility map (once the visibility map is 100% reliable) and not need to freeze the page, avoiding a page rewrite<br />
* http://archives.postgresql.org/message-id/4BF701CF.2090205@agliodbs.com<br />
* http://archives.postgresql.org/pgsql-hackers/2010-06/msg00082.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid reading in b-tree pages when replaying vacuum records in hot standby mode<br />
* [http://archives.postgresql.org/message-id/1272571938.4161.14739.camel@ebony <nowiki>Hot Standby tuning for btree_xlog_vacuum()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure truncation logic to be more resistant to failure<br />
|This also involves not writing dirty buffers for a truncated or dropped relation<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01032.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider adding logic to increase large tables by more than 8k<br />
|This would reduce file system fragmentation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00337.php<br />
}}<br />
<br />
== Miscellaneous Other ==<br />
<br />
{{TodoItem<br />
|Deal with encoding issues for filenames in the server filesystem<br />
* {{MessageLink|20090413184335.39BE.52131E4D@oss.ntt.co.jp|a proposed patch here}}<br />
* {{MessageLink|8484.1244655656@sss.pgh.pa.us|some issues about it here}}<br />
* {{MessageLink|20100107103740.97A5.52131E4D@oss.ntt.co.jp|Windows-specific patch here}}<br />
}}<br />
<br />
{{TodoItem<br />
|Deal with encoding issues in the output of localeconv()<br />
* [http://archives.postgresql.org/message-id/40c6d9160904210658y590377cfw6dbbecb53d2b8be0@mail.gmail.com bug report]<br />
* [http://archives.postgresql.org/message-id/49EF8DA0.90008@tpf.co.jp draft patch]<br />
* [http://archives.postgresql.org/message-id/21710.1243620986@sss.pgh.pa.us review of patch]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide schema name and other fields available from SQL GET DIAGNOSTICS in error reports<br />
* [http://archives.postgresql.org/message-id/dcc563d10810211907n3c59a920ia9eb7cd2a6d5ea58@mail.gmail.com <nowiki>How to get schema name which violates fk constraint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg00846.php <nowiki>patch - Report the schema along table name in a referential failure error message</nowiki>]<br />
* {{MessageLink|3191.1263306359@sss.pgh.pa.us|Re: NOT NULL violation and error-message}}<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00213.php <nowiki>the case for machine-readable error fields</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
| Provide [http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html#LIBPQ-CONNECT-FALLBACK-APPLICATION-NAME fallback_application_name] in contrib/pgbench, oid2name, and dblink.<br />
* {{MessageLink|w2g9837222c1004070216u3bc46b3ahbddfdffdbfb46212@mail.gmail.com|fallback_application_name and pgbench}}<br />
}}<br />
<br />
{{TodoItem<br />
|Add 64-bit support to /contrib/pgbench<br />
* http://archives.postgresql.org/pgsql-hackers/2010-07/msg00153.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00705.php<br />
}}<br />
<br />
== Source Code ==<br />
<br />
{{TodoItem<br />
|Add use of 'const' for variables in source tree}}<br />
<br />
{{TodoItemEasy<br />
|Remove warnings created by -Wcast-align}}<br />
<br />
{{TodoItem<br />
|Move platform-specific ps status display info from ps_status.c to ports}}<br />
<br />
{{TodoItem<br />
|Add optional CRC checksum to heap and index pages<br />
|One difficulty is how to prevent hint bit changes from affecting the computed CRC checksum.<br />
* http://archives.postgresql.org/message-id/19934.1226601952%40sss.pgh.pa.us<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00002.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01028.php <nowiki>double-buffering page writes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00524.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01101.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00011.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00249.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a faster CRC32 algorithm<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01112.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow cross-compiling by generating the zic database on the target system}}<br />
<br />
{{TodoItem<br />
|Improve NLS maintenance of libpgport messages linked onto applications}}<br />
<br />
{{TodoItemDone<br />
|Improve the module installation experience (/contrib, etc)<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00132.php <nowiki>modules</nowiki>]<br />
* {{messageLink|ca33c0a30807231640n6fb4035dod8121a18aa1fa29c@mail.gmail.com|Re: PostgreSQL extensions packaging}}<br />
* {{messageLink|ca33c0a30804061349s41b4d8fcsa9c579454b27ecd2@mail.gmail.com|Database owner installable modules patch}}<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-03/msg00855.php <nowiki>Re: contrib function naming, and upgrade issues</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00912.php <nowiki>search_path vs extensions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Use UTF8 encoding for NLS messages so all server encodings can read them properly}}<br />
<br />
{{TodoItem<br />
|Allow creation of universal binaries for Darwin<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00884.php <nowiki>Getting to universal binaries for Darwin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider GnuTLS if OpenSSL license becomes a problem<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00892.php<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00040.php <nowiki>[PATCH] Add support for GnuTLS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01213.php <nowiki>TODO: GNU TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider making NAMEDATALEN more configurable in future releases}}<br />
<br />
{{TodoItem<br />
|Research use of signals and sleep wake ups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00003.php <nowiki>Restartable signals 'n all that</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow C++ code to more easily access backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00302.php <nowiki>Mostly Harmless: Welcoming our C++ friends</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider simplifying how memory context resets handle child contexts<br />
* [http://archives.postgresql.org/pgsql-patches/2007-08/msg00067.php <nowiki>Re: Memory leak in nodeAgg</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Create three versions of libpgport to simplify client code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00154.php <nowiki>8.4 TODO item: make src/port support libpq and ecpg directly</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve detection of shared memory segments being used by others by checking the SysV shared memory field 'nattch'<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00656.php <nowiki>postgresql in FreeBSD jails: proposal</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00673.php <nowiki>Re: postgresql in FreeBSD jails: proposal</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using POSIX shared memory to avoid System V shared memory kernel limits<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00558.php<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the non-threaded Avahi service discovery protocol<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00939.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00097.php <nowiki>Re: Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg01211.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00001.php <nowiki>Re: [HACKERS] Avahi support for Postgresql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce data row alignment requirements on some 64-bit systems<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00369.php <nowiki>[WIP] Reduce alignment requirements on 64-bit systems.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure TOAST internal storage format for greater flexibility<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00049.php <nowiki>Re: PG_PAGE_LAYOUT_VERSION 5 - time for change</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Add regression tests for pg_dump/restore<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01967.php <nowiki>"make install-check-pg_dump" target in src/regress]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Research different memory allocation methods for lists<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01467.php <br />
}}<br />
<br />
=== /contrib/pg_upgrade ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Remove copy_dir() code, or use it<br />
}}<br />
<br />
{{TodoItem<br />
|Handle large object comments<br />
|This is difficult to do because the large object doesn't exist when --schema-only is loaded.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using pg_depend for checking object usage in version.c<br />
}}<br />
<br />
{{TodoItem<br />
|If reindex is necessary, allow it to be done in parallel with pg_dump custom format<br />
}}<br />
<br />
{{TodoItem<br />
|Migrate pg_statistic by dumping it out as a flat file, so analyze is not necessary<br />
|pg_class.oid is not preserved so schema.tablename must be used.<br />
}}<br />
<br />
{{TodoItem<br />
|Improve testing, perhaps using the buildfarm<br />
|The buildfarm has access to multiple versions of PostgreSQL.<br />
}}<br />
<br />
{{TodoItem<br />
|Create machine-readable output of pg_controldata<br />
|This would avoid parsing its output. The problem is we need pg_controldata output from both the old and new clusters so we would need to support both formats.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Windows ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Remove configure.in check for link failure when cause is found}}<br />
<br />
{{TodoItem<br />
|Remove readdir() errno patch when runtime/mingwex/dirent.c rev 1.4 is released}}<br />
<br />
{{TodoItem<br />
|Allow psql to use readline once non-US code pages work with backslashes}}<br />
<br />
{{TodoItem<br />
|Fix problem with shared memory on the Win32 Terminal Server}}<br />
<br />
{{TodoItem<br />
|Improve signal handling<br />
* [http://archives.postgresql.org/pgsql-patches/2005-06/msg00027.php <nowiki>Simplify Win32 Signaling code</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Convert MSVC build system to remove most batch files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00961.php <nowiki>MSVC build system</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support pgxs when using MSVC}}<br />
<br />
{{TodoItem<br />
|Fix MSVC NLS support, like for to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00485.php <nowiki>NLS on MSVC strikes back!</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php <nowiki>Fix for 8.3 MSVC locale (Was [HACKERS] NLS on MSVC strikes back!)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a correct rint() substitute on Windows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00808.php <nowiki>Minor bug in src/port/rint.c</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix global namespace issues when using multiple terminal server sessions<br />
* [http://archives.postgresql.org/message-id/48F3BFCC.8030107@dunslane.net problems with Windows global namespace]}}<br />
<br />
{{TodoItem<br />
|Change from the current autoconf/gmake build system to cmake<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01869.php <nowiki>About CMake (was Re: [COMMITTERS] pgsql: Append major version number and for libraries soname major)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve consistency of path separator usage<br />
* http://archives.postgresql.org/message-id/49C0BDC5.4010002@hagander.net<br />
}}<br />
<br />
{{TodoItem<br />
|Fix cross-compiling on Windows<br />
* http://archives.postgresql.org/pgsql-bugs/2010-10/msg00110.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multiple Postgres clusters running on the same machine to distinguish themselves in the event log<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01297.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00574.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Wire Protocol Changes ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dynamic character set handling}}<br />
<br />
{{TodoItem<br />
|Add decoded type, length, precision}}<br />
<br />
{{TodoItem<br />
|Mark result columns as known-not-null when possible<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-11/msg01029.php <nowiki>Adding nullable indicator to Describe</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more control over planner treatment of statements being prepared}}<br />
<br />
{{TodoItem<br />
|Use compression?}}<br />
<br />
{{TodoItem<br />
|Update clients to use data types, typmod, schema.table.column names of result sets using new statement protocol}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Documentation ==<br />
<br />
{{TodoItem<br />
|Convert single quotes to apostrophes in the PDF documentation<br />
* [http://archives.postgresql.org/pgsql-docs/2007-12/msg00059.php <nowiki>SGML docs and pdf single-quotes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a manpage for postgresql.conf<br />
* {{messageLink|20080819194311.GH4428@alvh.no-ip.org|A smaller default postgresql.conf}}<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Change the manpage-generating toolchain to use the new XML-based docbook2x tools<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing documentation format from SGML to XML<br />
* [http://archives.postgresql.org/pgsql-docs/2006-12/msg00152.php <nowiki>Re: Authoring Tools WAS: Switching to XML</nowiki>]<br />
* http://archives.postgresql.org/pgsql-docs/2011-04/msg00020.php<br />
* http://wiki.postgresql.org/wiki/Switching_PostgreSQL_documentation_from_SGML_to_XML<br />
}}<br />
<br />
{{TodoItem<br />
|Document support for N<nowiki>' '</nowiki> national character string literals, if it matches the SQL standard<br />
* http://archives.postgresql.org/message-id/1275895438.1849.1.camel@fsopti579.F-Secure.com<br />
}}<br />
<br />
{{TodoItem<br />
|Add diagrams to the documentation<br />
* http://archives.postgresql.org/pgsql-docs/2010-07/msg00001.php<br />
}}<br />
<br />
== Exotic Features ==<br />
<br />
{{TodoItem<br />
|Add pre-parsing phase that converts non-ISO syntax to supported syntax<br />
|This could allow SQL written for other databases to run without modification.}}<br />
<br />
{{TodoItem<br />
|Allow plug-in modules to emulate features from other databases}}<br />
<br />
{{TodoItem<br />
|Add features of Oracle-style packages<br />
|A package would be a schema with session-local variables, public/private functions, and initialization functions. It is also possible to implement these capabilities in any schema and not use a separate &quot;packages&quot; syntax at all.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00384.php <nowiki>proposal for PL packages for 8.3.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing control of upper/lower case folding of unquoted identifiers<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-04/msg00818.php <nowiki>Bringing PostgreSQL torwards the standard regarding case folding</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg01527.php <nowiki>Re: [SQL] Case Preservation disregarding case sensitivity?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00849.php <nowiki>TODO Item: Consider allowing control of upper/lower case folding of unquoted, identifiers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add autonomous transactions<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00893.php <nowiki>autonomous transactions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Give query progress indication<br />
* [[Query progress indication]]<br />
}}<br />
<br />
{{TodoItem<br />
|Rethink our type system<br />
* [[Rethinking datatypes]]<br />
}}<br />
<br />
== Features We Do ''Not'' Want ==<br />
<br />
The following features have been discussed ad nauseum on the PostgreSQL mailing lists and the consensus has been that the project is not interested in them. As such, if you are going to bring them up as potential features, you will want to be familiar with all of the arguments against these features which have been previously made over the years. If you decide to work on such features anyway, you should be aware that you face a higher-than-normal barrier to get the Project to accept them.<br />
<br />
{{TodoItem<br />
|All backends running as threads in a single process (not wanted)<br />
|This eliminates the process protection we get from the current setup. Thread creation is usually the same overhead as process creation on modern systems, so it seems unwise to use a pure threaded model, and MySQL and DB2 have demonstrated that threads introduce as many issues as they solve. Threading specific operations such as I/O, seq scans, and connection management has been discussed and will probably be implemented to enable specific performance features. Moving to a threaded engine would also require halting all other work on PostgreSQL for one to two years.}}<br />
<br />
{{TodoItem<br />
|"Oracle-style" optimizer hints (not wanted)<br />
|Optimizer hints, as implemented in Oracle and other RDBMSes, are used to work around problems in the optimizer and introduce upgrade and maintenance issues. We would rather have such problems reported and fixed. We have discussed a more sophisticated system of per-class cost adjustment instead, but a specification remains to be developed. See [[OptimizerHintsDiscussion|Optimizer Hints Discussion]] for further information.}}<br />
<br />
{{TodoItem<br />
|Embedded server (not wanted)<br />
|While PostgreSQL clients runs fine in limited-resource environments, the server requires multiple processes and a stable pool of resources to run reliably and efficiently. Stripping down the PostgreSQL server to run in the same process address space as the client application would add too much complexity and failure cases. Besides, there are several very mature embedded SQL databases already available.}}<br />
<br />
{{TodoItem<br />
|Obfuscated function source code (not wanted)<br />
|Obfuscating function source code has minimal protective benefits because anyone with super-user access can find a way to view the code. At the same time, it would greatly complicate backups and other administrative tasks. To prevent non-super-users from viewing function source code, remove SELECT permission on pg_proc.<br />
* [http://archives.postgresql.org/pgsql-general/2008-09/msg00668.php <nowiki>Obfuscated stored procedures (was Re: Oracle and Postgresql)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Indeterminate behavior for the GROUP BY clause (not wanted)<br />
|At least one other database product allows specification of a subset of the result columns which GROUP BY would need to be able to provide predictable results; the server is free to return any value from the group. This is not viewed as a desirable feature. PostgreSQL 9.1 will allow result columns that are not referenced by GROUP BY if a primary key for the same table is referenced in GROUP BY.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg00297.php <nowiki>Re: SQL compatibility reminder: MySQL vs PostgreSQL</nowiki>]<br />
}}<br />
<br />
</div><br />
<br />
[[Category:Todo]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Todo&diff=14342Todo2011-05-15T15:47:42Z<p>Schmiddy: /* psql */ We're now using pg_table_size() for \dt+ which should include TOAST size (though not index size)</p>
<hr />
<div><div style="margin: 1ex 1em; float: right;"><br />
__TOC__<br />
</div><br />
<br />
This list contains '''all known PostgreSQL bugs and feature requests'''. If you would like to work on an item, please read the [[Developer FAQ]] first. There is also a [[Development_information|development information page]].<br />
<br />
* {{TodoPending}} - marks ordinary, incomplete items<br />
* {{TodoEasy}} - marks items that are easier to implement<br />
* {{TodoDone}} - marks changes that are done, and will appear in the PostgreSQL 9.1 release.<br />
<br />
For help on editing this list, please see [[Talk:Todo]]. <b>Please do not add items here without discussion on the mailing list.</b><br />
<br />
<div style="padding: 1ex 4em;"><br />
== Administration ==<br />
<br />
{{TodoItem<br />
|Allow administrators to cancel multi-statement idle transactions<br />
|This allows locks to be released, but it is complex to report the cancellation back to the client.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01340.php <nowiki>Cancelling idle in transaction state</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00441.php <nowiki>Re: Cancelling idle in transaction state</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Check for unreferenced table files created by transactions that were in-progress when the server terminated abruptly<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00096.php <nowiki>Removing unreferenced files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Set proper permissions on non-system schemas during db creation<br />
|Currently all schemas are owned by the super-user because they are copied from the template1 database. However, since all objects are inherited from the template database, it is not clear that setting schemas to the db owner is correct.}}<br />
<br />
{{TodoItem<br />
|Allow log_min_messages to be specified on a per-module basis<br />
|This would allow administrators to see more detailed information from specific sections of the backend, e.g. checkpoints, autovacuum, etc. Another idea is to allow separate configuration files for each module, or allow arbitrary SET commands to be passed to them. See also [[Logging Brainstorm]].}}<br />
<br />
{{TodoItem<br />
|Simplify creation of partitioned tables<br />
|This would allow creation of partitioned tables without requiring creation of triggers or rules for INSERT/UPDATE/DELETE, and constraints for rapid partition selection. Options could include range and hash partition selection. See also [[Table partitioning]]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow auto-selection of partitioned tables for min/max() operations<br />
|There was a patch on -hackers from July 2009, but it has not been merged: [http://archives.postgresql.org/pgsql-hackers/2009-07/msg01115.php <nowiki>MIN/MAX optimization for partitioned table</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Allow custom variables to appear in pg_settings()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00850.php <nowiki>Re: count(*) performance improvement ideas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have custom variables be transaction-safe<br />
* {{MessageLink|4B577E9F.8000505@dunslane.net|Custom GUCs still a bit broken}}<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the SQL-standard mechanism whereby REVOKE ROLE revokes only the privilege granted by the invoking role, and not those granted by other roles<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00010.php <nowiki>Re: Grantor name gets lost when grantor role dropped</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Improve server security options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01875.php <nowiki>Re: [0/4] Proposal of SE-PostgreSQL patches</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00000.php <nowiki>Re: [0/4] Proposal of SE-PostgreSQL patches</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent query cancel packets from being replayed by an attacker, especially when using SSL<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00345.php <nowiki>Replay attack of query cancel</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a way to query the log collector subprocess to determine the name of the currently active log file<br />
* [http://archives.postgresql.org/pgsql-general/2008-11/msg00418.php <nowiki>Current log files when rotating?</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow the client to authenticate the server in a Unix-domain socket connection, e.g., using SO_PEERCRED<br />
* http://archives.postgresql.org/message-id/20090401173756.GB21229@svana.org<br />
}}<br />
<br />
{{TodoItem<br />
|Allow simpler reporting of the unix domain socket directory and allow easier configuration of its default location<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg01555.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow custom daemons to be automatically stopped/started along with the postmaster<br />
|This allows easier administration of daemons like user job schedulers or replication-related daemons.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01701.php <nowiki>Re: scheduler in core</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Increase maximum values for max_standby_streaming_delay and log_min_duration_statement<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg01517.php<br />
* Committed: http://archives.postgresql.org/pgsql-committers/2011-03/msg00210.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve logging of prepared transactions recovered during startup<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00092.php <nowiki>&quot;recovering prepared transaction&quot; after server restart message</nowiki>]<br />
}}<br />
<br />
=== Configuration files ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Allow pg_hba.conf to specify host names along with IP addresses<br />
|Host name lookup could occur when the postmaster reads the pg_hba.conf file, or when the backend starts. Another solution would be to reverse lookup the connection IP and check that hostname against the host names in pg_hba.conf. We could also then check that the host name maps to the IP address. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00569.php <nowiki>TODO Item: Allow pg_hba.conf to specify host names along with IP addresses</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00613.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf file values to be changed via an SQL API, perhaps using SET GLOBAL<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00764.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider normalizing fractions in postgresql.conf, perhaps using '%'<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00550.php <nowiki>Fractions in GUC variables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow Kerberos to disable stripping of realms so we can check the username@realm against multiple realms<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00009.php <nowiki>krb_match_realm patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve LDAP authentication configuration options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01745.php <nowiki>Proposed Patch - LDAPS support for servers on port 636 w/o TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add external tool to auto-tune some postgresql.conf parameters<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00000.php <nowiki>Re: Overhauling GUCS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00033.php <nowiki>Simple postgresql.conf wizard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add 'hostgss' pg_hba.conf option to allow GSS link-level encryption<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01454.php <nowiki>Re: Plans for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Process pg_hba.conf keywords as case-insensitive<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00432.php <nowiki>More robust pg_hba.conf parsing/error logging</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Have pg_hba.conf consider "replication" special only in the database field<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00632.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Rename unix domain socket 'ident' connections to 'peer', to avoid confusion with TCP 'ident'<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01053.php<br />
}}<br />
<br />
{{TodoItem<br />
|Create utility to compute accurate random_page_cost value}}<br />
<br />
{{TodoItem<br />
|Allow configuration files to be independently validated<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01831.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf settings to be accepted by backends even if some settings are invalid for those backends<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00330.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00375.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow all backends to receive postgresql.conf setting changes at the same time<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00330.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00375.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Tablespaces ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow a database in tablespace t1 with tables created in tablespace t2 to be used as a template for a new database created with default tablespace t2<br />
|Currently all objects in the default database tablespace must have default tablespace specifications. This is because new databases are created by copying directories. If you mix default tablespace tables and tablespace-specified tables in the same directory, creating a new database from such a mixed directory would create a new database with tables that had incorrect explicit tablespaces. To fix this would require modifying pg_class in the newly copied database, which we don't currently do.}}<br />
<br />
{{TodoItem<br />
|Allow reporting of which objects are in which tablespaces<br />
|This item is difficult because a tablespace can contain objects from multiple databases. There is a server-side function that returns the databases which use a specific tablespace, so this requires a tool that will call that function and connect to each database to find the objects in each database for that tablespace.}}<br />
<br />
{{TodoItem<br />
|Allow WAL replay of CREATE TABLESPACE to work when the directory structure on the recovery computer is different from the original}}<br />
<br />
{{TodoItem<br />
|Allow per-tablespace quotas}}<br />
<br />
{{TodoItem<br />
|Allow tablespaces on RAM-based partitions for unlogged tables<br />
* http://archives.postgresql.org/pgsql-advocacy/2011-05/msg00033.php<br />
}}<br />
<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Statistics Collector ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow statistics last vacuum/analyze execution times to be displayed without requiring track_counts to be enabled<br />
* [http://archives.postgresql.org/pgsql-docs/2007-04/msg00028.php <nowiki>row-level stats and last analyze time</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Clear table counters on TRUNCATE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00169.php <nowiki>Small TRUNCATE glitch</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Allow the clearing of cluster-level statistics<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-03/msg00917.php <nowiki>Resetting cluster-wide statistics</nowiki>]<br />
* ''pg_stat_reset_shared('bgwriter')'' (9.0) now handles the ''pg_stat_bgwriter'' subset of this<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SSL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SSL authentication/encryption over unix domain sockets<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00924.php <nowiki>Re: Spoofing as the postmaster</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL key file permission checks to be optionally disabled when sharing SSL keys with other applications<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php <nowiki>BUG #3809: SSL &quot;unsafe&quot; private key permissions bug</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL CRL files to be re-read during configuration file reload, rather than requiring a server restart<br />
|Unlike SSL CRT files, CRL (Certificate Revocation List) files are updated frequently<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00832.php <nowiki>Automatic CRL reload</nowiki>]<br />
Alternatively or additionally supporting OCSP (online certificate security protocol) would provide real-time revocation discovery without reloading<br />
}}<br />
<br />
{{TodoItem<br />
| Allow automatic selection of SSL client certificates from a certificate store<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00406.php <nowiki>Allow multiple certificates or keys in the postgresql.crt/.key files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Send the full certificate server chain to the client<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-12/msg00145.php BUG #5245: Full Server Certificate Chain Not Sent to client]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Point-In-Time Recovery (PITR) ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Create dump tool for write-ahead logs for use in determining transaction id for point-in-time recovery<br />
|This is useful for checking PITR recovery.}}<br />
<br />
{{TodoItemDone<br />
|Allow recovery.conf to support the same syntax as postgresql.conf, including quoting<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00497.php <nowiki>recovery.conf parsing problems</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg00684.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow archive_mode to be changed without server restart?<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01655.php <nowiki>Enabling archive_mode without restart</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider avoiding WAL switching via archive_timeout if there has been no database activity<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01469.php <nowiki>archive_timeout behavior for no activity</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00395.php <nowiki>Re: archive_timeout behavior for no activity</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Expose pg_controldata via an SQL interface<br />
|Helpful for monitoring replicated databases<br />
* http://archives.postgresql.org/message-id/4B901D73.8030003@agliodbs.com<br />
* [http://archives.postgresql.org/message-id/4B959D7A.6010907@joeconway.com initial patch]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Standby server mode ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
| Allow pg_xlogfile_name() to be used in recovery mode<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1001190135vd9f62f1sa7868abc1ea61d12@mail.gmail.com <nowiki>Streaming replication and pg_xlogfile_name()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Prevent variables inherited from the server environment from begin used for making streaming replication connections.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01011.php <nowiki>Re: Parameter name standby_mode</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Add a new privilege for connecting for streaming replication<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1003040247p6b092241of91784a505e9abd8@mail.gmail.com <nowiki>Streaming replication and privilege</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
| Add support for synchronous replication.<br />
}}<br />
<br />
{{TodoItemDone<br />
| Add capability to take and send a base backup over the streaming replication connection, making it possible to initialize a new standby server from a running primary server without a WAL archive or other access to the primary server. <br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00136.php<br />
}}<br />
<br />
{{TodoItem<br />
| Allow hot file system backups on standby servers<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01727.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php<br />
}}<br />
<br />
{{TodoItemDone<br />
| Allow the automatic removal of old directories when streaming base backups<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00558.php<br />
}}<br />
<br />
{{TodoItem<br />
| Change walsender so that it applies per-role settings<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00642.php<br />
}}<br />
<br />
{{TodoItem<br />
| Add more control over waiting for synchronous commit<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01611.php<br />
}}<br />
<br />
{{TodoItem<br />
| Restructure configuration parameters for standby mode<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01820.php<br />
}}<br />
<br />
{{TodoItem<br />
| Allow time-delayed application of logs on the standby<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00992.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Data Types ==<br />
<br />
{{TodoItemDone<br />
|Reduce storage space for small NUMERICs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01331.php <nowiki>Saving space for common kinds of numeric values</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-02/msg00505.php <nowiki>Numeric patch to add special-case representations for &lt; 8 bytes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php <nowiki>Re: Reducing NUMERIC size for 8.3</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix data types where equality comparison is not intuitive, e.g. box}}<br />
<br />
{{TodoItem<br />
|Add support for public SYNONYMs<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00519.php <nowiki>Proposal for SYNONYMS</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php<br />
* http://archives.postgresql.org/pgsql-general/2010-12/msg00139.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for SQL-standard GENERATED/IDENTITY columns<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg00543.php <nowiki>Re: Three weeks left until feature freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00038.php <nowiki>GENERATED ... AS IDENTITY, Was: Re: Feature Freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00344.php <nowiki>Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00076.php <nowiki>Re: [HACKERS] Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00604.php <nowiki>IDENTITY/GENERATED patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider placing all sequences in a single table, or create a system view<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a special data type for regular expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg01067.php <nowiki>Why is there a tsquery data type?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce BIT data type overhead using short varlena headers<br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00273.php <nowiki>storage size of &quot;bit&quot; data type..</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow adding enumerated values to an existing enumerated data type<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01718.php <nowiki>Re: [COMMITTERS] pgsql: Update: &lt; * Allow adding enumerated values to an existing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow renaming and deleting enumerated values from an existing enumerated data type<br />
}}<br />
<br />
{{TodoItem<br />
|Support scoped IPv6 addresses in the inet type<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00111.php <nowiki>strange problem with ip6</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a JSON (JavaScript Object Notation) data type<br />
|This would behave similar to the XML data type, which is stored as text, but allows element lookup and conversion functions.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01494.php <nowiki>PATCH: Add hstore_to_json()</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg00001.php <nowiki>Re: PATCH: Add hstore_to_json()</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg01092.php <nowiki>Proposal: Add JSON support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00057.php <nowiki>Re: Proposal: Add JSON support</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00481.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01694.php<br />
}}<br />
<br />
{{TodoItem<br />
|Considering improving performance of computing CHAR() value lengths<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00900.php <nowiki>char() overhead on read-only workloads not so insignifcant as the docs claim it is...</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01787.php <nowiki>Re: [PATCH] backend: compare word-at-a-time in bcTruelen</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add overlaps geometric operators that ignore point overlaps<br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00861.php<br />
}}<br />
<br />
=== Domains ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow functions defined as casts to domains to be called during casting<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-05/msg00072.php <nowiki>bug? non working casts for domain</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg01681.php <nowiki>TODO: Fix CREATE CAST on DOMAINs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow values to be cast to domain types<br />
* [http://archives.postgresql.org/pgsql-hackers/2003-06/msg01206.php <nowiki>Domain casting still doesn't work right</nowiki>] <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00289.php <nowiki>domain casting?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make domains work better with polymorphic functions<br />
* [http://archives.postgresql.org/message-id/4887.1228700773@sss.pgh.pa.us Polymorphic types vs. domains]<br />
* [http://archives.postgresql.org/message-id/15535.1238774571@sss.pgh.pa.us some difficulties with fixing it]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Dates and Times ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow infinite intervals just like infinite timestamps}}<br />
<br />
{{TodoItem<br />
|Allow TIMESTAMP WITH TIME ZONE to store the original timezone information, either zone name or offset from UTC<br />
|If the TIMESTAMP value is stored with a time zone name, interval computations should adjust based on the time zone rules. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-10/msg00705.php <nowiki>timestamp with time zone a la sql99</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have timestamp subtraction not call justify_hours()?<br />
* [http://archives.postgresql.org/pgsql-sql/2006-10/msg00059.php <nowiki>timestamp subtraction (was Re: formatting intervals with to_char)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve TIMESTAMP WITH TIME ZONE subtraction to be DST-aware<br />
|Currently subtracting one date from another that crosses a daylight savings time adjustment can return '1 day 1 hour', but adding that back to the first date returns a time one hour in the future. This is caused by the adjustment of '25 hours' to '1 day 1 hour', and '1 day' is the same time the next day, even if daylight savings adjustments are involved.}}<br />
<br />
{{TodoItem<br />
|Fix interval display to support values exceeding 2^31 hours}}<br />
<br />
{{TodoItem<br />
|Add overflow checking to timestamp and interval arithmetic}}<br />
<br />
{{TodoItem<br />
|Add function to allow the creation of timestamps using parameters<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00232.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Arrays ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add support for arrays of domains<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00114.php <nowiki>Re: updated WIP: arrays of composites</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single-byte header storage for array elements}}<br />
<br />
{{TodoItem<br />
|Add function to detect if an array is empty<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00475.php <nowiki>Re: array_length()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of empty arrays<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01033.php <nowiki>So what's an &quot;empty&quot; array anyway?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULLs in arrays<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-11/msg00009.php <nowiki>BUG #4509: array_cat's null behaviour is inconsistent</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01040.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Binary Data ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve vacuum of large objects, like contrib/vacuumlo?}}<br />
<br />
{{TodoItem<br />
|Auto-delete large objects when referencing row is deleted<br />
|contrib/lo offers this functionality.}}<br />
<br />
{{TodoItem<br />
|Allow read/write into TOAST values like large objects<br />
|This requires the TOAST column to be stored EXTERNAL.}}<br />
<br />
{{TodoItem<br />
|Add API for 64-bit large object access<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00781.php <nowiki>64-bit API for large objects</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01790.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== MONEY Data Type ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add locale-aware MONEY type, and support multiple currencies<br />
* [http://archives.postgresql.org/pgsql-general/2005-08/msg01432.php <nowiki>A real currency type</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01181.php <nowiki>Money type todos?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|MONEY dumps in a locale-specific format making it difficult to restore to a system with a different locale}}<br />
<br />
{{TodoItemDone<br />
|Allow MONEY to be easily cast to/from other numeric data types}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Text Search ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dictionaries to change the token that is passed on to later dictionaries<br />
* [http://archives.postgresql.org/pgsql-patches/2007-11/msg00081.php <nowiki>a tsearch2 (8.2.4) dictionary that only filters out stopwords</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a function-based API for '@@' searches<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00511.php <nowiki>Simplifying Text Search</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve text search error messages<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00966.php <nowiki>Poorly designed tsearch NOTICEs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01146.php <nowiki>Re: Poorly designed tsearch NOTICEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing error to warning for strings larger than one megabyte<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00190.php <nowiki>BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00062.php <nowiki>Re: [BUGS] BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|tsearch and tsdicts regression tests fail in Turkish locale on glibc<br />
* [http://archives.postgresql.org/message-id/49749645.5070801@gmx.net tsearch with Turkish locale]<br />
}}<br />
<br />
{{TodoItem<br />
|tsquery negator operator treated as part of lexeme<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-06/msg00346.php BUG #4887: inclusion operator (@>) on tsqeries behaves not conforming to documentation]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of plus signs in email address user names, and perhaps improve URL parsing<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00772.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve default parser, to more easily allow adding new tokens<br />
* http://archives.postgresql.org/message-id/23485.1297727826@sss.pgh.pa.us<br />
}}<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== XML ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow XML arrays to be cast to other data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00981.php <nowiki>proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00231.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00471.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add XML Schema validation and xmlvalidate functions (SQL:2008)}}<br />
<br />
{{TodoItem<br />
|Add xmlvalidatedtd variant to support validating against a DTD?}}<br />
<br />
{{TodoItem<br />
|Relax-NG validation; libxml2 supports this already}}<br />
<br />
{{TodoItem<br />
|Allow reliable XML operation non-UTF8 server encodings (xpath(), in particular, is known to not work)<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-01/msg00135.php <nowiki>BUG #4622: xpath only work in utf-8 server encoding</nowiki>] <br />
* http://archives.postgresql.org/message-id/4110.1238973350@sss.pgh.pa.us}}<br />
<br />
{{TodoItem<br />
|Add functions from SQL:2006: XMLDOCUMENT, XMLCAST, XMLTEXT}}<br />
<br />
{{TodoItem<br />
|Add XMLNAMESPACES support in XMLELEMENT and elsewhere}}<br />
<br />
{{TodoItem<br />
|Move XSLT from contrib/xml2 to a more reasonable location<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00539.php<br />
}}<br />
<br />
{{TodoItem<br />
|Report errors returned by the XSLT library<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00562.php<br />
}}<br />
<br />
{{TodoItem<br />
|Improve the XSLT parameter passing API<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00416.php<br />
}}<br />
<br />
{{TodoItem<br />
|XML Canonical: Convert XML documents to canonical form to compare them. libxml2 has support for this.}}<br />
<br />
{{TodoItem<br />
|Add pretty-printed XML output option<br />
|Parse a document and serialize it back in some indented form. libxml2 might support this.}}<br />
<br />
{{TodoItem<br />
|Add XMLQUERY (from the SQL/XML standard)}}<br />
<br />
{{TodoItem<br />
|Allow XML sthredding<br />
|In some cases shredding could be better option (if there is no need to keep XML docs entirely, e.g. if we have already developed tools that understand only relational data. This would be a separate module that implements annotated schema decomposition technique, similar to DB2 and SQL Server functionality.}}<br />
<br />
{{TodoItem<br />
|Fix Nested or repeated xpath() that apparently mess up namespaces [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00097.php] [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00144.php] [http://archives.postgresql.org/pgsql-general/2008-03/msg00295.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/message-id/004f01c90e91$138e9d10$3aabd730$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItem<br />
|XPath: Adding the <x> at the root causes problems [http://archives.postgresql.org/pgsql-bugs/2008-05/msg00184.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/pgsql-general/2008-07/msg00613.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table needs to be implemented/implementable to get rid of contrib/xml2 [http://archives.postgresql.org/pgsql-general/2008-05/msg00823.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table is pretty broken anyway [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02424.php]}}<br />
<br />
{{TodoItem<br />
|better handling of XPath data types [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00616.php] [http://archives.postgresql.org/message-id/004a01c90e90$4b986d90$e2c948b0$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItemDone<br />
|xpath_exists() is needed<br />
|This checks whether or not the path specified exists in the XML value. Without this function we need to use the weird "array_dims(xpath(...)) IS NOT NULL" syntax.}}<br />
<br />
{{TodoItem<br />
|Improve handling of PIs and DTDs in xmlconcat() [http://archives.postgresql.org/message-id/200904211211.n3LCB09p008988@wwwmaster.postgresql.org]}}<br />
<br />
{{TodoItem<br />
|Restructure XML and /contrib/xml2 functionality<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02314.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00017.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Functions ==<br />
<br />
{{TodoItem<br />
|Allow INET subnet comparisons using non-constants to be indexed}}<br />
<br />
{{TodoItem<br />
|Add an INET overlaps operator, for use by exclusion constraints <br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00845.php<br />
}}<br />
<br />
{{TodoItem<br />
|Enforce typmod for function inputs, function results and parameters for spi_prepare'd statements called from PLs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01403.php <nowiki>Re: BUG #2917: spi_prepare doesn't accept typename aliases</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01160.php <nowiki>RFC for adding typmods to functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix IS OF so it matches the ISO specification, and add documentation<br />
* [http://archives.postgresql.org/pgsql-patches/2003-08/msg00060.php <nowiki>Re: [HACKERS] IS OF</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00060.php <nowiki>ToDo: add documentation for operator IS OF</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement Boyer-Moore searching in LIKE queries<br />
* {{messageLink|27645.1220635769@sss.pgh.pa.us|TODO item: Implement Boyer-Moore searching (First time hacker)}}<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent malicious functions from being executed with the permissions of unsuspecting users<br />
|Index functions are safe, so VACUUM and ANALYZE are safe too. Triggers, CHECK and DEFAULT expressions, and rules are still vulnerable. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00268.php <nowiki>Some notes about the index-functions security vulnerability</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce memory usage of aggregates in set returning functions<br />
* [http://archives.postgresql.org/pgsql-performance/2008-01/msg00031.php <nowiki>Re: Performance of aggregates over set-returning functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/ltree operator<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-11/msg00044.php <nowiki>BUG #3720: wrong results at using ltree</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/btree_gist's implementation of inet indexing<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-10/msg00099.php <nowiki>BUG #5705: btree_gist: Index on inet changes query result</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|<nowiki>Fix inconsistent precedence of =, &gt;, and &lt; compared to &lt;&gt;, &gt;=, and &lt;=</nowiki><br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00145.php <nowiki>BUG #3822: Nonstandard precedence for comparison operators</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix regular expression bug when using complex back-references<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00000.php <nowiki>BUG #3645: regular expression back references seem broken</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have /contrib/dblink reuse unnamed connections<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00895.php <nowiki>dblink un-named connection doesn't get re-used</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve formatting of pg_get_viewdef() output<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01648.php <nowiki>pg_get_viewdef formattiing</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01885.php <nowiki>Re: pretty print viewdefs</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add printf()-like functionality<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add function to dump pg_depend information cleanly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00226.php <nowiki>Elementary dependency look-up</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve relation size functions such as pg_relation_size() to avoid producing an error when called against a no longer visible relation<br />
* [http://archives.postgresql.org/message-id/28488.1286461610@sss.pgh.pa.us pg_relation_size / could not open relation with OID #]<br />
}}<br />
<br />
=== Character Formatting ===<br />
<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Allow to_date() and to_timestamp() to accept localized month names}}<br />
<br />
{{TodoItem<br />
|Add missing parameter handling in to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg00948.php <nowiki>Re: to_char and i18n</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Throw an error from to_char() instead of printing a string of "#" when a number doesn't fit in the desired output format.<br />
* discussed in [http://archives.postgresql.org/message-id/37ed240d0907290836w42187222n18664dfcbcb445b1@mail.gmail.com "to_char, support for EEEE format"]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow to_char() on interval values to accumulate the highest unit requested<br />
|2= Some special format flag would be required to request such accumulation. Such functionality could also be added to EXTRACT. Prevent accumulation that crosses the month/day boundary because of the uneven number of days in a month.<br />
* to_char(INTERVAL '1 hour 5 minutes', 'MI') =&gt; 65<br />
* to_char(INTERVAL '43 hours 20 minutes', 'MI' ) =&gt; 2600<br />
* to_char(INTERVAL '43 hours 20 minutes', 'WK:DD:HR:MI') =&gt; 0:1:19:20<br />
* to_char(INTERVAL '3 years 5 months','MM') =&gt; 41<br />
}}<br />
<br />
{{TodoItem<br />
|Fix to_number() handling for values not matching the format string<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01447.php <nowiki>Re: numeric_to_number() function skipping some digits</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Multi-Language Support ==<br />
<br />
{{TodoItem<br />
|Add NCHAR (as distinguished from ordinary varchar),}}<br />
<br />
{{TodoItemDone<br />
|Allow more fine-grained collation selection<br />
|Right now the collation is fixed at database creation time.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-03/msg00932.php <nowiki>Re: Patch for collation using ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-08/msg00039.php <nowiki>FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-08/msg00309.php <nowiki>Re: FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00110.php <nowiki>Proof of concept COLLATE support with patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-09/msg00020.php <nowiki>For review: Initial support for COLLATE</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01121.php <nowiki>Proposed COLLATE implementation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-01/msg00767.php <nowiki>TODO item: locale per database patch (new iteration)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-03/msg00233.php <nowiki>Re: FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg00662.php <nowiki>Re: Fixed length data types issue</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00557.php <nowiki>[WIP] collation support revisited (phase 1)</nowiki>]<br />
* [[Todo:Collate]]<br />
* [[Todo:ICU]]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01362.php <nowiki>WIP patch: Collation support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00012.php <nowiki>Re: WIP patch: Collation support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00868.php <nowiki>PGDay.it collation discussion notes</nowiki>]<br />
* [http://www.unicode.org/unicode/reports/tr10/ Unicode collation algorithm]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a cares-about-collation column to pg_proc, so that unresolved-collation errors can be thrown at parse time<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-03/msg01520.php <nowiki>Open issues for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Integrate collations with text search configurations<br />
* [http://archives.postgresql.org/message-id/28887.1303579034@sss.pgh.pa.us <nowiki>Some TODO items for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Integrate collations with to_char() and related functions<br />
* [http://archives.postgresql.org/message-id/28887.1303579034@sss.pgh.pa.us <nowiki>Some TODO items for collations</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a LOCALE option to CREATE DATABASE, as a shorthand<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00119.php <nowiki> Re: 8.4 open items list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support multiple simultaneous character sets, per SQL:2008}}<br />
<br />
{{TodoItem<br />
|Improve UTF8 combined character handling?}}<br />
<br />
{{TodoItem<br />
|Add octet_length_server() and octet_length_client()}}<br />
<br />
{{TodoItem<br />
|Make octet_length_client() the same as octet_length()?}}<br />
<br />
{{TodoItem<br />
|Fix problems with wrong runtime encoding conversion for NLS message files}}<br />
<br />
{{TodoItem<br />
|Add URL to more complete multi-byte regression tests<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-07/msg00272.php <nowiki>Multi-byte and client side character encoding tests for copy command..</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix contrib/fuzzystrmatch to work with multibyte encodings<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00047.php <nowiki> soundex function returns UTF-16 characters</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00138.php <nowiki> dmetaphone woes</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Set client encoding based on the client operating system encoding<br />
|Currently client_encoding is set in postgresql.conf, which defaults to the server encoding. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg01696.php <nowiki>Re: [GENERAL] invalid byte sequence ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Change memory allocation for multi-byte functions so memory is allocated inside conversion functions<br />
|Currently we preallocate memory based on worst-case usage.}}<br />
<br />
{{TodoItem<br />
|Add ability to use case-insensitive regular expressions on multi-byte characters<br />
|Currently it works for UTF-8, but not other multi-byte encodings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00433.php <nowiki>Regexps vs. locale</nowiki>]<br />
* {{MessageLink|20091201210024.B1393753FB7@cvs.postgresql.org|A partial solution for UTF-8}}<br />
}}<br />
<br />
{{TodoItem<br />
|Improve encoding of connection startup messages sent to the client<br />
|Currently some authentication error messages are sent in the server encoding<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00801.php <nowiki>encoding of PostgreSQL messages</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-01/msg00005.php <nowiki>Re: encoding of PostgreSQL messages</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have pg_stat_activity display query strings in the correct client encoding<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00131.php <nowiki>pg_stats queries versus per-database encodings</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|More sensible support for Unicode combining characters, normal forms<br />
* http://archives.postgresql.org/message-id/200904141532.44618.peter_e@gmx.net<br />
}}<br />
<br />
== Views / Rules ==<br />
<br />
{{TodoItem<br />
|Automatically create rules on views so they are updateable, per SQL:2008<br />
|We can only auto-create rules for simple views. For more complex cases users will still have to write rules manually.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00586.php <nowiki>Proposal for updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-08/msg00255.php <nowiki>Updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01746.php <nowiki>Re: [COMMITTERS] pgsql: Automatic view update rules Bernd Helmle</nowiki>]<br />
* http://wiki.postgresql.org/wiki/Updatable_views<br />
}}<br />
<br />
{{TodoItem<br />
|Add the functionality of the WITH CHECK OPTION clause to CREATE VIEW}}<br />
<br />
{{TodoItem<br />
|Allow VIEW/RULE recompilation when the underlying tables change<br />
|This is both difficult and controversial.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01723.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01724.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
}}<br />
<br />
{{TodoItem<br />
|Make it possible to use RETURNING together with conditional DO INSTEAD rules, such as for partitioning setups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00577.php <nowiki>RETURNING and DO INSTEAD ... Intentional or not?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add the ability to automatically create materialized views<br />
|Right now materialized views require the user to create triggers on the main table to keep the summary table current. SQL syntax should be able to manage the triggers and summary table automatically. A more sophisticated implementation would automatically retrieve from the summary table when the main table is referenced, if possible. See [[Materialized Views]] for implementation details<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00479.php <nowiki>GSoC - proposal - Materialized Views in PostgreSQL</nowiki>] <br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to modify views via ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00691.php <nowiki>Re: idea: storing view source in system catalogs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01410.php <nowiki>modifying views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00300.php <nowiki>Re: patch: Add columns via CREATE OR REPLACE VIEW</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent low-cost functions from seeing unauthorized view rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg01346.php <nowiki>Using views for row-level access control is leaky</nowiki>]<br />
}}<br />
<br />
== SQL Commands ==<br />
<br />
{{TodoItem<br />
|Add CORRESPONDING BY to UNION/INTERSECT/EXCEPT}}<br />
<br />
{{TodoItem<br />
|Improve type determination of unknown (NULL or quoted literal) result columns for UNION/INTERSECT/EXCEPT<br />
* [http://archives.postgresql.org/message-id/9799.1302719551@sss.pgh.pa.us <nowiki>UNION construct type cast gives poor error message</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ROLLUP, CUBE, GROUPING SETS options to GROUP BY<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00838.php <nowiki>WIP: grouping sets support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00466.php <nowiki>Implementation of GROUPING SETS (T431: Extended grouping capabilities)</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Fix TRUNCATE ... RESTART IDENTITY so its effect on sequences is rolled back on transaction abort<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00550.php <nowiki>Re: [PATCHES] TRUNCATE TABLE with IDENTITY</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow prepared transactions with temporary tables created and dropped in the same transaction, and when an ON COMMIT DELETE ROWS temporary table is accessed<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00047.php <nowiki>Re: &quot;could not open relation 1663/16384/16584: No such file or directory&quot; in a specific combination of transactions with temp tables</nowiki>]<br />
* [http://archives.postgresql.org/message-id/492543D5.9050904@enterprisedb.com A suggestion on how to implement this]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a GUC variable to warn about non-standard SQL usage in queries}}<br />
<br />
{{TodoItem<br />
|Add SQL-standard MERGE/REPLACE/UPSERT command<br />
|MERGE is typically used to merge two tables. REPLACE or UPSERT command does UPDATE, or on failure, INSERT. See [[SQL MERGE]] for notes on the implementation details.<br />
}}<br />
<br />
{{TodoItem<br />
|Add NOVICE output level for helpful messages<br />
|For example, have it warn about unjoined tables. This could also control automatic sequence/index creation messages.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow NOTIFY in rules involving conditionals}}<br />
<br />
{{TodoItem<br />
|Allow EXPLAIN to identify tables that were skipped because of constraint_exclusion<br />
}}<br />
<br />
{{TodoItemDone<br />
|Enable standard_conforming_strings by default<br />
|When this is done, backslash-quote should be prohibited in non-E<nowiki>''</nowiki> strings because of possible confusion over how such strings treat backslashes. Basically, <nowiki>''</nowiki> is always safe for a literal single quote, while \' might or might not be based on the backslash handling rules.}}<br />
<br />
{{TodoItem<br />
|Simplify dropping roles that have objects in several databases}}<br />
<br />
{{TodoItem<br />
|Allow the count returned by SELECT, etc to be represented as an int64 to allow a higher range of values}}<br />
<br />
{{TodoItem<br />
|Add support for WITH RECURSIVE ... CYCLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00291.php <nowiki>WITH RECURSIVE ... CYCLE in vanilla SQL: issues with arrays of rows</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Add DEFAULT .. AS OWNER so permission checks are done as the table owner<br />
|This would be useful for SERIAL nextval() calls and CHECK constraints.}}<br />
<br />
{{TodoItem<br />
|Allow DISTINCT to work in multiple-argument aggregate calls}}<br />
<br />
{{TodoItem<br />
|Add column to pg_stat_activity that shows the progress of long-running commands like CREATE INDEX and VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00203.php <nowiki>EXPLAIN progress info</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow INSERT/UPDATE/DELETE ... RETURNING in common table expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00472.php <nowiki>Writeable CTEs and side effects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add comments on system tables/columns using the information in catalogs.sgml<br />
|Ideally the information would be pulled from the SGML file automatically.}}<br />
<br />
{{TodoItem<br />
|Prevent the specification of conflicting transaction read/write options<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00684.php <nowiki>Re: SET TRANSACTION and SQL Standard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support LATERAL subqueries<br />
|Lateral subqueries can reference columns of tables defined outside the subquery at the same level, i.e. ''laterally''.<br />
For example, a LATERAL subquery in a FROM clause could reference tables defined in the same FROM clause.<br />
Currently only the columns of tables defined ''above'' subqueries are recognized.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00292.php <nowiki>LATERAL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00991.php <nowiki>Re: LATERAL</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add support for functional dependencies<br />
|This would allow omitting GROUP BY columns when grouping by the primary key.<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent temporary tables created with ON COMMIT DELETE ROWS from repeatedly truncating the table on every commit if the table is already empty<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg00842.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-03/msg00392.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-04/msg00046.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow DELETE and UPDATE to be used with LIMIT and ORDER BY<br />
* http://archives.postgresql.org/pgadmin-hackers/2010-04/msg00078.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg01997.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00021.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow finer control over the caching of prepared query plans<br />
|Currently anonymous (un-named) queries prepared via the wire protocol are replanned every time bind parameters are supplied --- allow SQL PREPARE to do the same. Also, allow control over replanning prepared queries either manually or automatically when statistics for execute parameters differ dramatically from those used during planning.<br />
* http://archives.postgresql.org/message-id/201002151911.o1FJBYh22763@momjian.us<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00597.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow PREPARE of cursors}}<br />
<br />
{{TodoItem<br />
|Have DISCARD PLANS discard plans cached by functions<br />
|DISCARD all should do the same.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00431.php<br />
}}<br />
<br />
=== CREATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow CREATE TABLE AS to determine column lengths for complex expressions like SELECT col1 || col2}}<br />
<br />
{{TodoItem<br />
|Have WITH CONSTRAINTS also create constraint indexes<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00149.php <nowiki>Re: CREATE TABLE LIKE INCLUDING INDEXES support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move NOT NULL constraint information to pg_constraint<br />
|Currently NOT NULL constraints are stored in pg_attribute without any designation of their origins, e.g. primary keys. One manifest problem is that dropping a PRIMARY KEY constraint does not remove the NOT NULL constraint designation. Another issue is that we should probably force NOT NULL to be propagated from parent tables to children, just as CHECK constraints are. (But then does dropping PRIMARY KEY affect children?)<br />
* http://archives.postgresql.org/message-id/19768.1238680878@sss.pgh.pa.us<br />
* http://archives.postgresql.org/message-id/200909181005.n8IA5Ris061239@wwwmaster.postgresql.org<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent concurrent CREATE TABLE from sometimes returning a cryptic error message<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00169.php <nowiki>BUG #3692: Conflicting create table statements throw unexpected error</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow CREATE TABLE to optionally create a table if it does not already exist, without throwing an error<br />
|The fact that tables contain data makes this more complex than other CREATE OR REPLACE operations.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01300.php <nowiki>Add column if not exists (CINE)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add CREATE SCHEMA ... LIKE that copies a schema}}<br />
<br />
{{TodoItem<br />
|Fix CREATE OR REPLACE FUNCTION to not leave objects depending on the function in inconsistent state<br />
* [http://archives.postgresql.org/pgsql-general/2008-08/msg00985.php indexes on functions and create or replace function]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow temporary tables to exist as empty by default in all sessions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00006.php <nowiki>what is difference between LOCAL and GLOBAL TEMP TABLES in PostgreSQL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg01329.php <nowiki>idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-05/msg00016.php <nowiki>Re: idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01098.php <nowiki>global temporary tables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow the creation of "distinct" types<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01647.php <nowiki>Distinct types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider analyzing temporary tables when they are first used in a query<br />
|Autovacuum cannot analyze or vacuum temporary tables.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00416.php <nowiki>autovacuum and temp tables support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow an unlogged table to be changed to logged<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00315.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== UPDATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow UPDATE tab SET ROW (col, ...) = (SELECT...)</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg01308.php <nowiki>Re: [PATCHES] extension for sql update</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00865.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00315.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00237.php <nowiki>Re: UPDATE using sub selects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Research self-referential UPDATEs that see inconsistent row versions in read-committed mode<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00507.php <nowiki>Concurrently updating an updatable view</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00016.php <nowiki>Re: Do we need a TODO? (was Re: Concurrently updating anupdatable view)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve performance of EvalPlanQual mechanism that rechecks already-updated rows<br />
|This is related to the previous item, which questions whether it even has the right semantics<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-09/msg00045.php <nowiki>BUG #4401: concurrent updates to a table blocks one update indefinitely</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-07/msg00302.php <nowiki>BUG #4945: Parallel update(s) gone wild</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ALTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have ALTER TABLE RENAME of a SERIAL column rename the sequence<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have ALTER SEQUENCE RENAME rename the sequence name stored in the sequence table<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-09/msg00092.php <nowiki>BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00007.php <nowiki>Re: BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Allow ALTER TABLE ... ALTER CONSTRAINT ... RENAME or ALTER TABLE RENAME CONSTRAINT<br />
* [http://archives.postgresql.org/pgsql-patches/2006-02/msg00168.php <nowiki>ALTER CONSTRAINT RENAME patch reverted</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ALTER DOMAIN to modify the underlying data type}}<br />
<br />
{{TodoItemDone<br />
|Allow ALTER TABLE to change constraint deferrability}}<br />
<br />
{{TodoItemDone<br />
|Add missing object types for ALTER ... SET SCHEMA}}<br />
<br />
{{TodoItem<br />
|Allow ALTER TABLESPACE to move the tablespace to different directories}}<br />
<br />
{{TodoItem<br />
|Allow moving system tables to other tablespaces, where possible<br />
|Currently non-global system tables must be in the default database tablespace. Global system tables can never be moved.}}<br />
<br />
{{TodoItem<br />
|Have ALTER INDEX update the name of a constraint using that index}}<br />
<br />
{{TodoItem<br />
|Allow column display reordering by recording a display, storage, and permanent id for every column?<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00782.php <nowiki>Re: column ordering, was Re: [PATCHES] Enums patch v2</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01029.php <nowiki>Column reordering in pg_dump</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow an existing index to be marked as a table's primary key<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00500.php <nowiki>Setting a pre-existing index as a primary key</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00642.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00265.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow ALTER TYPE on composite types to perform operations similar to ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00245.php <nowiki>ALTER composite type does not work, but ALTER TABLE which ROWTYPE is used as a type - works fine</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Don't require table rewrite on ALTER TABLE ... ALTER COLUMN TYPE, when the old and new data types are binary compatible<br />
* http://archives.postgresql.org/message-id/200903040137.n241bAUV035002@wwwmaster.postgresql.org<br />
* [http://archives.postgresql.org/pgsql-patches/2006-10/msg00154.php <nowiki>Eliminating phase 3 requirement for varlen increases via ALTER COLUMN</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02360.php<br />
}}<br />
<br />
{{TodoItem|Allow deactivating (and reactivating) indexes via ALTER TABLE|{{messageLink|<87hbegz5ir.fsf@cbbrowne.afilias-int.info>|In discussion on FK activation/deactivation}} }}<br />
<br />
{{TodoItemDone<br />
|Reduce locking required for ALTER commands<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00533.php <nowiki>ALTER TABLE SET STATISTICS requires AccessExclusiveLock</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg01083.php <nowiki>Re: ALTER TABLE SET STATISTICS requires AccessExclusiveLock</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2008-10/msg01248.php<br />
* http://archives.postgresql.org/pgsql-hackers/2008-10/msg00242.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== CLUSTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Automatically maintain clustering on a table<br />
|This might require some background daemon to maintain clustering during periods of low usage. It might also require tables to be only partially filled for easier reorganization. Another idea would be to create a merged heap/index data file so an index lookup would automatically access the heap data too. A third idea would be to store heap rows in hashed groups, perhaps using a user-supplied hash function.<br />
* [http://archives.postgresql.org/pgsql-performance/2004-08/msg00350.php <nowiki>Equivalent praxis to CLUSTERED INDEX?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00155.php <nowiki>Re: Grouped Index Tuples</nowiki>]<br />
* http://community.enterprisedb.com/git/<br />
* [http://archives.postgresql.org/pgsql-performance/2009-10/msg00346.php <nowiki>Re: maintain_cluster_order_v5.patch</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Improve CLUSTER performance by sorting to reduce random I/O<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01371.php <nowiki>Our CLUSTER implementation is pessimal</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Make CLUSTER VERBOSE more verbose.<br />
|It is also used by new VACUUM FULL VERBOSE.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== COPY ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report error lines and continue<br />
|This requires the use of a savepoint before each COPY line is processed, with ROLLBACK on COPY failure. <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00572.php <nowiki>Re: VLDB Features</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY on a newly-created table to skip WAL logging<br />
|On crash recovery, the table involved in the COPY would be removed or have its heap and index files truncated. One issue is that no other backend should be able to add to the table at the same time, which is something that is currently allowed. This currently is done if the table is created inside the same transaction block as the COPY because no other backends can see the table.}}<br />
<br />
{{TodoItem<br />
|Allow COPY FROM to create index entries in bulk<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00811.php <nowiki>Batch update of indexes on data loading</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY in CSV mode to control whether a quoted zero-length string is treated as NULL<br />
|Currently this is always treated as a zero-length string, which generates an error when loading into an integer column <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00905.php <nowiki>Re: [PATCHES] allow CSV quote in NULL</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve COPY performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00954.php <nowiki>Re: 8.3 / 8.2.6 restore comparison</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01882.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report errors sooner<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01169.php <nowiki>Timely reporting of COPY errors</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to handle other number formats<br />
|E.g. the German notation. Best would be something like WITH DECIMAL ','.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow a stalled COPY to exit if the backend is terminated<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00067.php <nowiki>Re: possible bug not in open items</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GRANT/REVOKE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SERIAL sequences to inherit permissions from the base table?}}<br />
<br />
{{TodoItem<br />
|Allow dropping of a role that has connection rights<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00736.php <nowiki>DROP ROLE dependency tracking ...</nowiki>]<br />
}}<br />
{{TodoEndSubsection}}<br />
<br />
=== DECLARE CURSOR ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent DROP TABLE from dropping a table referenced by its own open cursor?}}<br />
<br />
{{TodoItem<br />
|Provide some guarantees about the behavior of cursors that invoke volatile functions<br />
* [http://archives.postgresql.org/message-id/20997.1244563664@sss.pgh.pa.us Re: Cursor with hold emits the same row more than once across commits in 8.3.7]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== INSERT ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow INSERT/UPDATE of the system-generated oid value for a row}}<br />
<br />
{{TodoItem<br />
|In rules, allow VALUES() to contain a mixture of 'old' and 'new' references}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SHOW/SET ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM ANALYZE, and CLUSTER}}<br />
<br />
{{TodoItem<br />
|Rationalize the discrepancy between settings that use values in bytes and SHOW that returns the object count<br />
* [http://archives.postgresql.org/pgsql-docs/2008-07/msg00007.php <nowiki>Re: [ADMIN] shared_buffers and shmmax</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ANALYZE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE issue NOTICE messages when the estimated and actual row counts differ by a specified percentage}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE report rows as floating-point numbers<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01363.php <nowiki>explain analyze rows=%.0f</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00108.php <nowiki>Re: explain analyze rows=%.0f</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve how ANALYZE computes in-doubt tuples<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00771.php <nowiki>VACUUM/ANALYZE counting of in-doubt tuples</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Window Functions ===<br />
See {{messageLink|357.1230492361@sss.pgh.pa.us|TODO items for window functions}}.<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Support creation of user-defined window functions<br />
|We have the ability to create new window functions written in C. Is it<br />
worth the effort to create an API that would let them be written in PL/pgsql, etc?}}<br />
<br />
{{TodoItem<br />
|Implement full support for window framing clauses<br />
|In addition to done clauses described in the [http://developer.postgresql.org/pgdocs/postgres/sql-expressions.html#SYNTAX-WINDOW-FUNCTIONS latest doc], these clauses are not implemented yet.<br />
* RANGE BETWEEN ... PRECEDING/FOLLOWING<br />
* EXCLUDE<br />
}}<br />
<br />
{{TodoItem<br />
|Investigate tuplestore performance issues<br />
|The tuplestore_in_memory() thing is just a band-aid, we ought to try to solve it properly. tuplestore_advance seems like a weak spot as well.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00152.php <nowiki>tuplestore potential performance problem</nowiki>]<br />
}}<br />
<br />
{{TodoItem|Do we really need so much duplicated code between Agg and WindowAgg?}}<br />
<br />
{{TodoItem<br />
|Teach planner to evaluate multiple windows in the optimal order<br />
|Currently windows are always evaluated in the query-specified order.<br />
* http://archives.postgresql.org/message-id/3CDAD71E9D70417290FCF66F0178D1E1@amd64<br />
}}<br />
<br />
{{TodoItem<br />
|Implement DISTINCT clause in window aggregates<br />
|Some proprietary RDBMSs have implemented it already, so it helps with porting from those.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Integrity Constraints ==<br />
=== Keys ===<br />
<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve deferrable unique constraints for cases with many conflicts<br />
|The current implementation fires a trigger for each potentially conflicting row. This might not scale well for an update that changes many key values at once.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Referential Integrity ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add MATCH PARTIAL referential integrity}}<br />
<br />
{{TodoItem<br />
|Change foreign key constraint for array -&gt; element to mean element in array?<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01814.php <nowiki>foreign keys for array/period contains relationships</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when cascading referential triggers make changes on cascaded tables, seeing the tables in an intermediate state<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00174.php <nowiki>Re: [PATCHES] Work-in-progress referential action trigger timing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Optimize referential integrity checks<br />
* [http://archives.postgresql.org/pgsql-performance/2005-10/msg00458.php <nowiki>Re: Effects of cascading references in foreign keys</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00744.php <nowiki>Can't ri_KeysEqual() consider two nulls as equal?</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Server-Side Languages ==<br />
<br />
{{TodoItem<br />
|Add support for polymorphic arguments and return types to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add support for OUT and INOUT parameters to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add more fine-grained specification of functions taking arbitrary data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement stored procedures<br />
|This might involve the control of transaction state and the return of multiple result sets<br />
* [http://archives.postgresql.org/pgsql-general/2008-10/msg00454.php <nowiki>PL/pgSQL stored procedure returning multiple result sets (SELECTs)?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01375.php <nowiki>Proposal: real procedures again (8.4)</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00542.php<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-04/msg01149.php <nowiki>Gathering specs and discussion on feature (post 9.1)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow holdable cursors in SPI}}<br />
<br />
{{TodoItemEasy<br />
|Add SPI_gettypmod() to return a field's typemod from a TupleDesc<br />
* http://archives.postgresql.org/pgsql-hackers/2005-11/msg00250.php<br />
}}<br />
<br />
=== SQL-Language Functions ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SQL-language functions to reference parameters by parameter name<br />
|Currently SQL-language functions can only refer to dollar parameters, e.g. $1<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01479.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01519.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00221.php<br />
}}<br />
<br />
{{TodoItem<br />
|Rethink query plan caching and timing of parse analysis within SQL-language functions<br />
|They should work more like plpgsql functions do ...<br />
* [http://archives.postgresql.org/pgsql-bugs/2011-05/msg00078.php <nowiki>Re: BUG #6019: invalid cached plan on inherited table</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/pgSQL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow handling of %TYPE arrays, e.g. tab.col%TYPE[]}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow listing of record column names, and access to record columns via variables, e.g. columns := r.(*), tval2 := r.(colname)</nowiki><br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00458.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00302.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00031.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow row and record variables to be set to NULL constants, and allow NULL tests on such variables<br />
|Because a row is not scalar, do not allow assignment from NULL-valued scalars.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00070.php <nowiki>NULL and plpgsql rows</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider keeping separate cached copies when search_path changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01009.php <nowiki>pl/pgsql Plan Invalidation and search_path</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULL row values vs. NULL rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg01758.php <nowiki>Null row vs. row of nulls in plpgsql</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg01973.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Perl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow regex operations in plperl using UTF8 characters in non-UTF8 encoded databases}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Python ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Add table function support}}<br />
<br />
{{TodoItemDone<br />
|Add tracebacks<br />
* [http://archives.postgresql.org/pgsql-patches/2006-02/msg00288.php <nowiki>Re: plpython tracebacks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Develop a trusted variant of PL/Python.}}<br />
<br />
{{TodoItem<br />
|Create a new restricted execution class that will allow passing function arguments in as locals. Passing them as globals means functions cannot be called recursively.<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-02/msg01468.php <nowiki>Re: pl/python do not delete function arguments</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve documentation}}<br />
<br />
{{TodoItem<br />
|Add a DB-API compliant interface on top of the SPI interface}}<br />
<br />
{{TodoItem<br />
|Improve behaviour of exception functions and types<br />
* http://archives.postgresql.org/pgsql-docs/2010-11/msg00022.php<br />
* http://archives.postgresql.org/pgsql-docs/2010-11/msg00031.php<br />
}}<br />
{{TodoItem<br />
|For functions returning a setof record with a composite type, cache the I/O functions for the composite type<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02007.php<br />
}}<br />
<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Tcl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add table function support}}<br />
<br />
{{TodoItem<br />
|Check encoding validity of values passed back to Postgres in function returns, trigger tuple changes, and SPI calls.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Clients ==<br />
<br />
{{TodoItem<br />
|Add a function like pg_get_indexdef() that report more detailed index information<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00166.php <nowiki>BUG #3829: Wrong index reporting from pgAdmin III (v1.8.0 rev 6766-6767)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Split out pg_resetxlog output into pre- and post-sections<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg02040.php<br />
}}<br />
<br />
=== pg_ctl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow pg_ctl to work properly with configuration files located outside the PGDATA directory<br />
|pg_ctl can not read the pid file because it isn't located in the config directory but in the PGDATA directory. The solution is to allow pg_ctl to read and understand postgresql.conf to find the data_directory value.<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-10/msg00024.php <nowiki>BUG #5103: &quot;pg_ctl -w (re)start&quot; fails with custom unix_socket_directory</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Modify pg_ctl behavior and exit codes to make it easier to write an LSB conforming init script<br />
|It may be desirable to condition some of the changes on a command-line switch, to avoid breaking existing scripts. A Linux shell (sh) script is referenced which has been tested and seems to provide a high degree of conformance in multiple environments. Study of this script might suggest areas where pg_ctl could be modified to make writing an LSB conforming script easier; however, some aspects of that script would be unnecessary with other suggested changes to pg_ctl, and discussion on the lists did not reach consensus on support for all aspects of this script. Further discussion of particular changes is needed before beginning any work.<br />
* [[Lsb_conforming_init_script|LSB conforming init script]]<br />
These threads should be studied for other ideas on improvements:<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01390.php <nowiki>We should Axe /contrib/start-scripts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01843.php <nowiki>Linux LSB init script</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00008.php <nowiki>Re: Linux LSB init script</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== psql ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have psql \ds show all sequences and their settings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00916.php <nowiki>Re: TODO item: Have psql show current values for a sequence</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00401.php <nowiki>Quick patch: Display sequence owner</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have \d on a sequence indicate if the sequences is owned by a table}}<br />
<br />
{{TodoItem<br />
|Move psql backslash database information into the backend, use mnemonic commands?<br />
|This would allow non-psql clients to pull the same information out of the database as psql. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-01/msg00191.php <nowiki>Re: psql \d option list overloaded</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make psql's \d commands more consistent in its handling of schemas<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00014.php <nowiki>Re: psql and schemas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consistently display privilege information for all objects in psql}}<br />
<br />
{{TodoItem<br />
|Add &quot;auto&quot; expanded mode that outputs in expanded format if &quot;wrapped&quot; mode can't wrap the output to the screen width<br />
|Consider using auto-expanded mode for backslash commands like \df+.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00417.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg01638.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent tab completion of SET TRANSACTION from querying the database and therefore preventing the transaction isolation level from being set.<br />
|Currently SET &lt;tab&gt; causes a database lookup to check all supported session variables. This query causes problems because setting the transaction isolation level must be the first statement of a transaction.}}<br />
<br />
{{TodoItem<br />
|Add a \set variable to control whether \s displays line numbers<br />
|Another option is to add \# which lists line numbers, and allows command execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00255.php <nowiki>Re: psql possible TODO</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have \d show child tables that inherit from the specified parent}}<br />
<br />
{{TodoItem<br />
|Include the symbolic SQLSTATE name in verbose error reports<br />
* [http://archives.postgresql.org/pgsql-general/2007-09/msg00438.php <nowiki>Re: Checking is TSearch2 query is valid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add prompt escape to display the client and server versions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00310.php <nowiki>WIP patch for TODO Item: Add prompt escape to display the client and server versions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to wrap column values at whitespace boundaries, rather than chopping them at a fixed width.<br />
|Currently, &quot;wrapped&quot; format chops values into fixed widths. Perhaps the word wrapping could use the same algorithm documented in the W3C specification. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00404.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
* http://www.w3.org/TR/CSS21/tables.html#auto-table-layout}}<br />
<br />
{{TodoItem<br />
|Support the ReST table output format<br />
|Details about the ReST format: http://docutils.sourceforge.net/rst.html#reference-documentation<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01007.php <nowiki>Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00518.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00609.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to print advice for people familiar with other databases<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01845.php <nowiki>MySQL-ism help patch for psql</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Consider showing TOAST and index sizes in \dt+<br />
* [http://archives.postgresql.org/pgsql-general/2010-01/msg00912.php <nowiki>\dt+ sizes don't include TOAST data</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2011-04/msg00485.php <nowiki>Re: psql \dt and table size</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow \dd to show constraint comments<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00436.php <nowiki>Re: More robust pg_hba.conf parsing/error logging</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-09/msg00199.php <nowiki>comment on constraint</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ability to edit views with \ev<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00023.php <nowiki>Adding \ev view editor?</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add \dL to show languages<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-07/msg00915.php <nowiki>Re: [PATCH] Psql List Languages</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Distinguish between unique indexes and unique constraints in \d+<br />
* http://archives.postgresql.org/message-id/8780.1271187360@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Fix FETCH_COUNT to handle SELECT ... INTO and WITH queries<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01565.php<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00192.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent psql from sending remaining single-line multi-statement queries after reconnecting<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00159.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01283.php<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Add \i option to bring in the specified file as a quoted literal. This would be useful for creating functions and other areas. Details still need to be worked out.<br />
* http://archives.postgresql.org/pgsql-bugs/2011-02/msg00016.php<br />
* http://archives.postgresql.org/pgsql-bugs/2011-02/msg00020.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having psql -c read .psqlrc, for consistency<br />
|psql -f already reads .psqlrc<br />
}}<br />
<br />
{{TodoItem<br />
|Allow processing of multiple -f (file) options<br />
}}<br />
<br />
{{TodoItem<br />
|Improve line drawing characters<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00386.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== pg_dump / pg_restore ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|<nowiki>Add full object name to the tag field. eg. for operators we need '=(integer, integer)', instead of just '='.</nowiki>}}<br />
<br />
{{TodoItem<br />
|Add pg_dumpall custom format dumps?<br />
* [http://archives.postgresql.org/pgsql-general/2010-05/msg00509.php pg_dumpall custom format]<br />
|}}<br />
<br />
{{TodoItem<br />
|Avoid using platform-dependent locale names in pg_dumpall output<br />
|Using native locale names puts roadblocks in the way of porting a dump to another platform. One possible solution is to get<br />
CREATE DATABASE to accept some agreed-on set of locale names and fix them up to meet the platform's requirements.<br />
* http://archives.postgresql.org/message-id/21396.1241716688@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Allow selection of individual object(s) of all types, not just tables}}<br />
<br />
{{TodoItem<br />
|In a selective dump, allow dumping of an object and all its dependencies}}<br />
<br />
{{TodoItem<br />
|Add options like pg_restore -l and -L to pg_dump}}<br />
<br />
{{TodoItem<br />
|Add support for multiple pg_restore -t options, like pg_dump<br />
|pg_restore's -t switch is less useful than pg_dump's in quite a few ways: no multiple switches, no pattern matching, no ability to pick up indexes and other dependent items for a selected table. It should be made to handle this switch just like pg_dump does.}}<br />
<br />
{{TodoItem<br />
|Stop dumping CASCADE on DROP TYPE commands in clean mode}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump --clean to drop roles that own objects or have privileges<br />
|tgl says: if this is about pg_dumpall, it's done as of 8.4. If it's really about pg_dump, what does it mean? pg_dump has no business dropping roles.}}<br />
<br />
{{TodoItem<br />
|Remove unnecessary function pointer abstractions in pg_dump source code}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump to utilize multiple CPUs and I/O channels by dumping multiple objects simultaneously<br />
|The difficulty with this is getting multiple dump processes to produce a single dump output file. It also would require several sessions to share the same snapshot. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php <nowiki>pg_dump additional options for performance</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00135.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00040.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02454.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow pg_restore to load different parts of the COPY data for a single table simultaneously}}<br />
<br />
{{TodoItem<br />
|Remove support for dumping from pre-7.3 servers<br />
|In 7.3 and later, we can get accurate dependency information from the server. pg_dump still contains a lot of crufty code<br />
to try to deal with the lack of dependency info in older servers, but the usefulness of maintaining that code grows small.}}<br />
<br />
{{TodoItem<br />
|Allow pre/data/post files when schema and data are dumped separately, for performance reasons<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php <nowiki>pg_dump additional options for performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00185.php <nowiki>Re: pg_dump additional options for performance</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00821.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00135.php<br />
}}<br />
<br />
{{TodoItem<br />
|Refactor handling of database attributes between pg_dump and pg_dumpall<br />
|Currently only pg_dumpall emits database attributes, such as ALTER DATABASE SET commands and database-level GRANTs.<br />
Many people wish that pg_dump would do that. One proposal is to let pg_dump issue such commands if the -C switch was used,<br />
but it's unclear whether that will satisfy the demand.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01031.php <nowiki>ALTER DATABASE vs pg_dump</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-05/msg00010.php summary of the issues]<br />
}}<br />
<br />
{{TodoItem<br />
|Change pg_dump so that a comment on the dumped database is applied to the loaded database, even if the database has a different name.<br />
|This will require new backend syntax, perhaps COMMENT ON CURRENT DATABASE. This is related to the previous item.}}<br />
<br />
{{TodoItem<br />
|Allow parallel restore of tar dumps<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-02/msg01154.php <nowiki>Re: parallel restore</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow pg_dumpall to output restorable ALTER USER/DATABASE SET settings<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00916.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00394.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02359.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ecpg ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Docs<br />
|Document differences between ecpg and the SQL standard and information about the Informix-compatibility module.}}<br />
<br />
{{TodoItem<br />
|Solve cardinality &gt; 1 for input descriptors / variables?}}<br />
<br />
{{TodoItem<br />
|Add a semantic check level, e.g. check if a table really exists}}<br />
<br />
{{TodoItem<br />
|fix handling of DB attributes that are arrays}}<br />
<br />
{{TodoItem<br />
|Fix nested C comments}}<br />
<br />
{{TodoItemEasy<br />
|sqlwarn[6] should be 'W' if the PRECISION or SCALE value specified}}<br />
<br />
{{TodoItem<br />
|Make SET CONNECTION thread-aware, non-standard?}}<br />
<br />
{{TodoItem<br />
|Allow multidimensional arrays}}<br />
<br />
{{TodoItem<br />
|Implement COPY FROM STDIN}} <br />
<br />
{{TodoItem<br />
|Provide a way to specify size of a bytea parameter<br />
* [http://archives.postgresql.org/message-id/200906192131.n5JLVoMo044178@wwwmaster.postgresql.org <nowiki>BUG #4866: ECPG and BYTEA</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Fix small memory leaks in ecpg<br />
|Memory leaks in a short running application like ecpg are not really a problem, but make debugging more complicated}} <br />
<br />
{{TodoItem<br />
|Allow reuse of cursor name variables<br />
* [http://archives.postgresql.org/message-id/20100329113435.GA3430@feivel.credativ.lan <nowiki>Problems with variable cursorname in ecpg</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== libpq ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent PQfnumber() from lowercasing unquoted column names<br />
|PQfnumber() should never have been doing lowercasing, but historically it has so we need a way to prevent it}}<br />
<br />
{{TodoItem<br />
|Allow statement results to be automatically batched to the client<br />
|Currently all statement results are transferred to the libpq client before libpq makes the results available to the application. This feature would allow the application to make use of the first result rows while the rest are transferred, or held on the server waiting for them to be requested by libpq. One complexity is that a statement like SELECT 1/col could error out mid-way through the result set.}}<br />
<br />
{{TodoItem<br />
|Consider disallowing multiple queries in PQexec() as an additional barrier to SQL injection attacks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00184.php <nowiki>Re: InitPostgres and flatfiles question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add PQexecf() that allows complex parameter substitution<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01803.php <nowiki>Last minute mini-proposal (I know, know) for PQexecf()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add SQLSTATE and severity to errors generated within libpq itself<br />
* [http://archives.postgresql.org/pgsql-interfaces/2007-11/msg00015.php <nowiki>v8.1: Error severity on libpq PGconn*</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01425.php<br />
}}<br />
<br />
{{TodoItemDone<br />
|Add code to detect client encoding and locale from the operating system environment<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01040.php <nowiki>Determining client_encoding from client locale</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for interface/ipaddress binding to libpq<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01811.php <nowiki>SR/libpq - outbound interface/ipaddress binding</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Triggers ==<br />
<br />
{{TodoItem<br />
|Improve storage of deferred trigger queue<br />
|Right now all deferred trigger information is stored in backend memory. This could exhaust memory for very large trigger queues. This item involves dumping large queues into files, or doing some kind of join to process all the triggers, some bulk operation, or a bitmap. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00876.php <nowiki>Re: BUG #4204: COPY to table with FK has memory leak</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00464.php <nowiki>Scaling up deferred unique checks and the after trigger queue</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow triggers to be disabled in only the current session.<br />
|This is currently possible by starting a multi-statement transaction, modifying the system tables, performing the desired SQL, restoring the system tables, and committing the transaction. ALTER TABLE ... TRIGGER requires a table lock so it is not ideal for this usage.}}<br />
<br />
{{TodoItem<br />
|With disabled triggers, allow pg_dump to use ALTER TABLE ADD FOREIGN KEY<br />
|If the dump is known to be valid, allow foreign keys to be added without revalidating the data.}}<br />
<br />
{{TodoItem<br />
|Allow statement-level triggers to access modified rows}}<br />
<br />
{{TodoItem<br />
|When statement-level triggers are defined on a parent table, have them fire only on the parent table, and fire child table triggers only where appropriate<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01883.php <nowiki>Statement-level triggers and inheritance</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow AFTER triggers on system tables<br />
|System tables are modified in many places in the backend without going through the executor and therefore not causing triggers to fire. To complete this item, the functions that modify system tables will have to fire triggers.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01665.php<br />
* http://wiki.postgresql.org/wiki/DDL_Triggers<br />
}}<br />
<br />
{{TodoItem<br />
|Tighten trigger permission checks<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00564.php <nowiki>Security leak with trigger functions?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow BEFORE INSERT triggers on views<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg01466.php <nowiki>Re: Why can't I put a BEFORE EACH ROW trigger on a view?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add database and transaction-level triggers<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00451.php <nowiki>Proposal for db level triggers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00620.php <nowiki>triggers on prepare, commit, rollback... ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce locking requirements for creating a trigger<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00635.php <nowiki>Re: Change lock requirements for adding a trigger</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid requirement for "AFTER" trigger functions to return a value<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02384.php<br />
}}<br />
<br />
== Inheritance ==<br />
<br />
{{TodoItem<br />
|Allow inherited tables to inherit indexes, UNIQUE constraints, and primary/foreign keys<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-05/msg00285.php <nowiki>Partitioning/inherited tables vs FKs</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00039.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00305.php<br />
}}<br />
<br />
{{TodoItem<br />
|Honor UNIQUE INDEX on base column in INSERTs/UPDATEs on inherited table, e.g. INSERT INTO inherit_table (unique_index_col) VALUES (dup) should fail<br />
|The main difficulty with this item is the problem of creating an index that can span multiple tables.}}<br />
<br />
{{TodoItem<br />
|Determine whether ALTER TABLE / SET SCHEMA should work on inheritance hierarchies (and thus support ONLY). If yes, implement it.}}<br />
<br />
{{TodoItem<br />
|ALTER TABLE variants sometimes support recursion and sometimes not, but this is poorly/not documented, and the ONLY marker would then be silently ignored. Clarify the documentation, and reject ONLY if it is not supported.}}<br />
<br />
== Indexes ==<br />
<br />
{{TodoItem<br />
|Prevent index uniqueness checks when UPDATE does not modify the column<br />
|Uniqueness (index) checks are done when updating a column even if the column is not modified by the UPDATE.<br />
However, HOT already short-circuits this in common cases, so more work might not be helpful.}}<br />
<br />
{{TodoItem<br />
|Allow the creation of on-disk bitmap indexes which can be quickly combined with other bitmap indexes<br />
|Such indexes could be more compact if there are only a few distinct values. Such indexes can also be compressed. Keeping such indexes updated can be costly.<br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00512.php <nowiki>Re: Bitmap index AM</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01107.php <nowiki>Bitmap index thoughts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00265.php <nowiki>Stream bitmaps</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01214.php <nowiki>Re: Bitmapscan changes - Requesting further feedback</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00013.php <nowiki>Updated bitmap index patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00741.php <nowiki>Reviewing new index types (was Re: [PATCHES] Updated bitmap indexpatch)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01023.php <nowiki>Bitmap Indexes: request for feedback</nowiki>]<br />
* http://archives.postgresql.org/message-id/800923.27831.qm@web29010.mail.ird.yahoo.com <br />
}}<br />
<br />
{{TodoItem<br />
|Allow accurate statistics to be collected on indexes with more than one column or expression indexes, perhaps using per-index statistics<br />
* [http://archives.postgresql.org/pgsql-performance/2006-10/msg00222.php <nowiki>Re: Simple join optimized badly?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01131.php <nowiki>Stats for multi-column indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00741.php <nowiki>Cross-column statistics revisited</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01431.php <nowiki>Multi-Dimensional Histograms</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00913.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg02179.php <br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00459.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02054.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01731.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having a larger statistics target for indexed columns and expression indexes. <br />
}}<br />
<br />
{{TodoItem<br />
|Consider smaller indexes that record a range of values per heap page, rather than having one index entry for every heap row<br />
|This is useful if the heap is clustered by the indexed values. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00341.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01264.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00465.php <nowiki>Grouped Index Tuples / Clustered Indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-03/msg00163.php <nowiki>Bitmapscan changes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00014.php <nowiki>Re: GIT patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00487.php <nowiki>Re: Index Tuple Compression Approach?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01589.php <nowiki>Re: Index AM change proposals, redux</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add REINDEX CONCURRENTLY, like CREATE INDEX CONCURRENTLY<br />
|This is difficult because you must upgrade to an exclusive table lock to replace the existing index file. CREATE INDEX CONCURRENTLY does not have this complication. This would allow index compaction without downtime. <br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00289.php <nowiki>Re: When/if to Reindex</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multiple indexes to be created concurrently, ideally via a single heap scan<br />
|pg_restore allows parallel index builds, but it is done via subprocesses, and there is no SQL interface for this.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider sorting entries before inserting into btree index<br />
* [http://archives.postgresql.org/pgsql-general/2008-01/msg01010.php <nowiki>Re: ATTN: Clodaldo was Performance problem. Could it be related to 8.3-beta4?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow index scans to return matching index keys, not just the matching heap locations<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01657.php <nowiki>Re: Is this TODO item done?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01477.php <nowiki>Index-only quals</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow creation of an index that can do comparisons to test if a value is between two column values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00757.php <nowiki>Proposal: temporal extension &quot;period&quot; data type</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using "effective_io_concurrency" for index scans<br />
* Currently only bitmap scans use this, which might be fine because most multi-row index scans use bitmap scans.<br />
}}<br />
<br />
=== GIST ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add more GIST index support for geometric data types}}<br />
<br />
{{TodoItem<br />
|Allow GIST indexes to create certain complex index types, like digital trees (see Aoki)}}<br />
<br />
{{TodoItem<br />
|Fix performance issues in contrib/seg and contrib/cube GiST support<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904161633160.4053@aragorn.flymine.org GiST index performance]<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904221704470.22330@aragorn.flymine.org draft patch]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-05/msg00069.php <nowiki>Re: GiST index performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-06/msg00068.php <nowiki>GiST index performance</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GIN ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Support empty indexed values (such as zero-element arrays) properly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00237.php contrib/intarray vs empty arrays]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-05/msg00118.php BUG #4806: Bug with GiST index and empty integer array]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Behave correctly for cases where some elements of an indexed value are NULL<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-03/msg01003.php <nowiki>GIN versus zero-key queries</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Support queries that require a full scan<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg00402.php Issue report]<br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01132.php Older issue report]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-10/msg00521.php Still another complaint]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01581.php Previous partial fix]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Improve GIN's handling of NULL array values<br />
* http://archives.postgresql.org/pgsql-bugs/2010-12/msg00032.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Hash ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add UNIQUE capability to hash indexes}}<br />
<br />
{{TodoItem<br />
|Add hash WAL logging for crash recovery}}<br />
<br />
{{TodoItem<br />
|Allow multi-column hash indexes}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Sorting ==<br />
<br />
{{TodoItem<br />
|Consider whether duplicate keys should be sorted by block/offset<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00558.php <nowiki>Remove hacks for old bad qsort() implementations?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider being smarter about memory and external files used during sorts<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01101.php <nowiki>Sorting Improvements for 8.4</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00045.php <nowiki>Re: Sorting Improvements for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider detoasting keys before sorting}}<br />
<br />
{{TodoItem<br />
|Allow sorts to use more available memory<br />
* http://archives.postgresql.org/pgsql-hackers/2007-11/msg01026.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg01123.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg01957.php<br />
}}<br />
<br />
== Fsync ==<br />
<br />
{{TodoItem<br />
|Determine optimal fdatasync/fsync, O_SYNC/O_DSYNC options and whether fsync does anything<br />
|Ideally this requires a separate test program like /contrib/pg_test_fsync that can be run at initdb time or optionally later.}}<br />
<br />
{{TodoItem<br />
|Consider sorting writes during checkpoint<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php <nowiki>Sorted writes in checkpoint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00050.php <nowiki>Re: Sorting writes during checkpoint</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg02012.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00278.php<br />
}}<br />
<br />
== Cache Usage ==<br />
<br />
{{TodoItem<br />
|Speed up COUNT(*)<br />
|We could use a fixed row count and a +/- count to follow MVCC visibility rules, or a single cached value could be used and invalidated if anyone modifies the table. Another idea is to get a count directly from a unique index, but for this to be faster than a sequential scan it must avoid access to the heap to obtain tuple visibility information.}}<br />
<br />
{{TodoItem<br />
|Provide a way to calculate an &quot;estimated COUNT(*)&quot;<br />
|Perhaps by using the optimizer's cardinality estimates or random sampling.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-11/msg00943.php <nowiki>Re: Improving count(*)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow data to be pulled directly from indexes<br />
|Currently indexes do not have enough tuple visibility information to allow data to be pulled from the index without also accessing the heap. The idea is to use the visibility map used for vacuum to avoid heap lookups on pages where all tuples are visible.<br />
* [http://wiki.postgresql.org/wiki/Index-only_scans Index-Only Scans wiki]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider automatic caching of statements at various levels:<br />
* Parsed query tree<br />
* Query execute plan<br />
* Query results <br />
<br />
:<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00823.php <nowiki>Cached Query Plans (was: global prepared statements)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing internal areas (NUM_CLOG_BUFFERS) when shared buffers is increased<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-10/msg01419.php <nowiki>Re: slru.c race condition (was Re: TRAP: FailedAssertion(&quot;!((itemid)-&gt;lp_flags &amp; 0x01)&quot;,)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00030.php <nowiki>clog_buffers to 64 in 8.3?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00024.php <nowiki>CLOG Patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the amount of memory used by PrivateRefCount<br />
|<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00797.php <nowiki>PrivateRefCount (for 8.3)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00752.php <nowiki>Re: PrivateRefCount (for 8.3)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing higher priority queries to have referenced buffer cache pages stay in memory longer<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00562.php <nowiki>Re: How to keep a table in memory?</nowiki>]<br />
}}<br />
<br />
== Vacuum ==<br />
<br />
{{TodoItem<br />
|Auto-fill the free space map by scanning the buffer cache or by checking pages written by the background writer<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-02/msg01125.php <nowiki>Dead Space Map</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00011.php <nowiki>Re: Automatic free space map filling</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow concurrent inserts to use recently created pages rather than creating new ones<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg00853.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having single-page pruning update the visibility map<br />
* <nowiki>https://commitfest.postgresql.org/action/patch_view?id=75</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02344.php <nowiki>Re: visibility maps and heap_prune</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve tracking of total relation tuple counts now that vacuum doesn't always scan the whole heap<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00531.php Partial vacuum versus pg_class.reltuples]<br />
}}<br />
<br />
{{TodoItem<br />
|Bias FSM towards returning free space near the beginning of the heap file, in hopes that empty pages at the end can be truncated by VACUUM<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01124.php <nowiki>FSM search modes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a more compact data representation for dead tuple locations within VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00143.php <nowiki>Re: Have vacuum emit a warning when it runs out of maintenance_work_mem</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more information in order to improve user-side estimates of dead space bloat in relations<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg01039.php <nowiki>Re: Bloated Table</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve locking behaviour of vacuum during trailing page truncation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00319.php<br />
* http://archives.postgresql.org/message-id/4D8DF88E.7080205@Yahoo.com<br />
}}<br />
<br />
=== Auto-vacuum ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Issue log message to suggest VACUUM FULL if a table is nearly empty?}}<br />
<br />
{{TodoItem<br />
|Prevent long-lived temporary tables from causing frozen-xid advancement starvation<br />
|The problem is that autovacuum cannot vacuum them to set frozen xids; only the session that created them can do that. <br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01645.php <nowiki>Re: AutoVacuum Behaviour Question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent autovacuum from running if an old transaction is still running from the last vacuum<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00899.php <nowiki>Re: Autovacuum and OldestXmin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have autoanalyze of parent tables occur when child tables are modified<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00137.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-10/msg00271.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Locking ==<br />
<br />
{{TodoItem<br />
|Fix priority ordering of read and write light-weight locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00893.php <nowiki>lwlocks and starvation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00905.php <nowiki>Re: lwlocks and starvation</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when multiple subtransactions of the same outer transaction hold different types of locks, and one subtransaction aborts<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg01011.php <nowiki>FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00001.php <nowiki>Re: FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00435.php <nowiki>Re: [PATCHES] [pgsql-patches] Phantom Command IDs, updated patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00773.php <nowiki>Re: savepoints and upgrading locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow UPDATEs on only non-referential integrity columns not to conflict with referential integrity locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00073.php <nowiki>Referential Integrity and SHARE locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add idle_in_transaction_timeout GUC so locks are not held for long periods of time}}<br />
<br />
{{TodoItem<br />
|Improve deadlock detection when a page cleaning lock conflicts with a shared buffer that is pinned<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-01/msg00138.php <nowiki>BUG #3883: Autovacuum deadlock with truncate?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-committers/2008-01/msg00365.php <nowiki>Re: pgsql: Add checks to TRUNCATE, CLUSTER, and REINDEX to prevent</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Detect deadlocks involving LockBufferForCleanup()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow finer control over who is cancelled in a deadlock<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01727.php<br />
}}<br />
<br />
<br />
{{TodoItem<br />
|Consider a lock timeout parameter<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00485.php <nowiki>SELECT ... FOR UPDATE [WAIT integer | NOWAIT] for 8.5</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Consider improving serialized transaction behavior to avoid anomalies<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00217.php <nowiki>Serializable Isolation without blocking</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01136.php <nowiki>User-facing aspects of serializable transactions</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00035.php <nowiki>Re: User-facing aspects of serializable transactions</nowiki>]<br />
}}<br />
<br />
== Startup Time Improvements ==<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for backend creation<br />
|This would prevent the overhead associated with process creation. Most operating systems have trivial process creation time compared to database startup overhead, but a few operating systems (Win32, Solaris) might benefit from threading. Also explore the idea of a single session using multiple threads to execute a statement faster.}}<br />
<br />
{{TodoItem<br />
|Allow backends to change their database without restart<br />
|This allows for faster server startup.<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00843.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00336.php<br />
}}<br />
<br />
== Write-Ahead Log ==<br />
<br />
{{TodoItem<br />
|Eliminate need to write full pages to WAL before page modification<br />
|Currently, to protect against partial disk page writes, we write full page images to WAL before they are modified so we can correct any partial page writes during recovery. These pages can also be eliminated from point-in-time archive files. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-06/msg00655.php <nowiki>Re: Index Scans become Seq Scans after VACUUM ANALYSE</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|When full page writes are off, write CRC to WAL and check file system blocks on recovery<br />
|If CRC check fails during recovery, remember the page in case a later CRC for that page properly matches.}}<br />
<br />
{{TodoItem<br />
|Write full pages during file system write and not when the page is modified in the buffer cache<br />
|This allows most full page writes to happen in the background writer. It might cause problems for applying WAL on recovery into a partially-written page, but later the full page will be replaced from WAL.}}<br />
<br />
{{TodoItem<br />
|Reduce WAL traffic so only modified values are written rather than entire rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01589.php <nowiki>Reduction in WAL for UPDATEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow WAL information to recover corrupted pg_controldata<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00025.php <nowiki>Re: [HACKERS] pg_resetxlog -r flag</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a way to reduce rotational delay when repeatedly writing last WAL page<br />
|Currently fsync of WAL requires the disk platter to perform a full rotation to fsync again. One idea is to write the WAL to different offsets that might reduce the rotational delay. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-11/msg00483.php <nowiki>500 tpsQL + WAL log implementation</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow WAL logging to be turned off for a table, but the table might be dropped or truncated during crash recovery<br />
|Allow tables to bypass WAL writes and just fsync() dirty pages on commit. This should be implemented using ALTER TABLE, e.g. <nowiki>ALTER TABLE PERSISTENCE [ DROP | TRUNCATE | DEFAULT ]</nowiki>. Tables using non-default logging should not use referential integrity with default-logging tables. A table without dirty buffers during a crash could perhaps avoid the drop/truncate. <br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01016.php <nowiki>Re: [Bizgres-general] WAL bypass for INSERT, UPDATE and</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Speed WAL recovery by allowing more than one page to be prefetched<br />
|This should be done utilizing the same infrastructure used for prefetching in general to avoid introducing complex error-prone code in WAL replay. <br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00683.php <nowiki>Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00497.php <nowiki>Re: [GENERAL] Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg01279.php <nowiki>Read-ahead and parallelism in redo recovery</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve WAL concurrency by increasing lock granularity<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00556.php <nowiki>Reworking WAL locking</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Be more aggressive about creating WAL files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01325.php <nowiki>Re: PANIC caused by open_sync on Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-07/msg01075.php <nowiki>PreallocXlogFiles</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-04/msg00556.php <nowiki>WAL/PITR additional items</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have resource managers report the duration of their status changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01468.php <nowiki>Recovery of Multi-stage WAL actions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move pgfoundry's xlogdump to /contrib and have it rely more closely on the WAL backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00035.php <nowiki>xlogdump</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Close deleted WAL files held open in *nix by long-lived read-only backends<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01754.php <nowiki>Deleted WAL files held open by backends in Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00060.php <nowiki>Re: Deleted WAL files held open by backends in Linux</nowiki>]<br />
}}<br />
<br />
== Optimizer / Executor ==<br />
<br />
{{TodoItem<br />
|Improve selectivity functions for geometric operators}}<br />
<br />
{{TodoItem<br />
|Consider increasing the default values of from_collapse_limit, join_collapse_limit, and/or geqo_threshold<br />
* [http://archives.postgresql.org/message-id/4136ffa0905210551u22eeb31bn5655dbe7c9a3aed5@mail.gmail.com from_collapse_limit vs. geqo_threshold]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to display optimizer analysis using OPTIMIZER_DEBUG}}<br />
<br />
{{TodoItem<br />
|Log statements where the optimizer row estimates were dramatically different from the number of rows actually found?}}<br />
<br />
{{TodoItem<br />
|Consider compressed annealing to search for query plans<br />
|This might replace GEQO.<br />
* http://archives.postgresql.org/message-id/15658.1241278636%40sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Improve use of expression indexes for ORDER BY <br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01553.php <nowiki>Resjunk sort columns, Heikki's index-only quals patch, and bug #5000</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Modify the planner to better estimate caching effects<br />
* http://archives.postgresql.org/pgsql-performance/2010-11/msg00117.php<br />
}}<br />
<br />
=== Hashing ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Consider using a hash for joining to a large IN (VALUES ...) list<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00450.php <nowiki>Planning large IN lists</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single batch hash joins to preserve outer pathkeys<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00806.php Re: Potential Join Performance Issue]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|"lazy" hash tables - look up only the tuples that are actually requested<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid building the same hash table more than once during the same query<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid hashing for distinct and then re-hashing for hash join<br />
* [http://archives.postgresql.org/message-id/4136ffa0902191346g62081081v8607f0b92c206f0a@mail.gmail.com Re: Fixing Grittner's planner issues]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow hashing to be used on arrays, if the element type is hashable<br />
* http://archives.postgresql.org/message-id/11087.1244905821@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Background Writer ==<br />
<br />
{{TodoItem<br />
|Consider having the background writer update the transaction status hint bits before writing out the page<br />
|Implementing this requires the background writer to have access to system catalogs and the transaction status log.}}<br />
<br />
{{TodoItem<br />
|Consider adding buffers the background writer finds reusable to the free list <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Automatically tune bgwriter_delay based on activity rather then using a fixed interval<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider whether increasing BM_MAX_USAGE_COUNT improves performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg01007.php <nowiki>Bgwriter LRU cleaning: we've been going at this all wrong</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Test to see if calling PreallocXlogFiles() from the background writer will help with WAL segment creation latency<br />
* [http://archives.postgresql.org/pgsql-patches/2007-06/msg00340.php <nowiki>Re: Load Distributed Checkpoints, final patch</nowiki>]<br />
}}<br />
<br />
== Concurrent Use of Resources ==<br />
<br />
{{TodoItem<br />
|Do async I/O for faster random read-ahead of data<br />
|Async I/O allows multiple I/O requests to be sent to the disk with results coming back asynchronously.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00820.php <nowiki>Asynchronous I/O Support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-09/msg00255.php <nowiki>Re: random_page_costs - are defaults of 4.0 realistic for SCSI RAID 1</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00027.php <nowiki>There's random access and then there's random access</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-01/msg00170.php <nowiki>Bitmap index scan preread using posix_fadvise (Was: There's random access and then there's random access)</nowiki>]<br />
The above patch is already applied as of 8.4, but it still remains to figure out how to handle plain indexscans effectively.<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-01/msg00806.php Problems with the patch submitted for posix_fadvise in index scans]<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better I/O utilization<br />
|This would allow a single query to make use of multiple I/O channels simultaneously. One idea is to create a background reader that can pre-fetch sequential and index scan pages needed by other backends. This could be expanded to allow concurrent reads from multiple devices in a partitioned table.<br />
* http://archives.postgresql.org/pgsql-performance/2011-02/msg00123.php<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better CPU utilization<br />
|This would allow several CPUs to be used for a single query, such as for sorting or query execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00945.php <nowiki>Multi CPU Queries - Feedback and/or suggestions wanted!</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|SMP scalability improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00439.php <nowiki>Straightforward changes for increased SMP scalability</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00206.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
== TOAST ==<br />
<br />
{{TodoItem<br />
|Allow user configuration of TOAST thresholds<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00213.php <nowiki>Re: Proposed adjustments in MaxTupleSize and toastthresholds</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00082.php <nowiki>pg_lzcompress strategy parameters</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce unnecessary cases of deTOASTing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00895.php <nowiki>Re: [PATCHES] Eliminate more detoast copies for packed varlenas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce costs of repeat de-TOASTing of values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01096.php <nowiki>WIP patch: reducing overhead for repeat de-TOASTing</nowiki>]<br />
}}<br />
<br />
== Miscellaneous Performance ==<br />
<br />
{{TodoItem<br />
|Use mmap() rather than SYSV for shared buffers?<br />
|This would remove the requirement for SYSV SHM but would introduce portability issues. Anonymous mmap (or mmap to /dev/zero) is required to prevent I/O overhead. We could also consider mmap() for writing WAL.<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00750.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00756.php<br />
}}<br />
<br />
{{TodoItem<br />
|Rather than consider mmap()-ing in 8k pages, consider mmap()'ing entire files into a backend?<br />
|Doing I/O to large tables would consume a lot of address space or require frequent mapping/unmapping. Extending the file also causes mapping problems that might require mapping only individual pages, leading to thousands of mappings. Another problem is that there is no way to _prevent_ I/O to disk from the dirty shared buffers so changes could hit disk before WAL is written.<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01239.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider ways of storing rows more compactly on disk:<br />
* Reduce the row header size?<br />
* Consider reducing on-disk varlena length from four bytes to two because a heap row cannot be more than 64k in length}}<br />
<br />
{{TodoItem<br />
|Consider transaction start/end performance improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00948.php <nowiki>Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow configuration of backend priorities via the operating system<br />
|Though backend priorities make priority inversion during lock waits possible, research shows that this is not a huge problem.<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg00493.php <nowiki>Priorities for users or queries?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing the minimum allowed number of shared buffers<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00157.php <nowiki>Re: [PATCH] Don't bail with legitimate -N/-B options</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider if CommandCounterIncrement() can avoid its AcceptInvalidationMessages() call<br />
* [http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php <nowiki>pgsql: Avoid incrementing the CommandCounter when</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider Cartesian joins when both relations are needed to form an indexscan qualification for a third relation<br />
* [http://archives.postgresql.org/pgsql-performance/2007-12/msg00090.php <nowiki>Re: TB-sized databases</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider not storing a NULL bitmap on disk if all the NULLs are trailing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00624.php <nowiki>Proposal for Null Bitmap Optimization(for Trailing NULLs)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-12/msg00109.php <nowiki>Re: [HACKERS] Proposal for Null Bitmap Optimization(for TrailingNULLs)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Sort large UPDATE/DELETEs so it is done in heap order<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01119.php <nowiki>Possible future performance improvement: sort updates/deletes by ctid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow one transaction to see tuples using the snapshot of another transaction<br />
|This would assist multiple backends in working together. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00400.php <nowiki>Transaction Snapshot Cloning</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00135.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00260.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00466.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the I/O caused by updating tuple hint bits<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00847.php <nowiki>Hint Bits and Write I/O</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00199.php <nowiki>Re: [HACKERS] Hint Bits and Write I/O</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-10/msg00695.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00792.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg01063.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01408.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01453.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid the requirement of freezing pages that are infrequently modified <br />
|If all rows on a page are visible, it is possible to set a bit in the visibility map (once the visibility map is 100% reliable) and not need to freeze the page, avoiding a page rewrite<br />
* http://archives.postgresql.org/message-id/4BF701CF.2090205@agliodbs.com<br />
* http://archives.postgresql.org/pgsql-hackers/2010-06/msg00082.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid reading in b-tree pages when replaying vacuum records in hot standby mode<br />
* [http://archives.postgresql.org/message-id/1272571938.4161.14739.camel@ebony <nowiki>Hot Standby tuning for btree_xlog_vacuum()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure truncation logic to be more resistant to failure<br />
|This also involves not writing dirty buffers for a truncated or dropped relation<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01032.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider adding logic to increase large tables by more than 8k<br />
|This would reduce file system fragmentation<br />
* http://archives.postgresql.org/pgsql-bugs/2011-03/msg00337.php<br />
}}<br />
<br />
== Miscellaneous Other ==<br />
<br />
{{TodoItem<br />
|Deal with encoding issues for filenames in the server filesystem<br />
* {{MessageLink|20090413184335.39BE.52131E4D@oss.ntt.co.jp|a proposed patch here}}<br />
* {{MessageLink|8484.1244655656@sss.pgh.pa.us|some issues about it here}}<br />
* {{MessageLink|20100107103740.97A5.52131E4D@oss.ntt.co.jp|Windows-specific patch here}}<br />
}}<br />
<br />
{{TodoItem<br />
|Deal with encoding issues in the output of localeconv()<br />
* [http://archives.postgresql.org/message-id/40c6d9160904210658y590377cfw6dbbecb53d2b8be0@mail.gmail.com bug report]<br />
* [http://archives.postgresql.org/message-id/49EF8DA0.90008@tpf.co.jp draft patch]<br />
* [http://archives.postgresql.org/message-id/21710.1243620986@sss.pgh.pa.us review of patch]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide schema name and other fields available from SQL GET DIAGNOSTICS in error reports<br />
* [http://archives.postgresql.org/message-id/dcc563d10810211907n3c59a920ia9eb7cd2a6d5ea58@mail.gmail.com <nowiki>How to get schema name which violates fk constraint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg00846.php <nowiki>patch - Report the schema along table name in a referential failure error message</nowiki>]<br />
* {{MessageLink|3191.1263306359@sss.pgh.pa.us|Re: NOT NULL violation and error-message}}<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00213.php <nowiki>the case for machine-readable error fields</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
| Provide [http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html#LIBPQ-CONNECT-FALLBACK-APPLICATION-NAME fallback_application_name] in contrib/pgbench, oid2name, and dblink.<br />
* {{MessageLink|w2g9837222c1004070216u3bc46b3ahbddfdffdbfb46212@mail.gmail.com|fallback_application_name and pgbench}}<br />
}}<br />
<br />
{{TodoItem<br />
|Add 64-bit support to /contrib/pgbench<br />
* http://archives.postgresql.org/pgsql-hackers/2010-07/msg00153.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00705.php<br />
}}<br />
<br />
== Source Code ==<br />
<br />
{{TodoItem<br />
|Add use of 'const' for variables in source tree}}<br />
<br />
{{TodoItemEasy<br />
|Remove warnings created by -Wcast-align}}<br />
<br />
{{TodoItem<br />
|Move platform-specific ps status display info from ps_status.c to ports}}<br />
<br />
{{TodoItem<br />
|Add optional CRC checksum to heap and index pages<br />
|One difficulty is how to prevent hint bit changes from affecting the computed CRC checksum.<br />
* http://archives.postgresql.org/message-id/19934.1226601952%40sss.pgh.pa.us<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00002.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01028.php <nowiki>double-buffering page writes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00524.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01101.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00011.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-11/msg00249.php<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a faster CRC32 algorithm<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01112.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow cross-compiling by generating the zic database on the target system}}<br />
<br />
{{TodoItem<br />
|Improve NLS maintenance of libpgport messages linked onto applications}}<br />
<br />
{{TodoItemDone<br />
|Improve the module installation experience (/contrib, etc)<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00132.php <nowiki>modules</nowiki>]<br />
* {{messageLink|ca33c0a30807231640n6fb4035dod8121a18aa1fa29c@mail.gmail.com|Re: PostgreSQL extensions packaging}}<br />
* {{messageLink|ca33c0a30804061349s41b4d8fcsa9c579454b27ecd2@mail.gmail.com|Database owner installable modules patch}}<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-03/msg00855.php <nowiki>Re: contrib function naming, and upgrade issues</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00912.php <nowiki>search_path vs extensions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Use UTF8 encoding for NLS messages so all server encodings can read them properly}}<br />
<br />
{{TodoItem<br />
|Allow creation of universal binaries for Darwin<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00884.php <nowiki>Getting to universal binaries for Darwin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider GnuTLS if OpenSSL license becomes a problem<br />
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg00892.php<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00040.php <nowiki>[PATCH] Add support for GnuTLS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01213.php <nowiki>TODO: GNU TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider making NAMEDATALEN more configurable in future releases}}<br />
<br />
{{TodoItem<br />
|Research use of signals and sleep wake ups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00003.php <nowiki>Restartable signals 'n all that</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow C++ code to more easily access backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00302.php <nowiki>Mostly Harmless: Welcoming our C++ friends</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider simplifying how memory context resets handle child contexts<br />
* [http://archives.postgresql.org/pgsql-patches/2007-08/msg00067.php <nowiki>Re: Memory leak in nodeAgg</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Create three versions of libpgport to simplify client code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00154.php <nowiki>8.4 TODO item: make src/port support libpq and ecpg directly</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve detection of shared memory segments being used by others by checking the SysV shared memory field 'nattch'<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00656.php <nowiki>postgresql in FreeBSD jails: proposal</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00673.php <nowiki>Re: postgresql in FreeBSD jails: proposal</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using POSIX shared memory to avoid System V shared memory kernel limits<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg00558.php<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the non-threaded Avahi service discovery protocol<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00939.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00097.php <nowiki>Re: Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg01211.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00001.php <nowiki>Re: [HACKERS] Avahi support for Postgresql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce data row alignment requirements on some 64-bit systems<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00369.php <nowiki>[WIP] Reduce alignment requirements on 64-bit systems.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure TOAST internal storage format for greater flexibility<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00049.php <nowiki>Re: PG_PAGE_LAYOUT_VERSION 5 - time for change</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Add regression tests for pg_dump/restore<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01967.php <nowiki>"make install-check-pg_dump" target in src/regress]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Research different memory allocation methods for lists<br />
* http://archives.postgresql.org/pgsql-hackers/2011-04/msg01467.php <br />
}}<br />
<br />
=== /contrib/pg_upgrade ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemDone<br />
|Remove copy_dir() code, or use it<br />
}}<br />
<br />
{{TodoItem<br />
|Handle large object comments<br />
|This is difficult to do because the large object doesn't exist when --schema-only is loaded.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using pg_depend for checking object usage in version.c<br />
}}<br />
<br />
{{TodoItem<br />
|If reindex is necessary, allow it to be done in parallel with pg_dump custom format<br />
}}<br />
<br />
{{TodoItem<br />
|Migrate pg_statistic by dumping it out as a flat file, so analyze is not necessary<br />
|pg_class.oid is not preserved so schema.tablename must be used.<br />
}}<br />
<br />
{{TodoItem<br />
|Improve testing, perhaps using the buildfarm<br />
|The buildfarm has access to multiple versions of PostgreSQL.<br />
}}<br />
<br />
{{TodoItem<br />
|Create machine-readable output of pg_controldata<br />
|This would avoid parsing its output. The problem is we need pg_controldata output from both the old and new clusters so we would need to support both formats.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Windows ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Remove configure.in check for link failure when cause is found}}<br />
<br />
{{TodoItem<br />
|Remove readdir() errno patch when runtime/mingwex/dirent.c rev 1.4 is released}}<br />
<br />
{{TodoItem<br />
|Allow psql to use readline once non-US code pages work with backslashes}}<br />
<br />
{{TodoItem<br />
|Fix problem with shared memory on the Win32 Terminal Server}}<br />
<br />
{{TodoItem<br />
|Improve signal handling<br />
* [http://archives.postgresql.org/pgsql-patches/2005-06/msg00027.php <nowiki>Simplify Win32 Signaling code</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Convert MSVC build system to remove most batch files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00961.php <nowiki>MSVC build system</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support pgxs when using MSVC}}<br />
<br />
{{TodoItem<br />
|Fix MSVC NLS support, like for to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00485.php <nowiki>NLS on MSVC strikes back!</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php <nowiki>Fix for 8.3 MSVC locale (Was [HACKERS] NLS on MSVC strikes back!)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a correct rint() substitute on Windows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00808.php <nowiki>Minor bug in src/port/rint.c</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix global namespace issues when using multiple terminal server sessions<br />
* [http://archives.postgresql.org/message-id/48F3BFCC.8030107@dunslane.net problems with Windows global namespace]}}<br />
<br />
{{TodoItem<br />
|Change from the current autoconf/gmake build system to cmake<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01869.php <nowiki>About CMake (was Re: [COMMITTERS] pgsql: Append major version number and for libraries soname major)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve consistency of path separator usage<br />
* http://archives.postgresql.org/message-id/49C0BDC5.4010002@hagander.net<br />
}}<br />
<br />
{{TodoItem<br />
|Fix cross-compiling on Windows<br />
* http://archives.postgresql.org/pgsql-bugs/2010-10/msg00110.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multiple Postgres clusters running on the same machine to distinguish themselves in the event log<br />
* http://archives.postgresql.org/pgsql-hackers/2011-03/msg01297.php<br />
* http://archives.postgresql.org/pgsql-hackers/2011-05/msg00574.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Wire Protocol Changes ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dynamic character set handling}}<br />
<br />
{{TodoItem<br />
|Add decoded type, length, precision}}<br />
<br />
{{TodoItem<br />
|Mark result columns as known-not-null when possible<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-11/msg01029.php <nowiki>Adding nullable indicator to Describe</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more control over planner treatment of statements being prepared}}<br />
<br />
{{TodoItem<br />
|Use compression?}}<br />
<br />
{{TodoItem<br />
|Update clients to use data types, typmod, schema.table.column names of result sets using new statement protocol}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Documentation ==<br />
<br />
{{TodoItem<br />
|Convert single quotes to apostrophes in the PDF documentation<br />
* [http://archives.postgresql.org/pgsql-docs/2007-12/msg00059.php <nowiki>SGML docs and pdf single-quotes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a manpage for postgresql.conf<br />
* {{messageLink|20080819194311.GH4428@alvh.no-ip.org|A smaller default postgresql.conf}}<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Change the manpage-generating toolchain to use the new XML-based docbook2x tools<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing documentation format from SGML to XML<br />
* [http://archives.postgresql.org/pgsql-docs/2006-12/msg00152.php <nowiki>Re: Authoring Tools WAS: Switching to XML</nowiki>]<br />
* http://archives.postgresql.org/pgsql-docs/2011-04/msg00020.php<br />
* http://wiki.postgresql.org/wiki/Switching_PostgreSQL_documentation_from_SGML_to_XML<br />
}}<br />
<br />
{{TodoItem<br />
|Document support for N<nowiki>' '</nowiki> national character string literals, if it matches the SQL standard<br />
* http://archives.postgresql.org/message-id/1275895438.1849.1.camel@fsopti579.F-Secure.com<br />
}}<br />
<br />
{{TodoItem<br />
|Add diagrams to the documentation<br />
* http://archives.postgresql.org/pgsql-docs/2010-07/msg00001.php<br />
}}<br />
<br />
== Exotic Features ==<br />
<br />
{{TodoItem<br />
|Add pre-parsing phase that converts non-ISO syntax to supported syntax<br />
|This could allow SQL written for other databases to run without modification.}}<br />
<br />
{{TodoItem<br />
|Allow plug-in modules to emulate features from other databases}}<br />
<br />
{{TodoItem<br />
|Add features of Oracle-style packages<br />
|A package would be a schema with session-local variables, public/private functions, and initialization functions. It is also possible to implement these capabilities in any schema and not use a separate &quot;packages&quot; syntax at all.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00384.php <nowiki>proposal for PL packages for 8.3.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing control of upper/lower case folding of unquoted identifiers<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-04/msg00818.php <nowiki>Bringing PostgreSQL torwards the standard regarding case folding</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg01527.php <nowiki>Re: [SQL] Case Preservation disregarding case sensitivity?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00849.php <nowiki>TODO Item: Consider allowing control of upper/lower case folding of unquoted, identifiers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add autonomous transactions<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00893.php <nowiki>autonomous transactions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Give query progress indication<br />
* [[Query progress indication]]<br />
}}<br />
<br />
{{TodoItem<br />
|Rethink our type system<br />
* [[Rethinking datatypes]]<br />
}}<br />
<br />
== Features We Do ''Not'' Want ==<br />
<br />
The following features have been discussed ad nauseum on the PostgreSQL mailing lists and the consensus has been that the project is not interested in them. As such, if you are going to bring them up as potential features, you will want to be familiar with all of the arguments against these features which have been previously made over the years. If you decide to work on such features anyway, you should be aware that you face a higher-than-normal barrier to get the Project to accept them.<br />
<br />
{{TodoItem<br />
|All backends running as threads in a single process (not wanted)<br />
|This eliminates the process protection we get from the current setup. Thread creation is usually the same overhead as process creation on modern systems, so it seems unwise to use a pure threaded model, and MySQL and DB2 have demonstrated that threads introduce as many issues as they solve. Threading specific operations such as I/O, seq scans, and connection management has been discussed and will probably be implemented to enable specific performance features. Moving to a threaded engine would also require halting all other work on PostgreSQL for one to two years.}}<br />
<br />
{{TodoItem<br />
|"Oracle-style" optimizer hints (not wanted)<br />
|Optimizer hints, as implemented in Oracle and other RDBMSes, are used to work around problems in the optimizer and introduce upgrade and maintenance issues. We would rather have such problems reported and fixed. We have discussed a more sophisticated system of per-class cost adjustment instead, but a specification remains to be developed. See [[OptimizerHintsDiscussion|Optimizer Hints Discussion]] for further information.}}<br />
<br />
{{TodoItem<br />
|Embedded server (not wanted)<br />
|While PostgreSQL clients runs fine in limited-resource environments, the server requires multiple processes and a stable pool of resources to run reliably and efficiently. Stripping down the PostgreSQL server to run in the same process address space as the client application would add too much complexity and failure cases. Besides, there are several very mature embedded SQL databases already available.}}<br />
<br />
{{TodoItem<br />
|Obfuscated function source code (not wanted)<br />
|Obfuscating function source code has minimal protective benefits because anyone with super-user access can find a way to view the code. At the same time, it would greatly complicate backups and other administrative tasks. To prevent non-super-users from viewing function source code, remove SELECT permission on pg_proc.<br />
* [http://archives.postgresql.org/pgsql-general/2008-09/msg00668.php <nowiki>Obfuscated stored procedures (was Re: Oracle and Postgresql)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Indeterminate behavior for the GROUP BY clause (not wanted)<br />
|At least one other database product allows specification of a subset of the result columns which GROUP BY would need to be able to provide predictable results; the server is free to return any value from the group. This is not viewed as a desirable feature. PostgreSQL 9.1 will allow result columns that are not referenced by GROUP BY if a primary key for the same table is referenced in GROUP BY.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg00297.php <nowiki>Re: SQL compatibility reminder: MySQL vs PostgreSQL</nowiki>]<br />
}}<br />
<br />
</div><br />
<br />
[[Category:Todo]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Developer_FAQ&diff=14316Developer FAQ2011-05-12T19:58:46Z<p>Schmiddy: /* What areas need work? */</p>
<hr />
<div>{{Languages}}<br />
<br />
== Getting Involved ==<br />
<br />
=== How do I get involved in PostgreSQL development? ===<br />
<br />
Download the code and have a look around. See [[#How_do_I_download.2Fupdate_the_current_source_tree.3F|downloading the source tree]].<br />
<br />
Subscribe to and read the [http://archives.postgresql.org/pgsql-hackers/ pgsql-hackers mailing list] (often termed "hackers"). This is where the major contributors and core members of the project discuss development.<br />
<br />
=== How do I download/update the current source tree? ===<br />
<br />
There are several ways to obtain the source tree. Occasional developers can just get the most recent source tree snapshot from ftp://ftp.postgresql.org/pub/snapshot/.<br />
<br />
Regular developers might want to take advantage of anonymous access to our source code management system. The source tree is currently hosted in git. For details of how to obtain the source from git see http://developer.postgresql.org/pgdocs/postgres/git.html and [[Working with Git]].<br />
<br />
=== What development environment is required to develop code? ===<br />
<br />
PostgreSQL is developed mostly in the C programming language. The source code is targeted at most of the popular Unix platforms and the Windows environment (XP, Windows 2000, and up).<br />
<br />
Most developers run a Unix-like operating system and use an open source tool chain with [http://gcc.gnu.org GCC], [http://www.gnu.org/software/make/make.html GNU Make], [http://www.gnu.org/software/gdb/gdb.html GDB], [http://www.gnu.org/software/autoconf/ Autoconf], and so on. If you have contributed to open source software before, you will probably be familiar with these tools. Developers using this tool chain on Windows make use of [http://www.mingw.org/ MinGW], though most development on Windows is currently done with the Microsoft Visual Studio 2005 (version 8) development environment and associated tools.<br />
<br />
The complete list of required software to build PostgreSQL can be found in the [http://developer.postgresql.org/pgdocs/postgres/install-requirements.html installation instructions].<br />
<br />
Developers who regularly rebuild the source often pass the --enable-depend flag to configure. The result is that if you make a modification to a C header file, all files depend upon that file are also rebuilt.<br />
<br />
src/Makefile.custom can be used to set environment variables, like CUSTOM_COPT, that are used for every compile.<br />
<br />
=== What areas need work? ===<br />
Outstanding features are detailed in [[Todo]].<br />
<br />
You can learn more about these features by consulting the [http://archives.postgresql.org/ archives], the SQL standards and the recommended texts (see [[#What_books_are_good_for_developers.3F|books for developers]]).<br />
<br />
=== How do I get involved in PostgreSQL web site development? ===<br />
<br />
PostgreSQL website development is discussed on the [http://archives.postgresql.org/pgsql-www/ pgsql-www mailing list]. There is a project page where the source code is available at http://pgweb.postgresql.org/.<br />
<br />
== Development Tools and Help ==<br />
<br />
=== How is the source code organized? ===<br />
<br />
If you point your browser at [http://www.postgresql.org/developer/ext.backend.html How PostgreSQL Processes a Query], (also in a checkout of the source code, under src/tools/backend/index.html), you will see few paragraphs describing the data flow, the backend components in a flow chart, and a description of the shared memory area. You can click on any flowchart box to see a description. If you then click on the directory name, you will be taken to the source directory, to browse the actual source code behind it. We also have several README files in some source directories to describe the function of the module. The browser will display these when you enter the directory also.<br />
<br />
Other than documentation in the source tree itself, you can find some papers/presentations discussing the code at http://www.postgresql.org/developer/coding. An excellent presentation is at http://neilconway.org/talks/hacking/<br />
<br />
=== What tools are available for developers? ===<br />
<br />
First, all the files in the src/tools directory are designed for developers.<br />
<br />
RELEASE_CHANGES changes we have to make for each release<br />
backend description/flowchart of the backend directories<br />
ccsym find standard defines made by your compiler<br />
copyright fixes copyright notices<br />
<br />
entab converts spaces to tabs, used by pgindent<br />
find_static finds functions that could be made static<br />
find_typedef finds typedefs in the source code<br />
find_badmacros finds macros that use braces incorrectly<br />
fsync a script to provide information about the cost of cache<br />
syncing system calls<br />
make_ctags make vi 'tags' file in each directory<br />
make_diff make *.orig and diffs of source<br />
make_etags make emacs 'etags' files<br />
make_keywords make comparison of our keywords and SQL'92<br />
make_mkid make mkid ID files<br />
git_changelog used to generate a list of changes for each release<br />
pginclude scripts for adding/removing include files<br />
pgindent indents source files<br />
pgtest a semi-automated build system<br />
thread a thread testing script<br />
<br />
In src/include/catalog:<br />
<br />
unused_oids a script that finds unused OIDs for use in system catalogs<br />
duplicate_oids finds duplicate OIDs in system catalog definitions<br />
<br />
tools/backend was already described in the question-and-answer above.<br />
<br />
Second, you really should have an editor that can handle tags, so you can tag a function call to see the function definition, and then tag inside that function to see an even lower-level function, and then back out twice to return to the original function. Most editors support this via tags or etags files.<br />
<br />
Third, you need to get id-utils from ftp://ftp.gnu.org/gnu/id-utils/<br />
<br />
By running tools/make_mkid, an archive of source symbols can be created that can be rapidly queried.<br />
<br />
Some developers make use of cscope, which can be found at http://cscope.sf.net/. Others use glimpse, which can be found at http://webglimpse.net/.<br />
<br />
tools/make_diff has tools to create patch diff files that can be applied to the distribution. This produces context diffs, which is our preferred format.<br />
<br />
pgindent is used to fix the source code style to conform to our standards, and is normally run at the end of each development cycle; see [[#What.27s_the_formatting_style_used_in_PostgreSQL_source_code.3F|this question]] for more information on our style.<br />
<br />
pginclude contains scripts used to add needed #include's to include files, and removed unneeded #include's.<br />
<br />
When adding built-in objects such as types or functions, you will need to assign OIDs to them. Our convention is that all hand-assigned OIDs are distinct values in the range 1-9999. (It would work mechanically for them to be unique within individual system catalogs, but for clarity we require them to be unique across the whole system.) There is a script called unused_oids in src/include/catalog that shows the currently unused OIDs. To assign a new OID, pick one that is free according to unused_oids, and for bonus points pick one that is nearby to related existing objects. See also the duplicate_oids script, which will complain if you made a mistake.<br />
<br />
=== What's the formatting style used in PostgreSQL source code? ===<br />
<br />
Our standard format BSD style, with each level of code indented one tab, where each tab is four spaces. You will need to set your editor or file viewer to display tabs as four spaces:<br />
<br />
For '''vi''' (in <code>.exrc</code> or <code>.vimrc</code>):<br />
set tabstop=4 shiftwidth=4 noexpandtab<br />
<br />
For '''less''' or '''more''', specify <code>-x4</code> to get the correct indentation.<br />
<br />
The tools/editors directory of the latest sources contains sample settings that can be used with the emacs, xemacs and vim editors, that assist in keeping to PostgreSQL coding standards.<br />
<br />
pgindent will the format code by specifying flags to your operating system's utility indent. pgindent is run on all source files just before each beta test period. It auto-formats all source files to make them consistent. Comment blocks that need specific line breaks should be formatted as block comments, where the comment starts as /*------. These comments will not be reformatted in any way.<br />
<br />
See also [http://developer.postgresql.org/pgdocs/postgres/source-format.html the Formatting section] in the documentation. [http://archives.postgresql.org/message-id/1221125165.5637.12.camel@abbas-laptop This posting] talks about our naming of variable and function names.<br />
<br />
If you're wondering why we bother with this, [http://ezine.daemonnews.org/200112/single_coding_style.html this article] describes the value of a consistent coding style.<br />
<br />
=== Is there a diagram of the system catalogs available? ===<br />
<br />
Yes, we have [http://dalibo.org/_media/articles/catalog.png at least one] ([http://svn.postgresql.fr/repos/materials/advocacy/trunk/posters/catalogs83.svg SVG version]).<br />
<br />
=== What books are good for developers? ===<br />
<br />
There are five good books:<br />
<br />
* An Introduction to Database Systems, by C.J. Date, Addison, Wesley<br />
* A Guide to the SQL Standard, by C.J. Date, et. al, Addison, Wesley<br />
* Fundamentals of Database Systems, by Elmasri and Navathe<br />
* Transaction Processing, by Jim Gray and Andreas Reuter, Morgan Kaufmann<br />
* Transactional Information Systems, by Gerhard Weikum and Gottfried Vossen, Morgan Kaufmann<br />
<br />
=== What is configure all about? ===<br />
<br />
The files configure and configure.in are part of the GNU autoconf package. Configure allows us to test for various capabilities of the OS, and to set variables that can then be tested in C programs and Makefiles. Autoconf is installed on the PostgreSQL main server. To add options to configure, edit configure.in, and then run autoconf to generate configure.<br />
<br />
When configure is run by the user, it tests various OS capabilities, stores those in config.status and config.cache, and modifies a list of *.in files. For example, if there exists a Makefile.in, configure generates a Makefile that contains substitutions for all @var@ parameters found by configure.<br />
<br />
When you need to edit files, make sure you don't waste time modifying files generated by configure. Edit the *.in file, and re-run configure to recreate the needed file. If you run make distclean from the top-level source directory, all files derived by configure are removed, so you see only the file contained in the source distribution.<br />
=== How do I add a new port? ===<br />
<br />
There are a variety of places that need to be modified to add a new port. First, start in the src/template directory. Add an appropriate entry for your OS. Also, use src/config.guess to add your OS to src/template/.similar. You shouldn't match the OS version exactly. The configure test will look for an exact OS version number, and if not found, find a match without version number. Edit src/configure.in to add your new OS. (See configure item above.) You will need to run autoconf, or patch src/configure too.<br />
<br />
Then, check src/include/port and add your new OS file, with appropriate values. Hopefully, there is already locking code in src/include/storage/s_lock.h for your CPU. There is also a src/makefiles directory for port-specific Makefile handling. There is a backend/port directory if you need special files for your OS.<br />
=== Why don't you use threads, raw devices, async-I/O, <insert your favorite wizz-bang feature here>? ===<br />
<br />
There is always a temptation to use the newest operating system features as soon as they arrive. We resist that temptation.<br />
<br />
First, we support 15+ operating systems, so any new feature has to be well established before we will consider it. Second, most new wizz-bang features don't provide dramatic improvements. Third, they usually have some downside, such as decreased reliability or additional code required. Therefore, we don't rush to use new features but rather wait for the feature to be established, then ask for testing to show that a measurable improvement is possible.<br />
<br />
As an example, threads are not currently used instead of multiple processes for backends because:<br />
<br />
* Historically, threads were poorly supported and buggy.<br />
* An error in one backend can corrupt other backends if they're threads within a single process<br />
* Speed improvements using threads are small compared to the remaining backend startup time.<br />
* The backend code would be more complex.<br />
* Terminating backend processes allows the OS to cleanly and quickly free all resources, protecting against memory and file descriptor leaks and making backend shutdown cheaper and faster<br />
* Debugging threaded programs is much harder than debugging worker processes, and core dumps are much less useful<br />
* Sharing of read-only executable mappings and the use of shared_buffers means processes, like threads, are very memory efficient<br />
* Regular creation and destruction of processes helps protect against memory fragmentation, which can be hard to manage in long-running processes<br />
<br />
(Whether individual backend processes should use multiple threads to make use of multiple cores for single queries is a separate question not covered here).<br />
<br />
So, we are not ignorant of new features. It is just that we are cautious about their adoption. The TODO list often contains links to discussions showing our reasoning in these areas.<br />
<br />
==== Why aren't there more compression options when dumping tables? ====<br />
<br />
pg_dump's built-in compression method is gzip.<br />
The primary alternative, bzip2, is normally far too slow to be useful when dumping large tables.<br />
<br />
The two main alternatives regularly proposed for better built-in compression at good speeds are LZO and LZMA/LZMA2/XZ. LZO is released under the GPL, incompatible with PostgreSQL. The LZMA2 code has been released into the public domain, but the C port is a secondary one (C++ is the main development focus) whose code quality hasn't seemed appropriate for this project. And this whole area has traditionally been filled with patent issues that go beyond just the restrictions of the software license.<br />
<br />
Another limitation on changing this is that pg_dump output is intended to be archivable, so we had better be prepared to support compression methods for a very long time. The "latest and greatest" compression method is exactly what we *don't* want.<br />
<br />
See the [http://archives.postgresql.org/pgsql-hackers/2009-02/msg00352.php archives] for an idea what characteristics an alternate compression tool would need to have in order to be considered for use in core PostgreSQL.<br />
<br />
=== How are branches managed? ===<br />
<br />
See [[Working_with_Git#Using_Back_Branches|Using Back Branches]] and [[Committing with Git]] for information about how branches and backporting are handled.<br />
<br />
=== Where can I get a copy of the SQL standards? ===<br />
You are supposed to buy them from [http://www.iso.ch/ ISO] or [http://www.ansi.org ANSI]. Search for ISO/ANSI 9075. ANSI's offer is less expensive, but the contents of the documents are the same between the two organizations.<br />
<br />
Since buying an official copy of the standard is quite expensive, most developers rely on one of the various draft versions available on the Internet. Some of these are:<br />
* SQL-92 http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt<br />
* SQL:1999 http://web.cs.ualberta.ca/~yuan/courses/db_readings/ansi-iso-9075-2-1999.pdf<br />
* SQL:2003 http://www.wiscorp.com/sql_2003_standard.zip<br />
* SQL:2008 (preliminary) http://www.wiscorp.com/sql200n.zip<br />
<br />
The PostgreSQL documentation contains information about PostgreSQL and [http://developer.postgresql.org/pgdocs/postgres/features.html SQL conformance].<br />
<br />
Some further web pages about the SQL standard are:<br />
* http://troels.arvin.dk/db/rdbms/links/#standards<br />
* http://www.wiscorp.com/SQLStandards.html<br />
* http://www.contrib.andrew.cmu.edu/~shadow/sql.html#syntax (SQL-92)<br />
* http://dbs.uni-leipzig.de/en/lokal/standards.pdf (paper)<br />
<br />
Note that having access to a copy of the SQL standard is not necessary to become a useful contributor to PostgreSQL development. Interpreting the standard is difficult and needs years of experience. And most features in PostgreSQL are not specified in the standard anyway.<br />
<br />
=== Where can I get technical assistance? ===<br />
<br />
Many technical questions held by those new to the code have been answered on the pgsql-hackers mailing list - the archives of which can be found at http://archives.postgresql.org/pgsql-hackers/.<br />
<br />
If you cannot find discussion or your particular question, feel free to put it to the list.<br />
<br />
Major contributors also answer technical questions, including questions about development of new features, on IRC at irc.freenode.net in the #postgresql channel.<br />
<br />
=== Why haven't you replaced CVS with SVN, Git, Monotone, VSS, <insert your favorite SCMS here>? ===<br />
The project switched to Git in September 2010.<br />
<br />
== Development Process ==<br />
<br />
=== What do I do after choosing an item to work on? ===<br />
<br />
Send an email to pgsql-hackers with a proposal for what you want to do (assuming your contribution is not trivial). Working in isolation is not advisable because others might be working on the same TODO item, or you might have misunderstood the TODO item. In the email, discuss both the internal implementation method you plan to use, and any user-visible changes (new syntax, etc). For complex patches, it is important to get community feedback on your proposal before starting work. Failure to do so might mean your patch is rejected. If your work is being sponsored by a company, read [http://momjian.us/main/writings/pgsql/company_contributions/ this article] for tips on being more effective.<br />
<br />
Our queue of patches to be reviewed is maintained via a custom [[CommitFest]] web application at http://commitfest.postgresql.org.<br />
<br />
=== How do I test my changes? ===<br />
<br />
==== Basic system testing ====<br />
<br />
The easiest way to test your code is to ensure that it builds against the latest version of the code and that it does not generate compiler warnings.<br />
<br />
It is worth advised that you pass --enable-cassert to configure. This will turn on assertions within the source which will often make bugs more visible because they cause data corruption or segmentation violations. This generally makes debugging much easier.<br />
<br />
Then, perform run time testing via psql.<br />
<br />
==== Runtime environment ====<br />
<br />
To test your modified version of PostgreSQL, it's convenient to install PostgreSQL into a local directory (in your home <br />
directory, for instance) to avoid conflicting with a system wide <br />
installation. Use the ''--prefix='' option to configure to specify an installation <br />
location; ''--with-pgport'' to specify a non-standard default port is <br />
helpful as well. To run this instance, you will need to make sure that the correct <br />
binaries are used; depending on your operating system, environment variables <br />
like PATH and LD_LIBRARY_PATH (on most Linux/Unix-like systems) need to be <br />
set. Setting PGDATA will also be useful.<br />
<br />
To avoid having to set this environment up manually, you may want to use <br />
Greg Smith's [https://github.com/gregs1104/peg peg] scripts,or the<br />
[https://github.com/PGBuildFarm/client-code scripts] that are used on the <br />
buildfarm.<br />
<br />
==== Regression test suite ====<br />
<br />
The next step is to test your changes against the existing regression test suite. To do this, issue "make check" in the root directory of the source tree. If any tests fail, investigate.<br />
<br />
If you've deliberately changed existing behavior, this change might cause a regression test failure but not any actual regression. If so, you should also patch the regression test suite.<br />
<br />
==== Other run time testing ====<br />
<br />
Some developers make use of tools such as valgrind (http://valgrind.kde.org) for memory testing, gprof (which comes with the GNU binutils suite) and oprofile (http://oprofile.sourceforge.net/) for profiling and other related tools.<br />
<br />
==== What about unit testing, static analysis, model checking...? ====<br />
<br />
There have been a number of discussions about other testing frameworks and some developers are exploring these ideas.<br />
<br />
Keep in mind the Makefiles do not have the proper dependencies for include files. You have to do a make clean and then another make. If you are using GCC you can use the --enable-depend option of configure to have the compiler compute the dependencies automatically.<br />
<br />
=== I have developed a patch, what next? ===<br />
<br />
You will need to submit the patch to pgsql-hackers@postgresql.org. To help ensure your patch is reviewed and committed in a timely fashion, please try to follow the guidelines at [[Submitting a Patch]].<br />
<br />
=== What happens to my patch once it is submitted? ===<br />
<br />
It will be reviewed by other contributors to the project and will be either accepted or sent back for further work. The process is explained in more detail at [[Submitting a Patch#Patch review and commit|Submitting a Patch]].<br />
<br />
=== How do I help with reviewing patches? ===<br />
<br />
If you would like to contribute by reviewing a patch in the [http://commitfest.postgresql.org CommitFest] queue, you are most welcome to do so. Please read the guide at [[Reviewing a Patch]] for more information.<br />
<br />
=== Do I need to sign a copyright assignment? ===<br />
<br />
No, contributors keeps their copyright (as is the case for most<br />
European countries anyway). They simply consider themselves to be part of<br />
the Postgres Global Development Group. (It's not even possible to assign<br />
copyright to PGDG, as it's not a legal entity). This is the same way that<br />
the Linux Kernel and many other Open Source projects works.<br />
<br />
=== May I add my own copyright notice where appropriate? ===<br />
<br />
No, please don't. We like to keep the legal information short and crisp.<br />
Additionally, we've heard that could possibly pose problems for<br />
corporate users.<br />
<br />
=== Doesn't the PostgreSQL license itself require to keep the copyright notice intact? ===<br />
<br />
Yes, it does. And it is, because the PostgreSQL Global Development Group<br />
covers all copyright holders. Also note that US law doesn't require any<br />
copyright notice for getting the copyright granted, just like most<br />
European laws.<br />
<br />
== Technical Questions ==<br />
=== How do I efficiently access information in system catalogs from the backend code? ===<br />
<br />
You first need to find the tuples (rows) you are interested in. There are two ways. First, SearchSysCache() and related functions allow you to query the system catalogs using predefined indexes on the catalogs. This is the preferred way to access system tables, because the first call to the cache loads the needed rows, and future requests can return the results without accessing the base table. A list of available caches is located in src/backend/utils/cache/syscache.c. src/backend/utils/cache/lsyscache.c contains many column-specific cache lookup functions.<br />
<br />
The rows returned are cache-owned versions of the heap rows. Therefore, you must not modify or delete the tuple returned by SearchSysCache(). What you should do is release it with ReleaseSysCache() when you are done using it; this informs the cache that it can discard that tuple if necessary. If you neglect to call ReleaseSysCache(), then the cache entry will remain locked in the cache until end of transaction, which is tolerable during development but not considered acceptable for release-worthy code.<br />
<br />
If you can't use the system cache, you will need to retrieve the data directly from the heap table, using the buffer cache that is shared by all backends. The backend automatically takes care of loading the rows into the buffer cache. To do this, open the table with heap_open(). You can then start a table scan with heap_beginscan(), then use heap_getnext() and continue as long as HeapTupleIsValid() returns true. Then do a heap_endscan(). Keys can be assigned to the scan. No indexes are used, so all rows are going to be compared to the keys, and only the valid rows returned.<br />
<br />
You can also use heap_fetch() to fetch rows by block number/offset. While scans automatically lock/unlock rows from the buffer cache, with heap_fetch(), you must pass a Buffer pointer, and ReleaseBuffer() it when completed.<br />
<br />
Once you have the row, you can get data that is common to all tuples, like t_self and t_oid, by merely accessing the HeapTuple structure entries. If you need a table-specific column, you should take the HeapTuple pointer, and use the GETSTRUCT() macro to access the table-specific start of the tuple. You then cast the pointer, for example as a Form_pg_proc pointer if you are accessing the pg_proc table, or Form_pg_type if you are accessing pg_type. You can then access fields of the tuple by using the structure pointer:<br />
<br />
((Form_pg_class) GETSTRUCT(tuple))->relnatts<br />
<br />
Note however that this only works for columns that are fixed-width and never null, and only when all earlier columns are likewise fixed-width and<br />
never null. Otherwise the column's location is variable and you must use heap_getattr() or related functions to extract it from the tuple.<br />
<br />
Also, avoid storing directly into struct fields as a means of changing live tuples. The best way is to use heap_modifytuple() and pass it your original tuple, plus the values you want changed. It returns a palloc'ed tuple, which you pass to heap_update(). You can delete tuples by passing the tuple's t_self to heap_delete(). You use t_self for heap_update() too. Remember, tuples can be either system cache copies, which might go away after you call ReleaseSysCache(), or read directly from disk buffers, which go away when you heap_getnext(), heap_endscan, or ReleaseBuffer(), in the heap_fetch() case. Or it may be a palloc'ed tuple, that you must pfree() when finished.<br />
=== Why are table, column, type, function, view names sometimes referenced as Name or NameData, and sometimes as char *? ===<br />
<br />
Table, column, type, function, and view names are stored in system tables in columns of type Name. Name is a fixed-length, null-terminated type of NAMEDATALEN bytes. (The default value for NAMEDATALEN is 64 bytes.)<br />
<br />
typedef struct nameData<br />
{<br />
char data[NAMEDATALEN];<br />
} NameData;<br />
typedef NameData *Name;<br />
<br />
Table, column, type, function, and view names that come into the backend via user queries are stored as variable-length, null-terminated character strings.<br />
<br />
Many functions are called with both types of names, ie. heap_open(). Because the Name type is null-terminated, it is safe to pass it to a function expecting a char *. Because there are many cases where on-disk names(Name) are compared to user-supplied names(char *), there are many cases where Name and char * are used interchangeably.<br />
<br />
=== Why do we use Node and List to make data structures? ===<br />
<br />
We do this because this allows a consistent way to pass data inside the backend in a flexible way. Every node has a NodeTag which specifies what type of data is inside the Node. Lists are groups of Nodes chained together as a forward-linked list. The ordering of the list elements might or might not be significant, depending on the usage of the particular list.<br />
<br />
Here are some of the List manipulation commands:<br />
<br />
;lfirst(i)<br />
;lfirst_int(i)<br />
;lfirst_oid(i)<br />
:return the data (a pointer, integer or OID respectively) of list cell i.<br />
<br />
;lnext(i)<br />
:return the next list cell after i.<br />
<br />
;foreach(i, list)<br />
:loop through list, assigning each list cell to i.<br />
<br />
It is important to note that i is a <code>ListCell *</code>, not the data in the List cell. You need to use one of the lfirst variants to get at the cell's data.<br />
<br />
Here is a typical code snippet that loops through a List containing <code>Var *</code> cells and processes each one:<br />
<br />
List *list;<br />
ListCell *i;<br />
...<br />
foreach(i, list)<br />
{<br />
Var *var = (Var *) lfirst(i);<br />
...<br />
/* process var here */<br />
}<br />
<br />
;lcons(node, list)<br />
:add node to the front of list, or create a new list with node if list is NIL.<br />
<br />
;lappend(list, node)<br />
:add node to the end of list.<br />
<br />
;list_concat(list1, list2)<br />
:Concatenate list2 on to the end of list1.<br />
<br />
;list_length(list)<br />
:return the length of the list.<br />
<br />
;list_nth(list, i)<br />
:return the i'th element in list, counting from zero.<br />
<br />
;lcons_int, ...<br />
:There are integer versions of these: lcons_int, lappend_int, etc. Also versions for OID lists: lcons_oid, lappend_oid, etc.<br />
<br />
You can print nodes easily inside gdb. First, to disable output truncation when you use the gdb print command:<br />
<br />
(gdb) set print elements 0<br />
<br />
Instead of printing values in gdb format, you can use the next two commands to print out List, Node, and structure contents in a verbose format that is easier to understand. Lists are unrolled into nodes, and nodes are printed in detail. The first prints in a short format, and the second in a long format:<br />
<br />
(gdb) call print(any_pointer)<br />
(gdb) call pprint(any_pointer)<br />
<br />
The output appears in the server log file, or on your screen if you are running a backend directly without a postmaster.<br />
<br />
=== I just added a field to a structure. What else should I do? ===<br />
<br />
The structures passed around in the parser, rewriter, optimizer, and executor require quite a bit of support. Most structures have support routines in src/backend/nodes used to create, copy, read, and output those structures -- in particular, most node types need support in the files copyfuncs.c and equalfuncs.c, and some need support in outfuncs.c and possibly readfuncs.c. Make sure you add support for your new field to these files. Find any other places the structure might need code for your new field -- searching for references to existing fields of the struct is a good way to do that. mkid is helpful with this (see [[#What_tools_are_available_for_developers.3F|available tools]]).<br />
<br />
=== Why do we use palloc() and pfree() to allocate memory? ===<br />
<br />
palloc() and pfree() are used in place of malloc() and free() because we find it easier to automatically free all memory allocated when a query completes. This assures us that all memory that was allocated gets freed even if we have lost track of where we allocated it. There are special non-query contexts that memory can be allocated in. These affect when the allocated memory is freed by the backend.<br />
=== What is ereport()? ===<br />
<br />
ereport() is used to send messages to the front-end, and optionally terminate the current query being processed. See [http://developer.postgresql.org/pgdocs/postgres/error-message-reporting.html here] for more details on how to use it.<br />
<br />
=== What is CommandCounterIncrement()? ===<br />
<br />
Normally, statements can not see the rows they modify. This allows UPDATE foo SET x = x + 1 to work correctly.<br />
<br />
However, there are cases where a transaction needs to see rows affected in previous parts of the transaction. This is accomplished using a Command Counter. Incrementing the counter allows transactions to be broken into pieces so each piece can see rows modified by previous pieces. CommandCounterIncrement() increments the Command Counter, creating a new part of the transaction.<br />
=== What debugging features are available? ===<br />
<br />
First, if you are developing new C code you should ALWAYS work in a build configured with the --enable-cassert and --enable-debug options. Enabling asserts turns on many sanity checking options. Enabling debug symbols supports use of debuggers (such as gdb) to trace through misbehaving code.<br />
<br />
The postgres server has a -d option that allows detailed information to be logged (elog or ereport DEBUGn printouts). The -d option takes a number that specifies the debug level. Be warned that high debug level values generate large log files.<br />
<br />
If the postmaster is running, start psql in one window, then find the PID of the postgres process used by psql using SELECT pg_backend_pid(). Use a debugger to attach to the postgres PID. You can set breakpoints in the debugger and then issue queries from the psql session. If you are looking to find the location that is generating an error or log message, set a breakpoint at errfinish. If you are debugging something that happens during session startup, you can set PGOPTIONS="-W n", then start psql. This will cause startup to delay for n seconds so you can attach to the process with the debugger, set appropriate breakpoints, then continue through the startup sequence.<br />
<br />
If the postmaster is not running, you can actually run the postgres backend from the command line, and type your SQL statement directly. This is almost always a bad way to do things, however, since the usage environment isn't nearly as friendly as psql (no command history for instance) and there's no chance to study concurrent behavior. You might have to use this method if you broke initdb, but otherwise it has nothing to recommend it.<br />
<br />
You can also compile with profiling to see what functions are taking execution time --- configuring with --enable-profiling is the recommended way to set this up. (You usually shouldn't use --enable-cassert when studying performance issues, since the checks it enables are not always cheap.) Profile files from server processes will be deposited in the pgsql/data directory. Profile files from clients such as psql will be put in the client's current directory.<br />
<br />
[[Category:FAQ]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Developer_FAQ&diff=14315Developer FAQ2011-05-12T19:54:15Z<p>Schmiddy: /* Doesn't the PostgreSQL license itself require to keep the copyright notice intact? */ typofix</p>
<hr />
<div>{{Languages}}<br />
<br />
== Getting Involved ==<br />
<br />
=== How do I get involved in PostgreSQL development? ===<br />
<br />
Download the code and have a look around. See [[#How_do_I_download.2Fupdate_the_current_source_tree.3F|downloading the source tree]].<br />
<br />
Subscribe to and read the [http://archives.postgresql.org/pgsql-hackers/ pgsql-hackers mailing list] (often termed "hackers"). This is where the major contributors and core members of the project discuss development.<br />
<br />
=== How do I download/update the current source tree? ===<br />
<br />
There are several ways to obtain the source tree. Occasional developers can just get the most recent source tree snapshot from ftp://ftp.postgresql.org/pub/snapshot/.<br />
<br />
Regular developers might want to take advantage of anonymous access to our source code management system. The source tree is currently hosted in git. For details of how to obtain the source from git see http://developer.postgresql.org/pgdocs/postgres/git.html and [[Working with Git]].<br />
<br />
=== What development environment is required to develop code? ===<br />
<br />
PostgreSQL is developed mostly in the C programming language. The source code is targeted at most of the popular Unix platforms and the Windows environment (XP, Windows 2000, and up).<br />
<br />
Most developers run a Unix-like operating system and use an open source tool chain with [http://gcc.gnu.org GCC], [http://www.gnu.org/software/make/make.html GNU Make], [http://www.gnu.org/software/gdb/gdb.html GDB], [http://www.gnu.org/software/autoconf/ Autoconf], and so on. If you have contributed to open source software before, you will probably be familiar with these tools. Developers using this tool chain on Windows make use of [http://www.mingw.org/ MinGW], though most development on Windows is currently done with the Microsoft Visual Studio 2005 (version 8) development environment and associated tools.<br />
<br />
The complete list of required software to build PostgreSQL can be found in the [http://developer.postgresql.org/pgdocs/postgres/install-requirements.html installation instructions].<br />
<br />
Developers who regularly rebuild the source often pass the --enable-depend flag to configure. The result is that if you make a modification to a C header file, all files depend upon that file are also rebuilt.<br />
<br />
src/Makefile.custom can be used to set environment variables, like CUSTOM_COPT, that are used for every compile.<br />
<br />
=== What areas need work? ===<br />
Outstanding features are detailed in [[Todo]].<br />
<br />
You can learn more about these features by consulting the [http://archives.postgresql.org/ archives], the SQL standards and the recommend texts (see [[#What_books_are_good_for_developers.3F|books for developers]]).<br />
<br />
=== How do I get involved in PostgreSQL web site development? ===<br />
<br />
PostgreSQL website development is discussed on the [http://archives.postgresql.org/pgsql-www/ pgsql-www mailing list]. There is a project page where the source code is available at http://pgweb.postgresql.org/.<br />
<br />
== Development Tools and Help ==<br />
<br />
=== How is the source code organized? ===<br />
<br />
If you point your browser at [http://www.postgresql.org/developer/ext.backend.html How PostgreSQL Processes a Query], (also in a checkout of the source code, under src/tools/backend/index.html), you will see few paragraphs describing the data flow, the backend components in a flow chart, and a description of the shared memory area. You can click on any flowchart box to see a description. If you then click on the directory name, you will be taken to the source directory, to browse the actual source code behind it. We also have several README files in some source directories to describe the function of the module. The browser will display these when you enter the directory also.<br />
<br />
Other than documentation in the source tree itself, you can find some papers/presentations discussing the code at http://www.postgresql.org/developer/coding. An excellent presentation is at http://neilconway.org/talks/hacking/<br />
<br />
=== What tools are available for developers? ===<br />
<br />
First, all the files in the src/tools directory are designed for developers.<br />
<br />
RELEASE_CHANGES changes we have to make for each release<br />
backend description/flowchart of the backend directories<br />
ccsym find standard defines made by your compiler<br />
copyright fixes copyright notices<br />
<br />
entab converts spaces to tabs, used by pgindent<br />
find_static finds functions that could be made static<br />
find_typedef finds typedefs in the source code<br />
find_badmacros finds macros that use braces incorrectly<br />
fsync a script to provide information about the cost of cache<br />
syncing system calls<br />
make_ctags make vi 'tags' file in each directory<br />
make_diff make *.orig and diffs of source<br />
make_etags make emacs 'etags' files<br />
make_keywords make comparison of our keywords and SQL'92<br />
make_mkid make mkid ID files<br />
git_changelog used to generate a list of changes for each release<br />
pginclude scripts for adding/removing include files<br />
pgindent indents source files<br />
pgtest a semi-automated build system<br />
thread a thread testing script<br />
<br />
In src/include/catalog:<br />
<br />
unused_oids a script that finds unused OIDs for use in system catalogs<br />
duplicate_oids finds duplicate OIDs in system catalog definitions<br />
<br />
tools/backend was already described in the question-and-answer above.<br />
<br />
Second, you really should have an editor that can handle tags, so you can tag a function call to see the function definition, and then tag inside that function to see an even lower-level function, and then back out twice to return to the original function. Most editors support this via tags or etags files.<br />
<br />
Third, you need to get id-utils from ftp://ftp.gnu.org/gnu/id-utils/<br />
<br />
By running tools/make_mkid, an archive of source symbols can be created that can be rapidly queried.<br />
<br />
Some developers make use of cscope, which can be found at http://cscope.sf.net/. Others use glimpse, which can be found at http://webglimpse.net/.<br />
<br />
tools/make_diff has tools to create patch diff files that can be applied to the distribution. This produces context diffs, which is our preferred format.<br />
<br />
pgindent is used to fix the source code style to conform to our standards, and is normally run at the end of each development cycle; see [[#What.27s_the_formatting_style_used_in_PostgreSQL_source_code.3F|this question]] for more information on our style.<br />
<br />
pginclude contains scripts used to add needed #include's to include files, and removed unneeded #include's.<br />
<br />
When adding built-in objects such as types or functions, you will need to assign OIDs to them. Our convention is that all hand-assigned OIDs are distinct values in the range 1-9999. (It would work mechanically for them to be unique within individual system catalogs, but for clarity we require them to be unique across the whole system.) There is a script called unused_oids in src/include/catalog that shows the currently unused OIDs. To assign a new OID, pick one that is free according to unused_oids, and for bonus points pick one that is nearby to related existing objects. See also the duplicate_oids script, which will complain if you made a mistake.<br />
<br />
=== What's the formatting style used in PostgreSQL source code? ===<br />
<br />
Our standard format BSD style, with each level of code indented one tab, where each tab is four spaces. You will need to set your editor or file viewer to display tabs as four spaces:<br />
<br />
For '''vi''' (in <code>.exrc</code> or <code>.vimrc</code>):<br />
set tabstop=4 shiftwidth=4 noexpandtab<br />
<br />
For '''less''' or '''more''', specify <code>-x4</code> to get the correct indentation.<br />
<br />
The tools/editors directory of the latest sources contains sample settings that can be used with the emacs, xemacs and vim editors, that assist in keeping to PostgreSQL coding standards.<br />
<br />
pgindent will the format code by specifying flags to your operating system's utility indent. pgindent is run on all source files just before each beta test period. It auto-formats all source files to make them consistent. Comment blocks that need specific line breaks should be formatted as block comments, where the comment starts as /*------. These comments will not be reformatted in any way.<br />
<br />
See also [http://developer.postgresql.org/pgdocs/postgres/source-format.html the Formatting section] in the documentation. [http://archives.postgresql.org/message-id/1221125165.5637.12.camel@abbas-laptop This posting] talks about our naming of variable and function names.<br />
<br />
If you're wondering why we bother with this, [http://ezine.daemonnews.org/200112/single_coding_style.html this article] describes the value of a consistent coding style.<br />
<br />
=== Is there a diagram of the system catalogs available? ===<br />
<br />
Yes, we have [http://dalibo.org/_media/articles/catalog.png at least one] ([http://svn.postgresql.fr/repos/materials/advocacy/trunk/posters/catalogs83.svg SVG version]).<br />
<br />
=== What books are good for developers? ===<br />
<br />
There are five good books:<br />
<br />
* An Introduction to Database Systems, by C.J. Date, Addison, Wesley<br />
* A Guide to the SQL Standard, by C.J. Date, et. al, Addison, Wesley<br />
* Fundamentals of Database Systems, by Elmasri and Navathe<br />
* Transaction Processing, by Jim Gray and Andreas Reuter, Morgan Kaufmann<br />
* Transactional Information Systems, by Gerhard Weikum and Gottfried Vossen, Morgan Kaufmann<br />
<br />
=== What is configure all about? ===<br />
<br />
The files configure and configure.in are part of the GNU autoconf package. Configure allows us to test for various capabilities of the OS, and to set variables that can then be tested in C programs and Makefiles. Autoconf is installed on the PostgreSQL main server. To add options to configure, edit configure.in, and then run autoconf to generate configure.<br />
<br />
When configure is run by the user, it tests various OS capabilities, stores those in config.status and config.cache, and modifies a list of *.in files. For example, if there exists a Makefile.in, configure generates a Makefile that contains substitutions for all @var@ parameters found by configure.<br />
<br />
When you need to edit files, make sure you don't waste time modifying files generated by configure. Edit the *.in file, and re-run configure to recreate the needed file. If you run make distclean from the top-level source directory, all files derived by configure are removed, so you see only the file contained in the source distribution.<br />
=== How do I add a new port? ===<br />
<br />
There are a variety of places that need to be modified to add a new port. First, start in the src/template directory. Add an appropriate entry for your OS. Also, use src/config.guess to add your OS to src/template/.similar. You shouldn't match the OS version exactly. The configure test will look for an exact OS version number, and if not found, find a match without version number. Edit src/configure.in to add your new OS. (See configure item above.) You will need to run autoconf, or patch src/configure too.<br />
<br />
Then, check src/include/port and add your new OS file, with appropriate values. Hopefully, there is already locking code in src/include/storage/s_lock.h for your CPU. There is also a src/makefiles directory for port-specific Makefile handling. There is a backend/port directory if you need special files for your OS.<br />
=== Why don't you use threads, raw devices, async-I/O, <insert your favorite wizz-bang feature here>? ===<br />
<br />
There is always a temptation to use the newest operating system features as soon as they arrive. We resist that temptation.<br />
<br />
First, we support 15+ operating systems, so any new feature has to be well established before we will consider it. Second, most new wizz-bang features don't provide dramatic improvements. Third, they usually have some downside, such as decreased reliability or additional code required. Therefore, we don't rush to use new features but rather wait for the feature to be established, then ask for testing to show that a measurable improvement is possible.<br />
<br />
As an example, threads are not currently used instead of multiple processes for backends because:<br />
<br />
* Historically, threads were poorly supported and buggy.<br />
* An error in one backend can corrupt other backends if they're threads within a single process<br />
* Speed improvements using threads are small compared to the remaining backend startup time.<br />
* The backend code would be more complex.<br />
* Terminating backend processes allows the OS to cleanly and quickly free all resources, protecting against memory and file descriptor leaks and making backend shutdown cheaper and faster<br />
* Debugging threaded programs is much harder than debugging worker processes, and core dumps are much less useful<br />
* Sharing of read-only executable mappings and the use of shared_buffers means processes, like threads, are very memory efficient<br />
* Regular creation and destruction of processes helps protect against memory fragmentation, which can be hard to manage in long-running processes<br />
<br />
(Whether individual backend processes should use multiple threads to make use of multiple cores for single queries is a separate question not covered here).<br />
<br />
So, we are not ignorant of new features. It is just that we are cautious about their adoption. The TODO list often contains links to discussions showing our reasoning in these areas.<br />
<br />
==== Why aren't there more compression options when dumping tables? ====<br />
<br />
pg_dump's built-in compression method is gzip.<br />
The primary alternative, bzip2, is normally far too slow to be useful when dumping large tables.<br />
<br />
The two main alternatives regularly proposed for better built-in compression at good speeds are LZO and LZMA/LZMA2/XZ. LZO is released under the GPL, incompatible with PostgreSQL. The LZMA2 code has been released into the public domain, but the C port is a secondary one (C++ is the main development focus) whose code quality hasn't seemed appropriate for this project. And this whole area has traditionally been filled with patent issues that go beyond just the restrictions of the software license.<br />
<br />
Another limitation on changing this is that pg_dump output is intended to be archivable, so we had better be prepared to support compression methods for a very long time. The "latest and greatest" compression method is exactly what we *don't* want.<br />
<br />
See the [http://archives.postgresql.org/pgsql-hackers/2009-02/msg00352.php archives] for an idea what characteristics an alternate compression tool would need to have in order to be considered for use in core PostgreSQL.<br />
<br />
=== How are branches managed? ===<br />
<br />
See [[Working_with_Git#Using_Back_Branches|Using Back Branches]] and [[Committing with Git]] for information about how branches and backporting are handled.<br />
<br />
=== Where can I get a copy of the SQL standards? ===<br />
You are supposed to buy them from [http://www.iso.ch/ ISO] or [http://www.ansi.org ANSI]. Search for ISO/ANSI 9075. ANSI's offer is less expensive, but the contents of the documents are the same between the two organizations.<br />
<br />
Since buying an official copy of the standard is quite expensive, most developers rely on one of the various draft versions available on the Internet. Some of these are:<br />
* SQL-92 http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt<br />
* SQL:1999 http://web.cs.ualberta.ca/~yuan/courses/db_readings/ansi-iso-9075-2-1999.pdf<br />
* SQL:2003 http://www.wiscorp.com/sql_2003_standard.zip<br />
* SQL:2008 (preliminary) http://www.wiscorp.com/sql200n.zip<br />
<br />
The PostgreSQL documentation contains information about PostgreSQL and [http://developer.postgresql.org/pgdocs/postgres/features.html SQL conformance].<br />
<br />
Some further web pages about the SQL standard are:<br />
* http://troels.arvin.dk/db/rdbms/links/#standards<br />
* http://www.wiscorp.com/SQLStandards.html<br />
* http://www.contrib.andrew.cmu.edu/~shadow/sql.html#syntax (SQL-92)<br />
* http://dbs.uni-leipzig.de/en/lokal/standards.pdf (paper)<br />
<br />
Note that having access to a copy of the SQL standard is not necessary to become a useful contributor to PostgreSQL development. Interpreting the standard is difficult and needs years of experience. And most features in PostgreSQL are not specified in the standard anyway.<br />
<br />
=== Where can I get technical assistance? ===<br />
<br />
Many technical questions held by those new to the code have been answered on the pgsql-hackers mailing list - the archives of which can be found at http://archives.postgresql.org/pgsql-hackers/.<br />
<br />
If you cannot find discussion or your particular question, feel free to put it to the list.<br />
<br />
Major contributors also answer technical questions, including questions about development of new features, on IRC at irc.freenode.net in the #postgresql channel.<br />
<br />
=== Why haven't you replaced CVS with SVN, Git, Monotone, VSS, <insert your favorite SCMS here>? ===<br />
The project switched to Git in September 2010.<br />
<br />
== Development Process ==<br />
<br />
=== What do I do after choosing an item to work on? ===<br />
<br />
Send an email to pgsql-hackers with a proposal for what you want to do (assuming your contribution is not trivial). Working in isolation is not advisable because others might be working on the same TODO item, or you might have misunderstood the TODO item. In the email, discuss both the internal implementation method you plan to use, and any user-visible changes (new syntax, etc). For complex patches, it is important to get community feedback on your proposal before starting work. Failure to do so might mean your patch is rejected. If your work is being sponsored by a company, read [http://momjian.us/main/writings/pgsql/company_contributions/ this article] for tips on being more effective.<br />
<br />
Our queue of patches to be reviewed is maintained via a custom [[CommitFest]] web application at http://commitfest.postgresql.org.<br />
<br />
=== How do I test my changes? ===<br />
<br />
==== Basic system testing ====<br />
<br />
The easiest way to test your code is to ensure that it builds against the latest version of the code and that it does not generate compiler warnings.<br />
<br />
It is worth advised that you pass --enable-cassert to configure. This will turn on assertions within the source which will often make bugs more visible because they cause data corruption or segmentation violations. This generally makes debugging much easier.<br />
<br />
Then, perform run time testing via psql.<br />
<br />
==== Runtime environment ====<br />
<br />
To test your modified version of PostgreSQL, it's convenient to install PostgreSQL into a local directory (in your home <br />
directory, for instance) to avoid conflicting with a system wide <br />
installation. Use the ''--prefix='' option to configure to specify an installation <br />
location; ''--with-pgport'' to specify a non-standard default port is <br />
helpful as well. To run this instance, you will need to make sure that the correct <br />
binaries are used; depending on your operating system, environment variables <br />
like PATH and LD_LIBRARY_PATH (on most Linux/Unix-like systems) need to be <br />
set. Setting PGDATA will also be useful.<br />
<br />
To avoid having to set this environment up manually, you may want to use <br />
Greg Smith's [https://github.com/gregs1104/peg peg] scripts,or the<br />
[https://github.com/PGBuildFarm/client-code scripts] that are used on the <br />
buildfarm.<br />
<br />
==== Regression test suite ====<br />
<br />
The next step is to test your changes against the existing regression test suite. To do this, issue "make check" in the root directory of the source tree. If any tests fail, investigate.<br />
<br />
If you've deliberately changed existing behavior, this change might cause a regression test failure but not any actual regression. If so, you should also patch the regression test suite.<br />
<br />
==== Other run time testing ====<br />
<br />
Some developers make use of tools such as valgrind (http://valgrind.kde.org) for memory testing, gprof (which comes with the GNU binutils suite) and oprofile (http://oprofile.sourceforge.net/) for profiling and other related tools.<br />
<br />
==== What about unit testing, static analysis, model checking...? ====<br />
<br />
There have been a number of discussions about other testing frameworks and some developers are exploring these ideas.<br />
<br />
Keep in mind the Makefiles do not have the proper dependencies for include files. You have to do a make clean and then another make. If you are using GCC you can use the --enable-depend option of configure to have the compiler compute the dependencies automatically.<br />
<br />
=== I have developed a patch, what next? ===<br />
<br />
You will need to submit the patch to pgsql-hackers@postgresql.org. To help ensure your patch is reviewed and committed in a timely fashion, please try to follow the guidelines at [[Submitting a Patch]].<br />
<br />
=== What happens to my patch once it is submitted? ===<br />
<br />
It will be reviewed by other contributors to the project and will be either accepted or sent back for further work. The process is explained in more detail at [[Submitting a Patch#Patch review and commit|Submitting a Patch]].<br />
<br />
=== How do I help with reviewing patches? ===<br />
<br />
If you would like to contribute by reviewing a patch in the [http://commitfest.postgresql.org CommitFest] queue, you are most welcome to do so. Please read the guide at [[Reviewing a Patch]] for more information.<br />
<br />
=== Do I need to sign a copyright assignment? ===<br />
<br />
No, contributors keeps their copyright (as is the case for most<br />
European countries anyway). They simply consider themselves to be part of<br />
the Postgres Global Development Group. (It's not even possible to assign<br />
copyright to PGDG, as it's not a legal entity). This is the same way that<br />
the Linux Kernel and many other Open Source projects works.<br />
<br />
=== May I add my own copyright notice where appropriate? ===<br />
<br />
No, please don't. We like to keep the legal information short and crisp.<br />
Additionally, we've heard that could possibly pose problems for<br />
corporate users.<br />
<br />
=== Doesn't the PostgreSQL license itself require to keep the copyright notice intact? ===<br />
<br />
Yes, it does. And it is, because the PostgreSQL Global Development Group<br />
covers all copyright holders. Also note that US law doesn't require any<br />
copyright notice for getting the copyright granted, just like most<br />
European laws.<br />
<br />
== Technical Questions ==<br />
=== How do I efficiently access information in system catalogs from the backend code? ===<br />
<br />
You first need to find the tuples (rows) you are interested in. There are two ways. First, SearchSysCache() and related functions allow you to query the system catalogs using predefined indexes on the catalogs. This is the preferred way to access system tables, because the first call to the cache loads the needed rows, and future requests can return the results without accessing the base table. A list of available caches is located in src/backend/utils/cache/syscache.c. src/backend/utils/cache/lsyscache.c contains many column-specific cache lookup functions.<br />
<br />
The rows returned are cache-owned versions of the heap rows. Therefore, you must not modify or delete the tuple returned by SearchSysCache(). What you should do is release it with ReleaseSysCache() when you are done using it; this informs the cache that it can discard that tuple if necessary. If you neglect to call ReleaseSysCache(), then the cache entry will remain locked in the cache until end of transaction, which is tolerable during development but not considered acceptable for release-worthy code.<br />
<br />
If you can't use the system cache, you will need to retrieve the data directly from the heap table, using the buffer cache that is shared by all backends. The backend automatically takes care of loading the rows into the buffer cache. To do this, open the table with heap_open(). You can then start a table scan with heap_beginscan(), then use heap_getnext() and continue as long as HeapTupleIsValid() returns true. Then do a heap_endscan(). Keys can be assigned to the scan. No indexes are used, so all rows are going to be compared to the keys, and only the valid rows returned.<br />
<br />
You can also use heap_fetch() to fetch rows by block number/offset. While scans automatically lock/unlock rows from the buffer cache, with heap_fetch(), you must pass a Buffer pointer, and ReleaseBuffer() it when completed.<br />
<br />
Once you have the row, you can get data that is common to all tuples, like t_self and t_oid, by merely accessing the HeapTuple structure entries. If you need a table-specific column, you should take the HeapTuple pointer, and use the GETSTRUCT() macro to access the table-specific start of the tuple. You then cast the pointer, for example as a Form_pg_proc pointer if you are accessing the pg_proc table, or Form_pg_type if you are accessing pg_type. You can then access fields of the tuple by using the structure pointer:<br />
<br />
((Form_pg_class) GETSTRUCT(tuple))->relnatts<br />
<br />
Note however that this only works for columns that are fixed-width and never null, and only when all earlier columns are likewise fixed-width and<br />
never null. Otherwise the column's location is variable and you must use heap_getattr() or related functions to extract it from the tuple.<br />
<br />
Also, avoid storing directly into struct fields as a means of changing live tuples. The best way is to use heap_modifytuple() and pass it your original tuple, plus the values you want changed. It returns a palloc'ed tuple, which you pass to heap_update(). You can delete tuples by passing the tuple's t_self to heap_delete(). You use t_self for heap_update() too. Remember, tuples can be either system cache copies, which might go away after you call ReleaseSysCache(), or read directly from disk buffers, which go away when you heap_getnext(), heap_endscan, or ReleaseBuffer(), in the heap_fetch() case. Or it may be a palloc'ed tuple, that you must pfree() when finished.<br />
=== Why are table, column, type, function, view names sometimes referenced as Name or NameData, and sometimes as char *? ===<br />
<br />
Table, column, type, function, and view names are stored in system tables in columns of type Name. Name is a fixed-length, null-terminated type of NAMEDATALEN bytes. (The default value for NAMEDATALEN is 64 bytes.)<br />
<br />
typedef struct nameData<br />
{<br />
char data[NAMEDATALEN];<br />
} NameData;<br />
typedef NameData *Name;<br />
<br />
Table, column, type, function, and view names that come into the backend via user queries are stored as variable-length, null-terminated character strings.<br />
<br />
Many functions are called with both types of names, ie. heap_open(). Because the Name type is null-terminated, it is safe to pass it to a function expecting a char *. Because there are many cases where on-disk names(Name) are compared to user-supplied names(char *), there are many cases where Name and char * are used interchangeably.<br />
<br />
=== Why do we use Node and List to make data structures? ===<br />
<br />
We do this because this allows a consistent way to pass data inside the backend in a flexible way. Every node has a NodeTag which specifies what type of data is inside the Node. Lists are groups of Nodes chained together as a forward-linked list. The ordering of the list elements might or might not be significant, depending on the usage of the particular list.<br />
<br />
Here are some of the List manipulation commands:<br />
<br />
;lfirst(i)<br />
;lfirst_int(i)<br />
;lfirst_oid(i)<br />
:return the data (a pointer, integer or OID respectively) of list cell i.<br />
<br />
;lnext(i)<br />
:return the next list cell after i.<br />
<br />
;foreach(i, list)<br />
:loop through list, assigning each list cell to i.<br />
<br />
It is important to note that i is a <code>ListCell *</code>, not the data in the List cell. You need to use one of the lfirst variants to get at the cell's data.<br />
<br />
Here is a typical code snippet that loops through a List containing <code>Var *</code> cells and processes each one:<br />
<br />
List *list;<br />
ListCell *i;<br />
...<br />
foreach(i, list)<br />
{<br />
Var *var = (Var *) lfirst(i);<br />
...<br />
/* process var here */<br />
}<br />
<br />
;lcons(node, list)<br />
:add node to the front of list, or create a new list with node if list is NIL.<br />
<br />
;lappend(list, node)<br />
:add node to the end of list.<br />
<br />
;list_concat(list1, list2)<br />
:Concatenate list2 on to the end of list1.<br />
<br />
;list_length(list)<br />
:return the length of the list.<br />
<br />
;list_nth(list, i)<br />
:return the i'th element in list, counting from zero.<br />
<br />
;lcons_int, ...<br />
:There are integer versions of these: lcons_int, lappend_int, etc. Also versions for OID lists: lcons_oid, lappend_oid, etc.<br />
<br />
You can print nodes easily inside gdb. First, to disable output truncation when you use the gdb print command:<br />
<br />
(gdb) set print elements 0<br />
<br />
Instead of printing values in gdb format, you can use the next two commands to print out List, Node, and structure contents in a verbose format that is easier to understand. Lists are unrolled into nodes, and nodes are printed in detail. The first prints in a short format, and the second in a long format:<br />
<br />
(gdb) call print(any_pointer)<br />
(gdb) call pprint(any_pointer)<br />
<br />
The output appears in the server log file, or on your screen if you are running a backend directly without a postmaster.<br />
<br />
=== I just added a field to a structure. What else should I do? ===<br />
<br />
The structures passed around in the parser, rewriter, optimizer, and executor require quite a bit of support. Most structures have support routines in src/backend/nodes used to create, copy, read, and output those structures -- in particular, most node types need support in the files copyfuncs.c and equalfuncs.c, and some need support in outfuncs.c and possibly readfuncs.c. Make sure you add support for your new field to these files. Find any other places the structure might need code for your new field -- searching for references to existing fields of the struct is a good way to do that. mkid is helpful with this (see [[#What_tools_are_available_for_developers.3F|available tools]]).<br />
<br />
=== Why do we use palloc() and pfree() to allocate memory? ===<br />
<br />
palloc() and pfree() are used in place of malloc() and free() because we find it easier to automatically free all memory allocated when a query completes. This assures us that all memory that was allocated gets freed even if we have lost track of where we allocated it. There are special non-query contexts that memory can be allocated in. These affect when the allocated memory is freed by the backend.<br />
=== What is ereport()? ===<br />
<br />
ereport() is used to send messages to the front-end, and optionally terminate the current query being processed. See [http://developer.postgresql.org/pgdocs/postgres/error-message-reporting.html here] for more details on how to use it.<br />
<br />
=== What is CommandCounterIncrement()? ===<br />
<br />
Normally, statements can not see the rows they modify. This allows UPDATE foo SET x = x + 1 to work correctly.<br />
<br />
However, there are cases where a transaction needs to see rows affected in previous parts of the transaction. This is accomplished using a Command Counter. Incrementing the counter allows transactions to be broken into pieces so each piece can see rows modified by previous pieces. CommandCounterIncrement() increments the Command Counter, creating a new part of the transaction.<br />
=== What debugging features are available? ===<br />
<br />
First, if you are developing new C code you should ALWAYS work in a build configured with the --enable-cassert and --enable-debug options. Enabling asserts turns on many sanity checking options. Enabling debug symbols supports use of debuggers (such as gdb) to trace through misbehaving code.<br />
<br />
The postgres server has a -d option that allows detailed information to be logged (elog or ereport DEBUGn printouts). The -d option takes a number that specifies the debug level. Be warned that high debug level values generate large log files.<br />
<br />
If the postmaster is running, start psql in one window, then find the PID of the postgres process used by psql using SELECT pg_backend_pid(). Use a debugger to attach to the postgres PID. You can set breakpoints in the debugger and then issue queries from the psql session. If you are looking to find the location that is generating an error or log message, set a breakpoint at errfinish. If you are debugging something that happens during session startup, you can set PGOPTIONS="-W n", then start psql. This will cause startup to delay for n seconds so you can attach to the process with the debugger, set appropriate breakpoints, then continue through the startup sequence.<br />
<br />
If the postmaster is not running, you can actually run the postgres backend from the command line, and type your SQL statement directly. This is almost always a bad way to do things, however, since the usage environment isn't nearly as friendly as psql (no command history for instance) and there's no chance to study concurrent behavior. You might have to use this method if you broke initdb, but otherwise it has nothing to recommend it.<br />
<br />
You can also compile with profiling to see what functions are taking execution time --- configuring with --enable-profiling is the recommended way to set this up. (You usually shouldn't use --enable-cassert when studying performance issues, since the checks it enables are not always cheap.) Profile files from server processes will be deposited in the pgsql/data directory. Profile files from clients such as psql will be put in the client's current directory.<br />
<br />
[[Category:FAQ]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=New_phpPgAdmin_Plugin_Architecture_GSoC_2011&diff=14254New phpPgAdmin Plugin Architecture GSoC 20112011-05-08T02:27:47Z<p>Schmiddy: Cleaned up grammar, syntax, and spelling</p>
<hr />
<div>== Developers ==<br />
Student: Leonardo Augusto Sápiras<br />
<br />
Mentor: Jehan-Guillaume de Rorthais<br />
<br />
Co-Mentor: Andreas Scherbaum<br />
<br />
== Synopsis ==<br />
<br />
This project will create a new plugin architecture for phpPgAdmin, using the Hook Pattern.<br />
<br />
<br />
== Benefits to the PostgreSQL Community ==<br />
<br />
The current phpPgAdmin plugin architecture is deprecated, having only one plugin, Slony. Today, to create a plugin, a developer needs to write intrusive code inside the phpPgAdmin core, as Slony does today, and it is not good. With a new architecture, more ideas could be developed inside phpPgAdmin without intrusive code.<br />
<br />
With a good plugin architecture, new plugins will be more easily created and maintained, phPgAdmin will have more users, and possibly more developers and collaborators as well.<br />
<br />
There are some ideas that are waiting for a new architecture to be developed as plugins. Like:<br />
<br />
* dbdesigner plugin<br />
* pgpooladmin plugin<br />
* crud plugin<br />
<br />
This project has been discussed in the phpPgAdmin e-mail list some time ago, and between me and Mr. Jehan-Guillaume de Rorthais some months ago. Last year Jehan-Guillaume was my mentor, so I would like to have him as my mentor again.<br />
<br />
<br />
== Quantifiable results ==<br />
<br />
# Refactor the current plugin architecture, creating a plugin manager to deal with the plugins;<br />
# Create a new plugin, to give the users a live example of how to create and integrate a plugin with the PPA core;<br />
# Tests;<br />
# Documentation: how to create a plugin for phpPgAdmin and integrate it with its components (action buttons, browser tree, trail, tabs, navigation links, top links);<br />
# If I have time enough, I will refactor the current Slony plugin, for this feature works with the new architecture.<br />
<br />
<br />
== Project Details ==<br />
<br />
=== Refactor the current plugin architecture ===<br />
<br />
In my participation in the Google Summer of Code 2011, I have the goal to refactor the current plugin architecture.<br />
<br />
The new plugins will have the following structure: <br />
<br />
<code><br />
-plugin<br />
|--conf/<br />
| |--config.inc.php<br />
|--lang/<br />
| |--recoded<br />
| | |--english.php<br />
| |--english.php<br />
| |--Makefile<br />
|--js/<br />
|--classes/<br />
|--images/<br />
|--help/<br />
|--tests/ (optional)<br />
|--themes/ (optional)<br />
|--ppa_plugin.php<br />
</code><br />
<br />
As can be seen above, the plugins will have their own pages, translation and configuration files.<br />
<br />
The new plugin architecture will be developed using the [http://stevenblack.com/HooksAndAnchorsDesignPattern.html Hooks Pattern]. This way, the plugins will register their functions to the events they want to hook to. There are a lot of applications that use this concept, like the [http://api.drupal.org/api/drupal/includes--module.inc/group/hooks/7 Drupal Open Source CMS] does.<br />
<br />
<br />
=== Inch-stones ===<br />
<br />
The plugins on this architecture will be able to:<br />
<br />
* Add an entry in the browser tree in any level<br />
* Add an entry in the tabs<br />
* Add an entry in the trailer<br />
* Add an entry in the navigation links<br />
* Add an entry in the action buttons<br />
* Add an entry in the top links<br />
<br />
[[File:ppa_gsoc2011_screen.jpg]]<br />
<br />
<br />
<br />
=== Plugin activation ===<br />
<br />
To activate new plugins in phpPgAdmin, a new variable (array) $conf['plugins'] will be added in the PPA configuration file (config.inc.php). So, to activate a plugin, the user just needs to add the plugin's name in this array. Example:<br />
<br />
<br />
<code><br />
/* config.inc.php */<br />
...<br />
$conf['plugins'][] = 'ppa_slony';<br />
$conf['plugins'][] = 'ppa_x';<br />
$conf['plugins'][] = 'ppa_another';<br />
...<br />
</code><br />
<br />
<br />
=== Plugin execution ===<br />
<br />
phpPgAdmin will have a plugin manager, which as the name implies, will manage activated plugins. Below is an example of how the plugin_manager.php might look:<br />
<br />
<code><br />
<?php<br />
class PluginManager {<br />
...<br />
function add_plugin($obj_plugin) {<br />
$this->plugins_list[$obj_plugin->get_name()] = $obj_plugin;<br />
}<br />
<br />
function get_plugin($plugin_name) {<br />
return $this->plugins_list[$plugin_name];<br />
}<br />
<br />
function add_plugins_functions($plugin_name, $when, $function_name) {<br />
$this->plugins_functions[$when][] = array('plugin_name' => $plugin_name, 'function_name'=> $function_name);<br />
}<br />
function execute_plugins_funtions($when) {<br />
foreach ($this->plugins_functions[$when] as $node) {<br />
$plugin_name = $node['plugin_name'];<br />
$function_name = $node['function_name'];<br />
$obj_plugin = $this->get_plugin($plugin_name); <br />
<br />
if (method_exists($obj_plugin, $function_name)) {<br />
call_user_func(array($obj_plugin, $function_name));<br />
}<br />
}<br />
}<br />
...<br />
}<br />
?><br />
</code><br />
<br />
<br />
This plugin manager will be used in the lib.inc.php file, instantiating and registering the plugins and their functions, like the example below:<br />
<br />
<br />
<code><br />
<?php<br />
/* lib.inc.php */<br />
...<br />
$obj_plugin_manager = new PluginManager(); <br />
//register the plugins and their functions<br />
foreach ($activated_plugins as $plugin) {<br />
include_once('./plugins/'.$plugin.'/plugin.php');<br />
$obj_plugin = new $plugin($obj_plugin_manager);<br />
$obj_plugin_manager->add_plugin($obj_plugin);<br />
}<br />
...<br />
?><br />
</code><br />
<br />
<br />
In the plugin, the functions will be registered with an attribute saying where they will be used by phpPgAdmin's core. Below an example of a simple plugin:<br />
<br />
<br />
<code><br />
<?php<br />
...<br />
class Plugin1 {<br />
...<br />
function __construct($obj_plugin_manager) {<br />
$obj_plugin_manager->add_plugins_functions($this->name,<br />
'before_trail_creation',<br />
'create_trail_links');<br />
$obj_plugin_manager->add_plugins_functions($this->name,<br />
'after_show_tabs',<br />
'show_tab_links');<br />
}<br />
<br />
function create_trail_links() {<br />
/* show the plugin's trail links */<br />
}<br />
<br />
function show_tab_links() {<br />
/* show the plugin's tab links */<br />
}<br />
}<br />
?><br />
</code><br />
<br />
<br />
So, in determined places of phpPgAdmin, e.g. the functions that create the browser tree, trail or tabs, the plugin's function will be called as below:<br />
<br />
<code><br />
<?php<br />
...<br />
$obj_plugin_manager->execute_plugins_funtions('before_trail_creation');<br />
...<br />
$obj_plugin_manager->execute_plugins_funtions('after_show_tabs');<br />
...<br />
?><br />
</code><br />
<br />
This way, the plugin will 'say' to phpPgAdmin core, what are its elements that will be shown in the PPA components.<br />
<br />
To demonstrate, I created a [http://fit.faccat.br/~leonardo/gsoc2011/projeto_plugin_3.tar.gz basic functional example].<br />
<br />
<br />
== Project Schedule ==<br />
<br />
During the Google Summer of Code 2011, I will be on-line on IRC (irc.freenode.net) channels #phppgadmin, #postgresql and #gsoc available to talk. And, at least once a week, I will contact my mentor reporting what was done, what I will do in the next week, and what is preventing me from carrying out a certain activity. I will also write my activities in my [http://sapiras.blogspot.com blog].<br />
<br />
If there are other projects ideas being developed at the same time, it is important that developers of these applications need to be contacted to provide their versions to run tests. But that will be defined together with my mentor and phpPgAdmin developers.<br />
<br />
A copy of this schedule can be found at [https://www.google.com/calendar/embed?src=j021kmt0up95b6iqfnpmbgav2o%40group.calendar.google.com&ctz=America/Sao_Paulo Google Calendar] (with some small changes).<br />
<br />
<br />
'''April 25 until May 22 (Community Bonding Period)''': During that period, I will be in contact to my mentor and phpPgAdmin developers, explaining more detail about the accepted proposal. I will also review my project with them to see if is not missing anything before start coding;<br />
<br />
'''May 23''': I start coding, creating the new basic plugin structure that will be used as example for other developers;<br />
<br />
'''May 27''': Meeting with the mentor, when I will show him the basic plugin architecture, and start developing the Plugin Manager;<br />
<br />
'''June 3''': Meeting with the mentor. Start developing the integration among the basic plugin example and the tabs and top links;<br />
<br />
'''June 10''': Meeting with the mentor;<br />
<br />
'''June 17''': Meeting with the mentor. At this date, I will start developing the integration between plugins and the trail;<br />
<br />
'''June 24''': Meeting with the mentor. At this date, I will start working in the integration between plugins and the navigation links;<br />
<br />
'''July 1''': Meeting with the mentor;<br />
<br />
'''July 8''': Meeting with the mentor. Finish the integration with navigation links. Start developing the integration with the action buttons;<br />
<br />
'''July 11 until 15''': Creation and submission of the mid-term evaluation;<br />
<br />
'''July 15''': Meeting with the mentor;<br />
<br />
'''July 22''': Meeting with the mentor. Start integrating the plugins with the browser tree;<br />
<br />
'''July 29''': Meeting with the mentor;<br />
<br />
'''August 5''': Meeting with the mentor. Finish the integration with the browser tree. Start testing and making final modifications;<br />
<br />
'''August 12''': Meeting with the mentor. On this day, I will stop coding, and start creating the final documentation;<br />
<br />
'''August 19''': Meeting with the mentor. Review of the documentation;<br />
<br />
'''August 22''': Finish the documentation;<br />
<br />
'''August 22 until 26''': Submission of the final evaluation;<br />
<br />
== Completeness Criteria ==<br />
<br />
This project will be accomplished if the new Architecture Plugin:<br />
<br />
* Enable developers to create plugins easily having a clear documentation<br />
* Enable the news plugins to be added in the action buttons, browser tree, tabs, trail, navigation links and top links<br />
* Enable plugins to not have intrusive codes in phpPgAdmin's core<br />
* Have an easy way to enable and disable plugins</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=14032User:Schmiddy2011-04-17T18:19:37Z<p>Schmiddy: /* TODO Items / Things to Investigate */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# <del>segfaults in openjade while building PDF of docs. Docs should be updated to tell users to avoid 1.4devel openjade.</del><br />
# Several users have complained about bookmarks disappearing/reappearing in the doc PDFs for 8.3 and 9.0. What's causing this?<br />
# <del>Why do psql's \z and friends not autocomplete tablenames?</del>[http://archives.postgresql.org/pgsql-hackers/2010-10/msg02002.php patch]<br />
# <del>On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.</del><br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
# ugliness in makefiles for doc builds<br />
# document steps to get doc builds working on OS X. Not easy at all.. having problems with gettext/getopt, which are dependencies of 'xmlto'.<br />
# have psql give an error message upon a badly formatted .pgpass file<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=13977User:Schmiddy2011-04-13T01:31:22Z<p>Schmiddy: /* TODO Items / Things to Investigate */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# segfaults in openjade while building PDF of docs. Docs should be updated to tell users to avoid 1.4devel openjade.<br />
# Several users have complained about bookmarks disappearing/reappearing in the doc PDFs for 8.3 and 9.0. What's causing this?<br />
# <del>Why do psql's \z and friends not autocomplete tablenames?</del>[http://archives.postgresql.org/pgsql-hackers/2010-10/msg02002.php patch]<br />
# <del>On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.</del><br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
# ugliness in makefiles for doc builds<br />
# document steps to get doc builds working on OS X. Not easy at all.. having problems with gettext/getopt, which are dependencies of 'xmlto'.<br />
# have psql give an error message upon a badly formatted .pgpass file<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=13903User:Schmiddy2011-04-10T23:28:49Z<p>Schmiddy: /* TODO Items / Things to Investigate */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# segfaults in openjade while building PDF of docs. Docs should be updated to tell users to avoid 1.4devel openjade.<br />
# <del>Why do psql's \z and friends not autocomplete tablenames?</del>[http://archives.postgresql.org/pgsql-hackers/2010-10/msg02002.php patch]<br />
# <del>On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.</del><br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
# ugliness in makefiles for doc builds<br />
# document steps to get doc builds working on OS X. Not easy at all.. having problems with gettext/getopt, which are dependencies of 'xmlto'.<br />
# have psql give an error message upon a badly formatted .pgpass file<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=13840User:Schmiddy2011-04-08T20:04:13Z<p>Schmiddy: /* TODO Items / Things to Investigate */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# segfaults in openjade while building PDF of docs. Docs should be updated to tell users to avoid 1.4devel openjade.<br />
# <del>Why do psql's \z and friends not autocomplete tablenames?</del>[http://archives.postgresql.org/pgsql-hackers/2010-10/msg02002.php patch]<br />
# <del>On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.</del><br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
# ugliness in makefiles for doc builds<br />
# document steps to get doc builds working on OS X. Not easy at all.. having problems with gettext/getopt, which are dependencies of 'xmlto'.<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=13839User:Schmiddy2011-04-08T20:01:16Z<p>Schmiddy: </p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# <del>segfaults in openjade while building PDF of docs</del><br />
# <del>Why do psql's \z and friends not autocomplete tablenames?</del>[http://archives.postgresql.org/pgsql-hackers/2010-10/msg02002.php patch]<br />
# <del>On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.</del><br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
# ugliness in makefiles for doc builds<br />
<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Warm_Standby&diff=12666Warm Standby2010-12-02T02:10:22Z<p>Schmiddy: Mention that archive_mode must be on for any of this to work. Make explicit whether we're talking about master or standby in a few places.</p>
<hr />
<div>__NOTOC__<br />
There are a couple available Projects available to help you setup a warm standby system: <br />
<br />
* Use the walmgr.py portion of Skype's [https://developer.skype.com/SkypeGarage/DbProjects/SkyTools SkyTools] package which will handle PITR backups from a primary to a single slave<br />
* Utilize Command Prompt's [https://projects.commandprompt.com/public/pitrtools PITR tools] to set everything up<br />
<br />
But to actually get a warm standby up manually is actually a pretty simple process. The following are notes only and intended to help your understanding. If you want to get this working correctly then please follow the manual, which is comprehensive and accurately maintained.<br />
<br />
[http://www.postgresql.org/docs/current/static/warm-standby.html Warm Standby Manual]<br />
<br />
== Pre-process recommendations ==<br />
*Use [http://www.postgresql.org/docs/current/static/pgstandby.html pg_standby] for your restore_command in the recovery.conf file on the standby. pg_standby is included in PostgreSQL 8.3, and you can copy the source from there to compile it for 8.2 yourself. It isn't compatible with 8.1. <br />
*Set up your standby host's environment and directory structure exactly the same as your primary. Otherwise you'll need to spend time changing any symlinks you've created on the primary for xlogs, tablespaces, or whatnot which is really just opportunity for error.<br />
*Pre-configure both the postgresql.conf and recovery.conf files for your standby. I usually keep all of my different config files for all of my different servers in a single, version-controlled directory that I can then check out and symlink to. Again, consistent environment & directory setups make symlinks your best friend.<br />
*Use ssh keys for simply, and safely, transferring files between hosts.<br />
*Follow all of the advice in the manual with respect to handling errors.<br />
<br />
== Outline of steps to get warm standby working ==<br />
* Make sure archive_mode is on in the master's postgresql.conf.<br />
* Set archive_command in the master's postgresql.conf. rysnc is a popular choice or you can just use one of the examples from the docs. I use: <br />
<code><pre><br />
rsync -a %p postgres@standbyhost:/path/to/wal_archive/%f<br />
</pre></code><br />
**You must use a command here that does atomic copies, meaning that the file will never appear under the destination filename until it has been completely copied over. This keeps the standby server from trying to read a partial file. rsync is known to work. A notable command that isn't atomic is scp. If you want to use scp for this purpose, you will need to transfer files into another directory on the secondary, then move them to where the restore command looks for them after the transfer is complete.<br />
***If you're using pg_standby, it will refuse to apply files unless they are the right length, which lowers the risk of non-atomic copies being applied. On Windows it even sleeps a bit after that to give time for things to settle. Performing the copy non-atomically is still a bad idea you should avoid.<br />
*Reload the master's config -- either: SELECT pg_reload_conf(); from psql or: pg_ctl reload -D data_dir/ . If you had to set archive_mode on, you'll have to restart your postgres server: pg_ctl restart -D data_dir/ .<br />
*Verify that the WALs are being shipped to their destination.<br />
*In psql, SELECT pg_start_backup('some_label');<br />
*Run your base backup. Again, rsync is good for this with something as simple as: <br />
<code><pre><br />
rsync -a --progress /path/to/data_dir/* postgres@standbyhost:/path/to/data_dir/ <br />
</pre></code><br />
*I'd suggest running this in a screen term window, the --progress flag will let you watch to see how far along the rsync is. The -a flag will preserve symlinks as well as all file permissions & ownership.<br />
*In psql, SELECT pg_stop_backup();<br />
**This drops a file to be archived that will have the same name as the first WAL shipped after the call to pg_start_backup() with a .backup suffix. Inside will be the start & stop WAL records defining the range of WAL files needed to be replayed before you can consider bringing the standby out of recovery.<br />
*Drop in, or symlink, your recovery.conf file in the standby's data_dir.<br />
**The restore command should use pg_standby (its help/README are simple and to the point). I'd recommend redirecting all output from pg_standby to a log file that you can then watch to verify that everything is working correctly once you've started things.<br />
*Drop in, or symlink, your standby's postgresql.conf file.<br />
**If you don't symlink your pg_xlog directory to write WALs to a separate drive, you can safely delete everything under data_dir/pg_xlog on the standby host.<br />
*Start the standby db server with a normal: pg_ctl start -D /path/to/data_dir/<br />
*run a: tail -f on your standby log and watch to make sure that it's replaying logs. If everything's cool you'll see some info on each WAL file, in order, that the standby looks for along with 'success' messages. If it can't find the files for some reason, you'll see repeated messages like: 'WAL file not present yet. Checking for trigger file...' (assuming you set up pg_standby to look for a trigger file in your recovery_command).<br />
<br />
Execute this entire process at least a couple times, bringing up the standby into normal operations mode once it's played through all of the necessary WAL files (as noted in the .backup file) so that you can connect to it and verify that everything looks good, before doing all of this and leaving it running indefinitely. Once you do it a couple times, it becomes dirt simple. <br />
<br />
== Adjusting frequency of WAL updates in 8.1 ==<br />
<br />
Often people want to know that their secondary is never more than some amount behind the primary. The archive_timeout feature introduced into 8.2 allows doing that. If you're using WAL replication with 8.1, you can force 16MB worth of WAL activity that doesn't leave any changes behind with a hack like this:<br />
<br />
<code><pre><br />
create table xlog_switch as<br />
select '0123456789ABCDE' from generate_series(1,1000000);<br />
drop table xlog_switch;<br />
</pre></code><br />
<br />
If you put that into cron etc. to run via psql and you can make the window for log shipping as fine as you'd like even with no activity. <br />
If you do it too often you're increasing the odds it will interfere with real transactions though and it will use up more disk space; every couple of minutes is probably as often as you'd want to do this. Using archive_timeout doesn't have this issue, the manual suggests it can be set to only a few seconds if necessary.<br />
<br />
== Additional resources ==<br />
*[http://www.kennygorman.com/wordpress/?p=249 pg_standby lag monitoring]<br />
*[http://scale-out-blog.blogspot.com/2009/02/simple-ha-with-postgresql-point-in-time.html Simple HA with PITR]<br />
*[http://www.travishegner.com/2009/06/postgresql-83-warm-stand-by-replication.html PostgreSQL 8.3 Warm Stand-by Replication]: tutorial with Ubuntu specifics<br />
*[http://michsan.blogspot.com/2008/08/using-pgstandby-for-high-availability.html Using pg_standby for high availability of Postgresql]: tutorial that covers Debian, using 8.3 pg_standby on 8.2<br />
*Source material: <br />
** [http://archives.postgresql.org/pgsql-general/2008-01/msg01587.php warm standby examples] <br />
** [http://archives.postgresql.org/sydpug/2006-10/msg00001.php Creating an 8.2 warm-standby demo system]<br />
** [http://archives.postgresql.org/pgsql-general/2007-06/msg00015.php PITR Base Backup on an idle 8.1 server]<br />
<br />
[[Category:Replication]][[Category:Backup]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=12334User:Schmiddy2010-10-29T19:12:21Z<p>Schmiddy: /* PostgreSQL Notes and Misc */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# segfaults in openjade while building PDF of docs<br />
# <del>Why do psql's \z and friends not autocomplete tablenames?</del>[http://archives.postgresql.org/pgsql-hackers/2010-10/msg02002.php patch]<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
<br />
<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=12278User:Schmiddy2010-10-17T03:39:23Z<p>Schmiddy: /* TODO Items / Things to Investigate */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# segfaults in openjade while building PDF of docs<br />
# Why do psql's \z and friends not autocomplete tablenames?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
# make distclean doesn't get rid of .html files created by "make html"?<br />
<br />
<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=User:Schmiddy&diff=12277User:Schmiddy2010-10-17T01:00:53Z<p>Schmiddy: /* Josh Kupershmidt */</p>
<hr />
<div>== PostgreSQL Notes and Misc ==<br />
<br />
=== TODO Items / Things to Investigate ===<br />
<br />
# CREATE TABLE newschema.newtable (LIKE oldschema.oldtable INCLUDING INDEXES) -> on 8.3, renames indexes named "foo_idx" to "foo_key", though looks like this is fixed in later branches?<br />
# <del>column-level UPDATE privs + LOCK TABLE: [http://archives.postgresql.org/pgsql-hackers/2010-10/msg01042.php patch]</del><br />
# segfaults in openjade while building PDF of docs<br />
# Why do psql's \z and friends not autocomplete tablenames?<br />
# On OS X only: \dn pub[TAB] gives "public/" as the autocompleted version of schema "public"<br />
# On OS X only: segfault in psql's autocomplete, due to buggy readline library in OS X.<br />
# Fix up html builds of docs to move towards XHTML compliance<br />
# PL/pgSQL function to handle atomic table swaps - useful for snapshot materialized view refreshes with no locks held on the original table until close to COMMIT time. <br />
<br />
<br />
<br />
; Contact Info<br />
: [mailto:josh**at**kupershmidt.org Email: Josh Kupershmidt]<br />
: [http://kupershmidt.org Josh Kupershmidt's Homepage]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Todo&diff=12060Todo2010-09-26T17:45:13Z<p>Schmiddy: /* CLUSTER */ add thread link to "clustering system catalog indexes"</p>
<hr />
<div><div style="margin: 1ex 1em; float: right;"><br />
__TOC__<br />
</div><br />
<br />
This list contains '''all known PostgreSQL bugs and feature requests'''. If you would like to work on an item, please read the [[Developer FAQ]] first. There is also a [[Development_information|development information page]].<br />
<br />
* {{TodoPending}} - marks ordinary, incomplete items<br />
* {{TodoEasy}} - marks items that are easier to implement<br />
* {{TodoDone}} - marks changes that are done, and will appear in the PostgreSQL 9.1 release.<br />
<br />
For help on editing this list, please see [[Talk:Todo]]. <b>Please do not add items here without discussion on the mailing list.</b><br />
<br />
<div style="padding: 1ex 4em;"><br />
== Administration ==<br />
<br />
{{TodoItem<br />
|Allow administrators to cancel multi-statement idle transactions<br />
|This allows locks to be released, but it is complex to report the cancellation back to the client.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01340.php <nowiki>Cancelling idle in transaction state</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00441.php <nowiki>Re: Cancelling idle in transaction state</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Check for unreferenced table files created by transactions that were in-progress when the server terminated abruptly<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00096.php <nowiki>Removing unreferenced files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Set proper permissions on non-system schemas during db creation<br />
|Currently all schemas are owned by the super-user because they are copied from the template1 database. However, since all objects are inherited from the template database, it is not clear that setting schemas to the db owner is correct.}}<br />
<br />
{{TodoItem<br />
|Allow log_min_messages to be specified on a per-module basis<br />
|This would allow administrators to see more detailed information from specific sections of the backend, e.g. checkpoints, autovacuum, etc. Another idea is to allow separate configuration files for each module, or allow arbitrary SET commands to be passed to them. See also [[Logging Brainstorm]].}}<br />
<br />
{{TodoItem<br />
|Simplify ability to create partitioned tables<br />
|This would allow creation of partitioned tables without requiring creation of triggers or rules for INSERT/UPDATE/DELETE, and constraints for rapid partition selection. Options could include range and hash partition selection. See also [[Table partitioning]]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow auto-selection of partitioned tables for min/max() operations<br />
|There was a patch on -hackers from July 2009, but it has not been merged: [http://archives.postgresql.org/pgsql-hackers/2009-07/msg01115.php <nowiki>MIN/MAX optimization for partitioned table</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Allow custom variables to appear in pg_settings()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00850.php <nowiki>Re: count(*) performance improvement ideas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have custom variables be transaction safe<br />
* {{MessageLink|4B577E9F.8000505@dunslane.net|Custom GUCs still a bit broken}}<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the SQL standard mechanism whereby REVOKE ROLE revokes only the privilege granted by the invoking role, and not those granted by other roles<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00010.php <nowiki>Re: Grantor name gets lost when grantor role dropped</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve server security options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01875.php <nowiki>Re: [0/4] Proposal of SE-PostgreSQL patches</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00000.php <nowiki>Re: [0/4] Proposal of SE-PostgreSQL patches</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent query cancel packets from being replayed by an attacker, especially when using SSL<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00345.php <nowiki>Replay attack of query cancel</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a way to query the log collector subprocess to determine what the currently active log file is<br />
* [http://archives.postgresql.org/pgsql-general/2008-11/msg00418.php <nowiki>Current log files when rotating?</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow the client to authenticate the server in a Unix-domain socket connection, e.g., using SO_PEERCRED<br />
* http://archives.postgresql.org/message-id/20090401173756.GB21229@svana.org<br />
}}<br />
<br />
{{TodoItem<br />
|Allow custom daemons to be automatically stopped/started along postmaster<br />
|This allows easier administration of daemons like user job schedulers or replication-related daemons.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01701.php <nowiki>Re: scheduler in core</nowiki>]<br />
}}<br />
<br />
=== Configuration files ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow pg_hba.conf to specify host names along with IP addresses<br />
|Host name lookup could occur when the postmaster reads the pg_hba.conf file, or when the backend starts. Another solution would be to reverse lookup the connection IP and check that hostname against the host names in pg_hba.conf. We could also then check that the host name maps to the IP address. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00569.php <nowiki>TODO Item: Allow pg_hba.conf to specify host names along with IP addresses</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg00613.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow postgresql.conf file values to be changed via an SQL API, perhaps using SET GLOBAL}}<br />
<br />
{{TodoItem<br />
|Allow the server to be stopped/restarted via an SQL API}}<br />
<br />
{{TodoItem<br />
|Consider normalizing fractions in postgresql.conf, perhaps using '%'<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00550.php <nowiki>Fractions in GUC variables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow Kerberos to disable stripping of realms so we can check the username@realm against multiple realms<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00009.php <nowiki>krb_match_realm patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add functions to check correctness of configuration files before they are loaded "live"}}<br />
<br />
{{TodoItem<br />
|Improve LDAP authentication configuration options<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01745.php <nowiki>Proposed Patch - LDAPS support for servers on port 636 w/o TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add external tool to auto-tune some postgresql.conf parameters<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00000.php <nowiki>Re: Overhauling GUCS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00033.php <nowiki>Simple postgresql.conf wizard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add 'hostgss' pg_hba.conf option to allow GSS link-level encryption<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01454.php <nowiki>Re: Plans for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Process pg_hba.conf keywords as case-insensitive<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00432.php <nowiki>More robust pg_hba.conf parsing/error logging</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Tablespaces ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow a database in tablespace t1 with tables created in tablespace t2 to be used as a template for a new database created with default tablespace t2<br />
|Currently all objects in the default database tablespace must have default tablespace specifications. This is because new databases are created by copying directories. If you mix default tablespace tables and tablespace-specified tables in the same directory, creating a new database from such a mixed directory would create a new database with tables that had incorrect explicit tablespaces. To fix this would require modifying pg_class in the newly copied database, which we don't currently do.}}<br />
<br />
{{TodoItem<br />
|Allow reporting of which objects are in which tablespaces<br />
|This item is difficult because a tablespace can contain objects from multiple databases. There is a server-side function that returns the databases which use a specific tablespace, so this requires a tool that will call that function and connect to each database to find the objects in each database for that tablespace.}}<br />
<br />
{{TodoItem<br />
|Allow WAL replay of CREATE TABLESPACE to work when the directory structure on the recovery computer is different from the original}}<br />
<br />
{{TodoItem<br />
|Allow per-tablespace quotas}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Statistics Collector ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow statistics last vacuum/analyze execution times to be displayed without requiring track_counts to be enabled<br />
* [http://archives.postgresql.org/pgsql-docs/2007-04/msg00028.php <nowiki>row-level stats and last analyze time</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Clear table counters on TRUNCATE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00169.php <nowiki>Small TRUNCATE glitch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Allow the clearing of cluster-level statistics<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-03/msg00917.php <nowiki>Resetting cluster-wide statistics</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Point-In-Time Recovery (PITR) ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Create dump tool for write-ahead logs for use in determining transaction id for point-in-time recovery<br />
|This is useful for checking PITR recovery.}}<br />
<br />
{{TodoItem<br />
|Allow recovery.conf to support the same syntax as postgresql.conf, including quoting<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00497.php <nowiki>recovery.conf parsing problems</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg00684.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow archive_mode to be changed without server restart?<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01655.php <nowiki>Enabling archive_mode without restart</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider avoiding WAL switching via archive_timeout if there has been no database activity<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01469.php <nowiki>archive_timeout behavior for no activity</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg00395.php <nowiki>Re: archive_timeout behavior for no activity</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|[http://archives.postgresql.org/message-id/4B901D73.8030003@agliodbs.com Expose pg_controldata via SQL interface]<br />
|Helpful for monitoring replicated databases; [http://archives.postgresql.org/message-id/4B959D7A.6010907@joeconway.com initial patch]}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SSL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SSL authentication/encryption over unix domain sockets<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00924.php <nowiki>Re: Spoofing as the postmaster</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL key file permission checks to be optionally disabled when sharing SSL keys with other applications<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php <nowiki>BUG #3809: SSL &quot;unsafe&quot; private key permissions bug</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow SSL CRL files to be re-read during configuration file reload, rather than requiring a server restart<br />
|Unlike SSL CRT files, CRL (Certificate Revocation List) files are updated frequently<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00832.php <nowiki>Automatic CRL reload</nowiki>]<br />
Alternatively or additionally supporting OCSP (online certificate security protocol) would provide real-time revocation discovery without reloading<br />
}}<br />
<br />
{{TodoItem<br />
| Allow automatic selection of SSL client certificates from a certificate store<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00406.php <nowiki>Allow multiple certificates or keys in the postgresql.crt/.key files</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Send full certificate server chain to client<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-12/msg00145.php BUG #5245: Full Server Certificate Chain Not Sent to client]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Standby server mode ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
| Allow pg_xlogfile_name() to be used in recovery mode<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1001190135vd9f62f1sa7868abc1ea61d12@mail.gmail.com <nowiki>Streaming replication and pg_xlogfile_name()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Fix things so that any such variables inherited from the server environment are intentionally *NOT* used for making SR connections.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01011.php <nowiki>Re: Parameter name standby_mode</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Add a new privilege for connecting for streaming replication<br />
* [http://archives.postgresql.org/message-id/3f0b79eb1003040247p6b092241of91784a505e9abd8@mail.gmail.com <nowiki>Streaming replication and privilege</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Add support for synchronous replication.<br />
}}<br />
<br />
{{TodoItem<br />
| Add capability to take and send a base backup over the streaming replication connection, making it possible to initialize a new standby server from a running primary server without a WAL archive or other access to the primary server. <br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00136.php<br />
}}<br />
<br />
{{TodoItem<br />
| Find a way to do hot file system backups on standby servers<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01727.php<br />
}}<br />
<br />
{{TodoItem<br />
| Change walsender so that it applies per-role settings<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00642.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Data Types ==<br />
<br />
{{TodoItem<br />
|Change NUMERIC to enforce the maximum precision}}<br />
<br />
{{TodoItemDone<br />
|Reduce storage space for small NUMERICs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01331.php <nowiki>Saving space for common kinds of numeric values</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-02/msg00505.php <nowiki>Numeric patch to add special-case representations for &lt; 8 bytes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php <nowiki>Re: Reducing NUMERIC size for 8.3</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix data types where equality comparison isn't intuitive, e.g. box}}<br />
<br />
{{TodoItem<br />
|Add support for public SYNONYMs<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00519.php <nowiki>Proposal for SYNONYMS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for SQL-standard GENERATED/IDENTITY columns<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg00543.php <nowiki>Re: Three weeks left until feature freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00038.php <nowiki>GENERATED ... AS IDENTITY, Was: Re: Feature Freeze</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00344.php <nowiki>Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00076.php <nowiki>Re: [HACKERS] Behavior of GENERATED columns per SQL2003</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00604.php <nowiki>IDENTITY/GENERATED patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider placing all sequences in a single table, or create a system view<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a special data type for regular expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg01067.php <nowiki>Why is there a tsquery data type?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce BIT data type overhead using short varlena headers<br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00273.php <nowiki>storage size of &quot;bit&quot; data type..</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow adding/renaming/removing enumerated values to an existing enumerated data type<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01718.php <nowiki>Re: [COMMITTERS] pgsql: Update: &lt; * Allow adding enumerated values to an existing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support scoped IPv6 addresses in inet type<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-05/msg00111.php <nowiki>strange problem with ip6</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add JSON (JavaScript Object Notation) data type<br />
|This would behave similar to the XML data type, which is stored as text, but allows element lookup and conversion functions.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01494.php <nowiki>PATCH: Add hstore_to_json()</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg00001.php <nowiki>Re: PATCH: Add hstore_to_json()</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg01092.php <nowiki>Proposal: Add JSON support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00057.php <nowiki>Re: Proposal: Add JSON support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Considering improving performance of computing CHAR() value lengths<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00900.php <nowiki>char() overhead on read-only workloads not so insignifcant as the docs claim it is...</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01787.php <nowiki>Re: [PATCH] backend: compare word-at-a-time in bcTruelen</nowiki>]<br />
}}<br />
<br />
=== Domains ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Fix CREATE CAST on DOMAINs<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-05/msg00072.php <nowiki>bug? non working casts for domain</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg01681.php <nowiki>TODO: Fix CREATE CAST on DOMAINs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow domains to be cast<br />
* [http://archives.postgresql.org/pgsql-hackers/2003-06/msg01206.php <nowiki>Domain casting still doesn't work right</nowiki>] <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00289.php <nowiki>domain casting?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make domains work better with polymorphic functions<br />
* [http://archives.postgresql.org/message-id/4887.1228700773@sss.pgh.pa.us Polymorphic types vs. domains]<br />
* [http://archives.postgresql.org/message-id/15535.1238774571@sss.pgh.pa.us some difficulties with fixing it]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Dates and Times ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow infinite intervals just like infinite timestamps}}<br />
<br />
{{TodoItem<br />
|Allow TIMESTAMP WITH TIME ZONE to store the original timezone information, either zone name or offset from UTC<br />
|If the TIMESTAMP value is stored with a time zone name, interval computations should adjust based on the time zone rules. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-10/msg00705.php <nowiki>timestamp with time zone a la sql99</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix SELECT '0.01 years'::interval, '0.01 months'::interval}}<br />
<br />
{{TodoItem<br />
|Have timestamp subtraction not call justify_hours()?<br />
* [http://archives.postgresql.org/pgsql-sql/2006-10/msg00059.php <nowiki>timestamp subtraction (was Re: formatting intervals with to_char)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve timestamptz subtraction to be DST-aware<br />
|Currently subtracting one date from another that crosses a daylight savings time adjustment can return '1 day 1 hour', but adding that back to the first date returns a time one hour in the future. This is caused by the adjustment of '25 hours' to '1 day 1 hour', and '1 day' is the same time the next day, even if daylight savings adjustments are involved.}}<br />
<br />
{{TodoItem<br />
|Fix interval display to support values exceeding 2^31 hours}}<br />
<br />
{{TodoItem<br />
|Add overflow checking to timestamp and interval arithmetic}}<br />
<br />
{{TodoItem<br />
|Add function to allow the creation of timestamps using parameters<br />
* http://archives.postgresql.org/pgsql-performance/2010-06/msg00232.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Arrays ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add support for arrays of domains<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00114.php <nowiki>Re: updated WIP: arrays of composites</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single-byte header storage for array elements}}<br />
<br />
{{TodoItem<br />
|Add function to detect if an array is empty<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00475.php <nowiki>Re: array_length()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of empty arrays<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01033.php <nowiki>So what's an &quot;empty&quot; array anyway?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULLs in arrays<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-11/msg00009.php <nowiki>BUG #4509: array_cat's null behaviour is inconsistent</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Binary Data ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve vacuum of large objects, like contrib/vacuumlo?}}<br />
<br />
{{TodoItem<br />
|Auto-delete large objects when referencing row is deleted<br />
|contrib/lo offers this functionality.}}<br />
<br />
{{TodoItem<br />
|Allow read/write into TOAST values like large objects<br />
|This requires the TOAST column to be stored EXTERNAL.}}<br />
<br />
{{TodoItem<br />
|Add API for 64-bit large object access<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00781.php <nowiki>64-bit API for large objects</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== MONEY Data Type ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add locale-aware MONEY type, and support multiple currencies<br />
* [http://archives.postgresql.org/pgsql-general/2005-08/msg01432.php <nowiki>A real currency type</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01181.php <nowiki>Money type todos?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|MONEY dumps in a locale-specific format making it difficult to restore to a system with a different locale}}<br />
<br />
{{TodoItem<br />
|Allow MONEY to be easily cast to/from other numeric data types}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Text Search ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dictionaries to change the token that is passed on to later dictionaries<br />
* [http://archives.postgresql.org/pgsql-patches/2007-11/msg00081.php <nowiki>a tsearch2 (8.2.4) dictionary that only filters out stopwords</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a function-based API for '@@' searches<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00511.php <nowiki>Simplifying Text Search</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve text search error messages<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00966.php <nowiki>Poorly designed tsearch NOTICEs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01146.php <nowiki>Re: Poorly designed tsearch NOTICEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing error to warning for strings larger than one megabyte<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00190.php <nowiki>BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00062.php <nowiki>Re: [BUGS] BUG #3975: tsearch2 index should not bomb out of 1Mb limit</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|tsearch and tsdicts regression tests fail in Turkish locale on glibc<br />
* [http://archives.postgresql.org/message-id/49749645.5070801@gmx.net tsearch with Turkish locale]<br />
}}<br />
<br />
{{TodoItem<br />
|tsquery negator operator treated as part of lexeme<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-06/msg00346.php BUG #4887: inclusion operator (@>) on tsqeries behaves not conforming to documentation]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== XML ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow xml arrays to be cast to other data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00981.php <nowiki>proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00231.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00471.php <nowiki>Re: proposal casting from XML[] to int[], numeric[], text[]</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add XML Schema validation and xmlvalidate function (SQL:2008)}}<br />
<br />
{{TodoItem<br />
|Add xmlvalidatedtd variant to support validating against a DTD?}}<br />
<br />
{{TodoItem<br />
|Relax-NG validation; libxml2 supports this already}}<br />
<br />
{{TodoItem<br />
|Make it work reliably for non-UTF8 server encoding (xpath()) in particular is known to not work)<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-01/msg00135.php <nowiki>BUG #4622: xpath only work in utf-8 server encoding</nowiki>] <br />
* http://archives.postgresql.org/message-id/4110.1238973350@sss.pgh.pa.us}}<br />
<br />
{{TodoItem<br />
|Extra functions from SQL:2006: XMLDOCUMENT, XMLCAST, XMLTEXT}}<br />
<br />
{{TodoItem<br />
|XMLNAMESPACES support in XMLELEMENT and elsewhere}}<br />
<br />
{{TodoItem<br />
|XSLT support; already available in contrib/xml2, but needs API fixes and adaptation to xml type.}}<br />
<br />
{{TodoItem<br />
|XML Canonical: Convert XML documents to canonical form to compare them. libxml2 has support for this.}}<br />
<br />
{{TodoItem<br />
|Pretty-printing XML: Parse a document and serialize it back in some indented form. libxml2 might support this.}}<br />
<br />
{{TodoItem<br />
|XMLQUERY (from SQL/XML standard)}}<br />
<br />
{{TodoItem<br />
|In some cases shredding could be better option (if there is no need in keeping XML docs entirely; if we have already developed tools that understand only relational data; etc) -- it would be a separate module that implements annotated schema decomposition technique, similar to DB2 and SQL Server functionality.}}<br />
<br />
{{TodoItem<br />
| Nested or repeated xpath() apparently mess up namespaces [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00097.php] [http://archives.postgresql.org/pgsql-bugs/2008-03/msg00144.php] [http://archives.postgresql.org/pgsql-general/2008-03/msg00295.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/message-id/004f01c90e91$138e9d10$3aabd730$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItem<br />
|XPath: Adding the <x> at the root causes problems [http://archives.postgresql.org/pgsql-bugs/2008-05/msg00184.php] [http://archives.postgresql.org/pgsql-bugs/2008-07/msg00054.php] [http://archives.postgresql.org/pgsql-general/2008-07/msg00613.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table needs to be implemented/implementable to get rid of contrib/xml2 [http://archives.postgresql.org/pgsql-general/2008-05/msg00823.php]}}<br />
<br />
{{TodoItem<br />
|xpath_table is pretty broken anyway [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02424.php]}}<br />
<br />
{{TodoItem<br />
|better handling of XPath data types [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00616.php] [http://archives.postgresql.org/message-id/004a01c90e90$4b986d90$e2c948b0$@anstett@iaas.uni-stuttgart.de]}}<br />
<br />
{{TodoItemDone<br />
|xpath_exists() is needed. It checks, whether or not the path specified exists in the XML value. (W/o this function we need to use weird "array_dims(xpath(...)) IS NOT NULL" syntax.)}}<br />
<br />
{{TodoItem<br />
|better handling of PIs and DTDs in xmlconcat() [http://archives.postgresql.org/message-id/200904211211.n3LCB09p008988@wwwmaster.postgresql.org]}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Functions ==<br />
<br />
{{TodoItem<br />
|Allow INET subnet tests using non-constants to be indexed}}<br />
<br />
{{TodoItem<br />
|Add INET overlaps operator, for use by exclusion constraints <br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00845.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow to_date() and to_timestamp() to accept localized month names}}<br />
<br />
{{TodoItem<br />
|Add missing parameter handling in to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg00948.php <nowiki>Re: to_char and i18n</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Throw an error from to_char() instead of printing a string of "#" when a number doesn't fit in the desired output format.<br />
* discussed in [http://archives.postgresql.org/message-id/37ed240d0907290836w42187222n18664dfcbcb445b1@mail.gmail.com "to_char, support for EEEE format"]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow to_char() on interval values to accumulate the highest unit requested<br />
|2= Some special format flag would be required to request such accumulation. Such functionality could also be added to EXTRACT. Prevent accumulation that crosses the month/day boundary because of the uneven number of days in a month.<br />
* to_char(INTERVAL '1 hour 5 minutes', 'MI') =&gt; 65<br />
* to_char(INTERVAL '43 hours 20 minutes', 'MI' ) =&gt; 2600<br />
* to_char(INTERVAL '43 hours 20 minutes', 'WK:DD:HR:MI') =&gt; 0:1:19:20<br />
* to_char(INTERVAL '3 years 5 months','MM') =&gt; 41}}<br />
<br />
{{TodoItem<br />
|Allow SQL-language functions to reference parameters by parameter name<br />
|Currently SQL-language functions can only refer to dollar parameters, e.g. $1}}<br />
<br />
{{TodoItem<br />
|Add SPI_gettypmod() to return the typemod for a TupleDesc}}<br />
<br />
{{TodoItem<br />
|Enforce typmod for function inputs, function results and parameters for spi_prepare'd statements called from PLs<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01403.php <nowiki>Re: BUG #2917: spi_prepare doesn't accept typename aliases</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01160.php <nowiki>RFC for adding typmods to functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow holdable cursors in SPI}}<br />
<br />
{{TodoItem<br />
|Tighten function permission checks<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00568.php <nowiki>Re: Security leak with trigger functions?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix IS OF so it matches the ISO specification, and add documentation<br />
* [http://archives.postgresql.org/pgsql-patches/2003-08/msg00060.php <nowiki>Re: [HACKERS] IS OF</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00060.php <nowiki>ToDo: add documentation for operator IS OF</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add overlaps geometric operators that ignore point overlaps<br />
* http://archives.postgresql.org/pgsql-hackers/2010-03/msg00861.php<br />
}}<br />
<br />
{{TodoItem<br />
|Implement Boyer-Moore searching in LIKE queries<br />
* {{messageLink|27645.1220635769@sss.pgh.pa.us|TODO item: Implement Boyer-Moore searching (First time hacker)}}<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent malicious functions from being executed with the permissions of unsuspecting users<br />
|Index functions are safe, so VACUUM and ANALYZE are safe too. Triggers, CHECK and DEFAULT expressions, and rules are still vulnerable. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00268.php <nowiki>Some notes about the index-functions security vulnerability</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce memory usage of aggregates in set returning functions<br />
* [http://archives.postgresql.org/pgsql-performance/2008-01/msg00031.php <nowiki>Re: Performance of aggregates over set-returning functions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix /contrib/ltree operator<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-11/msg00044.php <nowiki>BUG #3720: wrong results at using ltree</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|<nowiki>Fix inconsistent precedence of =, &gt;, and &lt; compared to &lt;&gt;, &gt;=, and &lt;=</nowiki><br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00145.php <nowiki>BUG #3822: Nonstandard precedence for comparison operators</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix regular expression bug when using complex back-references<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00000.php <nowiki>BUG #3645: regular expression back references seem broken</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have /contrib/dblink reuse unnamed connections<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00895.php <nowiki>dblink un-named connection doesn't get re-used</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve formatting of pg_get_viewdef() output<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01648.php <nowiki>pg_get_viewdef formattiing</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01885.php <nowiki>Re: pretty print viewdefs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add printf()-like functionality<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix to_number() handling for values not matching the format string<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01447.php <nowiki>Re: numeric_to_number() function skipping some digits</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add function to dump pg_depend information cleanly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00226.php <nowiki>Elementary dependency look-up</nowiki>]<br />
}}<br />
<br />
== Multi-Language Support ==<br />
<br />
{{TodoItem<br />
|Add NCHAR (as distinguished from ordinary varchar),}}<br />
<br />
{{TodoItem<br />
|Allow more fine-grained collation selection; add CREATE COLLATION.<br />
|Right now the collation is fixed at database creation time.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-03/msg00932.php <nowiki>Re: Patch for collation using ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-08/msg00039.php <nowiki>FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-08/msg00309.php <nowiki>Re: FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00110.php <nowiki>Proof of concept COLLATE support with patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2005-09/msg00020.php <nowiki>For review: Initial support for COLLATE</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01121.php <nowiki>Proposed COLLATE implementation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-01/msg00767.php <nowiki>TODO item: locale per database patch (new iteration)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-03/msg00233.php <nowiki>Re: FW: Win32 unicode vs ICU</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-09/msg00662.php <nowiki>Re: Fixed length data types issue</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00557.php <nowiki>[WIP] collation support revisited (phase 1)</nowiki>]<br />
* [[Todo:Collate]]<br />
* [[Todo:ICU]]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01362.php <nowiki>WIP patch: Collation support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00012.php <nowiki>Re: WIP patch: Collation support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00868.php <nowiki>PGDay.it collation discussion notes</nowiki>]<br />
* [http://www.unicode.org/unicode/reports/tr10/ Unicode collation algorithm]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a LOCALE option to CREATE DATABASE, as a shorthand<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00119.php <nowiki> Re: 8.4 open items list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support multiple simultaneous character sets, per SQL:2008}}<br />
<br />
{{TodoItem<br />
|Improve UTF8 combined character handling?}}<br />
<br />
{{TodoItem<br />
|Add octet_length_server() and octet_length_client()}}<br />
<br />
{{TodoItem<br />
|Make octet_length_client() the same as octet_length()?}}<br />
<br />
{{TodoItem<br />
|Fix problems with wrong runtime encoding conversion for NLS message files}}<br />
<br />
{{TodoItem<br />
|Add URL to more complete multi-byte regression tests<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-07/msg00272.php <nowiki>Multi-byte and client side character encoding tests for copy command..</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix contrib/fuzzystrmatch to work with multibyte encodings<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00047.php <nowiki> soundex function returns UTF-16 characters</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00138.php <nowiki> dmetaphone woes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Set client encoding based on the client operating system encoding<br />
|Currently client_encoding is set in postgresql.conf, which defaults to the server encoding. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg01696.php <nowiki>Re: [GENERAL] invalid byte sequence ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Change memory allocation for multi-byte functions so memory is allocated inside conversion functions<br />
|Currently we preallocate memory based on worst-case usage.}}<br />
<br />
{{TodoItem<br />
|Add ability to use case-insensitive regular expressions on multi-byte characters<br />
|ILIKE already works with multi-byte characters<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00433.php <nowiki>Regexps vs. locale</nowiki>]<br />
* {{MessageLink|20091201210024.B1393753FB7@cvs.postgresql.org|A partial solution for UTF-8}}<br />
}}<br />
<br />
{{TodoItem<br />
|Improve encoding of connection startup messages sent to the client<br />
|Currently some authentication error messages are sent in the server encoding<br />
* [http://archives.postgresql.org/pgsql-general/2008-12/msg00801.php <nowiki>encoding of PostgreSQL messages</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-01/msg00005.php <nowiki>Re: encoding of PostgreSQL messages</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have pg_stat_activity display query strings in the correct client encoding<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00131.php <nowiki>pg_stats queries versus per-database encodings</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|More sensible support for Unicode combining characters, normal forms<br />
* http://archives.postgresql.org/message-id/200904141532.44618.peter_e@gmx.net<br />
}}<br />
<br />
== Views / Rules ==<br />
<br />
{{TodoItem<br />
|Automatically create rules on views so they are updateable, per SQL:2008<br />
|We can only auto-create rules for simple views. For more complex cases users will still have to write rules manually.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00586.php <nowiki>Proposal for updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-08/msg00255.php <nowiki>Updatable views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg01746.php <nowiki>Re: [COMMITTERS] pgsql: Automatic view update rules Bernd Helmle</nowiki>]<br />
* http://wiki.postgresql.org/wiki/Updatable_views<br />
}}<br />
<br />
{{TodoItem<br />
|Add the functionality for WITH CHECK OPTION clause of CREATE VIEW}}<br />
<br />
{{TodoItem<br />
|Allow VIEW/RULE recompilation when the underlying tables change<br />
|This is both difficult and controversial.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01723.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg01724.php Re: About "Allow VIEW/RULE recompilation when the underlying tables change"]<br />
}}<br />
<br />
{{TodoItem<br />
|Make it possible to use RETURNING together with conditional DO INSTEAD rules, such as for partitioning setups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00577.php <nowiki>RETURNING and DO INSTEAD ... Intentional or not?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add the ability to automatically create materialized views<br />
|Right now materialized views require the user to create triggers on the main table to keep the summary table current. SQL syntax should be able to manage the triggers and summary table automatically. A more sophisticated implementation would automatically retrieve from the summary table when the main table is referenced, if possible. See [[Materialized Views]] for implementation details<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00479.php <nowiki>GSoC - proposal - Materialized Views in PostgreSQL</nowiki>] <br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to modify views via ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00691.php <nowiki>Re: idea: storing view source in system catalogs</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg01410.php <nowiki>modifying views</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg00300.php <nowiki>Re: patch: Add columns via CREATE OR REPLACE VIEW</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent low-cost functions from seeing unauthorized view rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg01346.php <nowiki>Using views for row-level access control is leaky</nowiki>]<br />
}}<br />
<br />
== SQL Commands ==<br />
<br />
{{TodoItem<br />
|Add CORRESPONDING BY to UNION/INTERSECT/EXCEPT}}<br />
<br />
{{TodoItem<br />
|Add ROLLUP, CUBE, GROUPING SETS options to GROUP BY<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00838.php <nowiki>WIP: grouping sets support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00466.php <nowiki>Implementation of GROUPING SETS (T431: Extended grouping capabilities)</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Fix TRUNCATE ... RESTART IDENTITY so its effect on sequences is rolled back on transaction abort<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00550.php <nowiki>Re: [PATCHES] TRUNCATE TABLE with IDENTITY</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow PREPARE of cursors}}<br />
<br />
{{TodoItem<br />
|Allow finer control over the caching of prepared query plans<br />
|Currently anonymous (un-named) queries prepared via the wire protocol are replanned every time bind parameters are supplied --- allow SQL PREPARE to do the same. Also, allow control over replanning prepared queries either manually or automatically when statistics for execute parameters differ dramatically from those used during planning.<br />
* http://archives.postgresql.org/message-id/201002151911.o1FJBYh22763@momjian.us<br />
}}<br />
<br />
{{TodoItem<br />
|Improve logging of prepared transactions recovered during startup<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00092.php <nowiki>&quot;recovering prepared transaction&quot; after server restart message</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow prepared transactions with temporary tables created and dropped in the same transaction, and when an ON COMMIT DELETE ROWS temporary table is accessed<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00047.php <nowiki>Re: &quot;could not open relation 1663/16384/16584: No such file or directory&quot; in a specific combination of transactions with temp tables</nowiki>]<br />
* [http://archives.postgresql.org/message-id/492543D5.9050904@enterprisedb.com A suggestion on how to implement this]<br />
}}<br />
<br />
{{TodoItem<br />
|Add a GUC variable to warn about non-standard SQL usage in queries}}<br />
<br />
{{TodoItem<br />
|Add SQL-standard MERGE/REPLACE/UPSERT command<br />
|MERGE is typically used to merge two tables. REPLACE or UPSERT command does UPDATE, or on failure, INSERT. See [[SQL MERGE]] for notes on the implementation details.<br />
}}<br />
<br />
{{TodoItem<br />
|Add NOVICE output level for helpful messages like automatic sequence/index creation}}<br />
<br />
{{TodoItem<br />
|Add GUC to issue notice about statements that use unjoined tables}}<br />
<br />
{{TodoItem<br />
|Allow EXPLAIN to identify tables that were skipped because of constraint_exclusion}}<br />
<br />
{{TodoItemDone<br />
|Enable standard_conforming_strings by default<br />
|When this is done, backslash-quote should be prohibited in non-E<nowiki>''</nowiki> strings because of possible confusion over how such strings treat backslashes. Basically, <nowiki>''</nowiki> is always safe for a literal single quote, while \' might or might not be based on the backslash handling rules.}}<br />
<br />
{{TodoItem<br />
|Simplify dropping roles that have objects in several databases}}<br />
<br />
{{TodoItem<br />
|Allow the count returned by SELECT, etc to be represented as an int64 to allow a higher range of values}}<br />
<br />
{{TodoItem<br />
|Add support for WITH RECURSIVE ... CYCLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00291.php <nowiki>WITH RECURSIVE ... CYCLE in vanilla SQL: issues with arrays of rows</nowiki>]}}<br />
<br />
{{TodoItem<br />
|Add DEFAULT .. AS OWNER so permission checks are done as the table owner<br />
|This would be useful for SERIAL nextval() calls and CHECK constraints.}}<br />
<br />
{{TodoItem<br />
|Allow DISTINCT to work in multiple-argument aggregate calls}}<br />
<br />
{{TodoItem<br />
|Add column to pg_stat_activity that shows the progress of long-running commands like CREATE INDEX and VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00203.php <nowiki>EXPLAIN progress info</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow INSERT/UPDATE/DELETE ... RETURNING inside a SELECT 'FROM' clause or target list<br />
|Actually it would be saner to allow this in WITH<br />
* [http://archives.postgresql.org/pgsql-general/2006-09/msg00803.php <nowiki>8.2: select from an INSERT returning?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00693.php <nowiki>Re: SQL functions, INSERT/UPDATE/DELETE RETURNING, and triggers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00124.php <nowiki>cannot use result of (insert..returning)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00979.php <nowiki>insert ... delete ... returning ... ?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-06/msg00357.php Using results from DELETE ... RETURNING]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow INSERT/UPDATE/DELETE ... RETURNING in common table expressions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00472.php <nowiki>Writeable CTEs and side effects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add comments on system tables/columns using the information in catalogs.sgml<br />
|Ideally the information would be pulled from the SGML file automatically.}}<br />
<br />
{{TodoItem<br />
|Prevent the specification of conflicting transaction read/write options<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00684.php <nowiki>Re: SET TRANSACTION and SQL Standard</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support LATERAL subqueries<br />
|Lateral subqueries can reference columns of tables defined outside the subquery at the same level, i.e. ''laterally''.<br />
For example, a LATERAL subquery in a FROM clause could reference tables defined in the same FROM clause.<br />
Currently only the columns of tables defined ''above'' subqueries are recognized.<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00292.php <nowiki>LATERAL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00991.php <nowiki>Re: LATERAL</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for functional dependencies<br />
|This would allow omitting GROUP BY columns when grouping by the primary key.<br />
}}<br />
<br />
{{TodoItem<br />
|Optimize ON COMMIT DELETE ROWS<br />
|Currently deletions are excessively performed<br />
* http://archives.postgresql.org/pgsql-performance/2010-03/msg00392.php<br />
* http://archives.postgresql.org/pgsql-performance/2010-04/msg00046.php<br />
}}<br />
<br />
=== CREATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow CREATE TABLE AS to determine column lengths for complex expressions like SELECT col1 || col2}}<br />
<br />
{{TodoItem<br />
|Have WITH CONSTRAINTS also create constraint indexes<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00149.php <nowiki>Re: CREATE TABLE LIKE INCLUDING INDEXES support</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move NOT NULL constraint information to pg_constraint<br />
|Currently NOT NULL constraints are stored in pg_attribute without any designation of their origins, e.g. primary keys. One manifest problem is that dropping a PRIMARY KEY constraint does not remove the NOT NULL constraint designation. Another issue is that we should probably force NOT NULL to be propagated from parent tables to children, just as CHECK constraints are. (But then does dropping PRIMARY KEY affect children?)<br />
* http://archives.postgresql.org/message-id/19768.1238680878@sss.pgh.pa.us<br />
* http://archives.postgresql.org/message-id/200909181005.n8IA5Ris061239@wwwmaster.postgresql.org<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent concurrent CREATE TABLE from sometimes returning a cryptic error message<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00169.php <nowiki>BUG #3692: Conflicting create table statements throw unexpected error</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Allow CREATE TABLE to optionally create a table if it does not already exist, without throwing an error<br />
|The fact that tables contain data makes this more complex than other CREATE OR REPLACE operations.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01300.php <nowiki>Add column if not exists (CINE)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add CREATE SCHEMA ... LIKE that copies a schema}}<br />
<br />
{{TodoItem<br />
|CREATE OR REPLACE FUNCTION might leave dependent objects depending on the function in inconsistent state<br />
* [http://archives.postgresql.org/pgsql-general/2008-08/msg00985.php indexes on functions and create or replace function]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow temporary tables to exist as empty by default in all sessions<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00006.php <nowiki>what is difference between LOCAL and GLOBAL TEMP TABLES in PostgreSQL</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg01329.php <nowiki>idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-05/msg00016.php <nowiki>Re: idea: global temp tables</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg01098.php <nowiki>global temporary tables</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow the creation of "distinct" types<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01647.php <nowiki>Distinct types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider analyzing temporary tables when they are first used in a query<br />
|Autovacuum cannot analyze or vacuum temporary tables.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-04/msg00416.php <nowiki>autovacuum and temp tables support</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== UPDATE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow UPDATE tab SET ROW (col, ...) = (SELECT...)</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg01308.php <nowiki>Re: [PATCHES] extension for sql update</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00865.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-04/msg00315.php <nowiki>UPDATE using sub selects</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-03/msg00237.php <nowiki>Re: UPDATE using sub selects</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Research self-referential UPDATEs that see inconsistent row versions in read-committed mode<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00507.php <nowiki>Concurrently updating an updatable view</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00016.php <nowiki>Re: Do we need a TODO? (was Re: Concurrently updating anupdatable view)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve performance of EvalPlanQual mechanism that rechecks already-updated rows<br />
|This is related to the previous item, which questions whether it even has the right semantics<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-09/msg00045.php <nowiki>BUG #4401: concurrent updates to a table blocks one update indefinitely</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-07/msg00302.php <nowiki>BUG #4945: Parallel update(s) gone wild</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ALTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have ALTER TABLE RENAME rename SERIAL sequence names<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Allow ALTER TABLE ... ALTER CONSTRAINT ... RENAME<br />
* [http://archives.postgresql.org/pgsql-patches/2006-02/msg00168.php <nowiki>ALTER CONSTRAINT RENAME patch reverted</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ALTER TABLE RENAME CONSTRAINT}}<br />
<br />
{{TodoItem<br />
|Have ALTER SEQUENCE RENAME rename the sequence name stored in the sequence table<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-09/msg00092.php <nowiki>BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-10/msg00007.php <nowiki>Re: BUG #3619: Renaming sequence does not update its 'sequence_name' field</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00008.php <nowiki>Re: newbie: renaming sequences task</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ALTER DOMAIN to modify the underlying data type}}<br />
<br />
{{TodoItemEasy<br />
|Allow ALTER TABLE to change constraint deferrability and actions}}<br />
<br />
{{TodoItem<br />
|Add missing object types for ALTER ... SET SCHEMA}}<br />
<br />
{{TodoItem<br />
|Allow ALTER TABLESPACE to move to different directories}}<br />
<br />
{{TodoItem<br />
|Allow moving system tables to other tablespaces, where possible<br />
|Currently non-global system tables must be in the default database tablespace. Global system tables can never be moved.}}<br />
<br />
{{TodoItem<br />
|Have ALTER INDEX update the name of a constraint using that index}}<br />
<br />
{{TodoItem<br />
|Allow column display reordering by recording a display, storage, and permanent id for every column?<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00782.php <nowiki>Re: column ordering, was Re: [PATCHES] Enums patch v2</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01029.php <nowiki>Column reordering in pg_dump</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow an existing index to be marked as a table's primary key<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00500.php <nowiki>Setting a pre-existing index as a primary key</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow ALTER TYPE on composite types to perform operations similar to ALTER TABLE<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00245.php <nowiki>ALTER composite type does not work, but ALTER TABLE which ROWTYPE is used as a type - works fine</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Don't require table rewrite on ALTER TABLE ... ALTER COLUMN TYPE, when the old and new data types are binary compatible<br />
* http://archives.postgresql.org/message-id/200903040137.n241bAUV035002@wwwmaster.postgresql.org<br />
* [http://archives.postgresql.org/pgsql-patches/2006-10/msg00154.php <nowiki>Eliminating phase 3 requirement for varlen increases via ALTER COLUMN</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Reduce locking required for ALTER commands<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00533.php <nowiki>ALTER TABLE SET STATISTICS requires AccessExclusiveLock</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg01083.php <nowiki>Re: ALTER TABLE SET STATISTICS requires AccessExclusiveLock</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2008-10/msg01248.php<br />
* http://archives.postgresql.org/pgsql-hackers/2008-10/msg00242.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== CLUSTER ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Automatically maintain clustering on a table<br />
|This might require some background daemon to maintain clustering during periods of low usage. It might also require tables to be only partially filled for easier reorganization. Another idea would be to create a merged heap/index data file so an index lookup would automatically access the heap data too. A third idea would be to store heap rows in hashed groups, perhaps using a user-supplied hash function.<br />
* [http://archives.postgresql.org/pgsql-performance/2004-08/msg00350.php <nowiki>Equivalent praxis to CLUSTERED INDEX?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00155.php <nowiki>Re: Grouped Index Tuples</nowiki>]<br />
* http://community.enterprisedb.com/git/<br />
* [http://archives.postgresql.org/pgsql-performance/2009-10/msg00346.php <nowiki>Re: maintain_cluster_order_v5.patch</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Add default clustering to system tables<br />
|To do this, determine the ideal cluster index for each system table and set the cluster setting during initdb.<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-05/msg00989.php <nowiki>Re: Clustering system catalog indexes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve CLUSTER performance by sorting to reduce random I/O<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01371.php <nowiki>Our CLUSTER implementation is pessimal</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Make CLUSTER VERBOSE more verbose.<br />
|It is also used by new VACUUM FULL VERBOSE.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== COPY ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report error lines and continue<br />
|This requires the use of a savepoint before each COPY line is processed, with ROLLBACK on COPY failure. <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00572.php <nowiki>Re: VLDB Features</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY on a newly-created table to skip WAL logging<br />
|On crash recovery, the table involved in the COPY would be removed or have its heap and index files truncated. One issue is that no other backend should be able to add to the table at the same time, which is something that is currently allowed. This currently is done if the table is created inside the same transaction block as the COPY because no other backends can see the table.}}<br />
<br />
{{TodoItem<br />
|Allow COPY FROM to create index entries in bulk<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00811.php <nowiki>Batch update of indexes on data loading</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY in CSV mode to control whether a quoted zero-length string is treated as NULL<br />
|Currently this is always treated as a zero-length string, which generates an error when loading into an integer column <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00905.php <nowiki>Re: [PATCHES] allow CSV quote in NULL</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve COPY performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00954.php <nowiki>Re: 8.3 / 8.2.6 restore comparison</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to report errors sooner<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01169.php <nowiki>Timely reporting of COPY errors</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow COPY to handle other number formats eg. the German notation. Best would be something like WITH DECIMAL ','.<br />
}}<br />
<br />
{{TodoItem<br />
|Allow a stalled COPY to exit if the backend is terminated<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-04/msg00067.php <nowiki>Re: possible bug not in open items</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GRANT/REVOKE ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow SERIAL sequences to inherit permissions from the base table?}}<br />
<br />
{{TodoItem<br />
|Allow dropping of a role that has connection rights<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00736.php <nowiki>DROP ROLE dependency tracking ...</nowiki>]<br />
}}<br />
{{TodoEndSubsection}}<br />
<br />
=== DECLARE CURSOR ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent DROP TABLE from dropping a table referenced by its own open cursor?}}<br />
<br />
{{TodoItem<br />
|Provide some guarantees about the behavior of cursors that invoke volatile functions<br />
* [http://archives.postgresql.org/message-id/20997.1244563664@sss.pgh.pa.us Re: Cursor with hold emits the same row more than once across commits in 8.3.7]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== INSERT ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow INSERT/UPDATE of the system-generated oid value for a row}}<br />
<br />
{{TodoItem<br />
|In rules, allow VALUES() to contain a mixture of 'old' and 'new' references}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== SHOW/SET ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM ANALYZE, and CLUSTER}}<br />
<br />
{{TodoItem<br />
|Rationalize the discrepancy between settings that use values in bytes and SHOW that returns the object count<br />
* [http://archives.postgresql.org/pgsql-docs/2008-07/msg00007.php <nowiki>Re: [ADMIN] shared_buffers and shmmax</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== LISTEN/NOTIFY ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow NOTIFY in rules involving conditionals}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Window Functions ===<br />
See {{messageLink|357.1230492361@sss.pgh.pa.us|TODO items for window functions}}.<br />
{{TodoSubsection}}<br />
{{TodoItem<br />
|Support creation of user-defined window functions.<br />
|We have the ability to create new window functions written in C. Is it<br />
worth the effort to create an API that would let them be written in PL/pgsql, etc?}}<br />
<br />
{{TodoItem<br />
|Implement full support for window framing clauses.<br />
|In addition to done clauses described in the [http://developer.postgresql.org/pgdocs/postgres/sql-expressions.html#SYNTAX-WINDOW-FUNCTIONS latest doc], these clauses are not implemented yet.<br />
* RANGE BETWEEN ... PRECEDING/FOLLOWING<br />
* EXCLUDE<br />
}}<br />
<br />
{{TodoItem<br />
|Look at tuplestore performance issues.<br />
|The tuplestore_in_memory() thing is just a band-aid, we ought to try to solve it properly. tuplestore_advance seems like a weak spot as well.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00152.php <nowiki>tuplestore potential performance problem</nowiki>]<br />
}}<br />
<br />
{{TodoItem|Do we really need so much duplicated code between Agg and WindowAgg?}}<br />
<br />
{{TodoItem<br />
|Teach planner to evaluate multiple windows in the optimal order.<br />
|Currently windows are always evaluated in the query-specified order.<br />
* http://archives.postgresql.org/message-id/3CDAD71E9D70417290FCF66F0178D1E1@amd64<br />
}}<br />
<br />
{{TodoItem<br />
|Implement DISTINCT clause in window aggregates.<br />
|Some proprietary RDBMSs have implemented it already, so it helps with porting from those.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Integrity Constraints ==<br />
=== Keys ===<br />
<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Improve deferrable unique constraints for cases with many conflicts<br />
|The current implementation fires a trigger for each potentially conflicting row. This might not scale well for an update that changes many key values at once.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Referential Integrity ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add MATCH PARTIAL referential integrity}}<br />
<br />
{{TodoItem<br />
|Change foreign key constraint for array -&gt; element to mean element in array?}}<br />
<br />
{{TodoItem<br />
|Fix problem when cascading referential triggers make changes on cascaded tables, seeing the tables in an intermediate state<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-09/msg00174.php <nowiki>Re: [PATCHES] Work-in-progress referential action trigger timing</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Optimize referential integrity checks<br />
* [http://archives.postgresql.org/pgsql-performance/2005-10/msg00458.php <nowiki>Re: Effects of cascading references in foreign keys</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00744.php <nowiki>Can't ri_KeysEqual() consider two nulls as equal?</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Server-Side Languages ==<br />
<br />
{{TodoItem<br />
|Add support for polymorphic arguments and return types to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add support for OUT and INOUT parameters to languages other than PL/PgSQL}}<br />
<br />
{{TodoItem<br />
|Add more fine-grained specification of functions taking arbitrary data types<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00367.php <nowiki>RfD: more powerful &quot;any&quot; types</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement stored procedures<br />
|This might involve the control of transaction state and the return of multiple result sets<br />
* [http://archives.postgresql.org/pgsql-general/2008-10/msg00454.php <nowiki>PL/pgSQL stored procedure returning multiple result sets (SELECTs)?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01375.php <nowiki>Proposal: real procedures again (8.4)</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00542.php<br />
}}<br />
<br />
=== PL/pgSQL ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow handling of %TYPE arrays, e.g. tab.col%TYPE[]}}<br />
<br />
{{TodoItem<br />
|<nowiki>Allow listing of record column names, and access to record columns via variables, e.g. columns := r.(*), tval2 := r.(colname)</nowiki><br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00458.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00302.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00031.php <nowiki>Re: PL/PGSQL: Dynamic Record Introspection</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for SCROLL cursors}}<br />
<br />
{{TodoItem<br />
|Add support for WITH HOLD cursors}}<br />
<br />
{{TodoItem<br />
|Allow row and record variables to be set to NULL constants, and allow NULL tests on such variables<br />
|Because a row is not scalar, do not allow assignment from NULL-valued scalars.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00070.php <nowiki>NULL and plpgsql rows</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider keeping separate cached copies when search_path changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01009.php <nowiki>pl/pgsql Plan Invalidation and search_path</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve handling of NULL row values vs. NULL rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg01758.php <nowiki>Null row vs. row of nulls in plpgsql</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Perl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow regex operations in plperl using UTF8 characters in non-UTF8 encoded databases.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Python ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add table function support}}<br />
<br />
{{TodoItem<br />
|Add tracebacks<br />
* [http://archives.postgresql.org/pgsql-patches/2006-02/msg00288.php <nowiki>Re: plpython tracebacks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Develop a trusted variant of PL/Python.}}<br />
<br />
{{TodoItem<br />
|Create a new restricted execution class that will allow passing function arguments in as locals. Passing them as globals means functions cannot be called recursively.}}<br />
<br />
{{TodoItem<br />
|Functions cache the input and output functions for their arguments, so the following will make PostgreSQL unhappy:<br />
<br />
create table users (first_name text, last_name text);<br />
create function user_name(user) returns text as 'mycode' language plpython;<br />
select user_name(user) from users;<br />
alter table add column user_id integer;<br />
select user_name(user) from users;<br />
<br />
You have to drop and create the function(s) each time its arguments<br />
are modified (not nice), or don't cache the input and output functions<br />
(slower?), or check if the structure of the argument has been<br />
altered (is this possible, easy, quick?) and recreate cache.}}<br />
<br />
{{TodoItem<br />
|Better documentation}}<br />
<br />
{{TodoItem<br />
|Add a DB-API compliant interface on top of the SPI interface.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== PL/Tcl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add table function support}}<br />
<br />
{{TodoItem<br />
|check encoding validity of values passed back to Postgres in function returns, trigger tuple changes, or SPI calls.}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Clients ==<br />
<br />
{{TodoItem<br />
|Add a function like pg_get_indexdef() that report more detailed index information<br />
* [http://archives.postgresql.org/pgsql-bugs/2007-12/msg00166.php <nowiki>BUG #3829: Wrong index reporting from pgAdmin III (v1.8.0 rev 6766-6767)</nowiki>]<br />
}}<br />
<br />
=== pg_ctl ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow pg_ctl to work properly with configuration files located outside the PGDATA directory<br />
|pg_ctl can not read the pid file because it isn't located in the config directory but in the PGDATA directory. The solution is to allow pg_ctl to read and understand postgresql.conf to find the data_directory value.<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-10/msg00024.php <nowiki>BUG #5103: &quot;pg_ctl -w (re)start&quot; fails with custom unix_socket_directory</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have the postmaster write a random number to a file on startup that pg_ctl checks against the contents of a pg_ping response on its initial connection (without login)<br />
|This will protect against connecting to an old instance of the postmaster in a different or deleted subdirectory.<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-10/msg00110.php <nowiki>Re: BUG #5118: start-status-insert-fatal</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-10/msg00156.php <nowiki>Re: BUG #5118: start-status-insert-fatal</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Modify pg_ctl behavior and exit codes to make it easier to write an LSB conforming init script<br />
|It may be desirable to condition some of the changes on a command-line switch, to avoid breaking existing scripts. A Linux shell (sh) script is referenced which has been tested and seems to provide a high degree of conformance in multiple environments. Study of this script might suggest areas where pg_ctl could be modified to make writing an LSB conforming script easier; however, some aspects of that script would be unnecessary with other suggested changes to pg_ctl, and discussion on the lists did not reach consensus on support for all aspects of this script. Further discussion of particular changes is needed before beginning any work.<br />
* [[Lsb_conforming_init_script|LSB conforming init script]]<br />
These threads should be studied for other ideas on improvements:<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01390.php <nowiki>We should Axe /contrib/start-scripts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01843.php <nowiki>Linux LSB init script</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00008.php <nowiki>Re: Linux LSB init script</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== psql ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Have psql \ds show all sequences and their settings<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00916.php <nowiki>Re: TODO item: Have psql show current values for a sequence</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00401.php <nowiki>Quick patch: Display sequence owner</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have \d on a sequence indicate if the sequences is owned by a table}}<br />
<br />
{{TodoItem<br />
|Move psql backslash database information into the backend, use mnemonic commands?<br />
|This would allow non-psql clients to pull the same information out of the database as psql. <br />
* [http://archives.postgresql.org/pgsql-hackers/2004-01/msg00191.php <nowiki>Re: psql \d option list overloaded</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Make psql's \d commands more consistent in its handling of schemas<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00014.php <nowiki>Re: psql and schemas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consistently display privilege information for all objects in psql}}<br />
<br />
{{TodoItem<br />
|Add auto-expanded mode so expanded output is used if the row length is wider than the screen width.<br />
|Consider using auto-expanded mode for backslash commands like \df+.}}<br />
<br />
{{TodoItem<br />
|Prevent tab completion of SET TRANSACTION from querying the database and therefore preventing the transaction isolation level from being set.<br />
|Currently SET &lt;tab&gt; causes a database lookup to check all supported session variables. This query causes problems because setting the transaction isolation level must be the first statement of a transaction.}}<br />
<br />
{{TodoItem<br />
|Add a \set variable to control whether \s displays line numbers<br />
|Another option is to add \# which lists line numbers, and allows command execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00255.php <nowiki>Re: psql possible TODO</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent escape string warnings when object names have backslashes<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00227.php <nowiki>Psql command-line completion bug</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have \d show child tables that inherit from the specified parent}}<br />
<br />
{{TodoItem<br />
|Include the symbolic SQLSTATE name in verbose error reports<br />
* [http://archives.postgresql.org/pgsql-general/2007-09/msg00438.php <nowiki>Re: Checking is TSearch2 query is valid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add prompt escape to display the client and server versions<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00310.php <nowiki>WIP patch for TODO Item: Add prompt escape to display the client and server versions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to wrap column values at whitespace boundaries, rather than chopping them at a fixed width.<br />
|Currently, &quot;wrapped&quot; format chops values into fixed widths. Perhaps the word wrapping could use the same algorithm documented in the W3C specification. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00404.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
* http://www.w3.org/TR/CSS21/tables.html#auto-table-layout}}<br />
<br />
{{TodoItem<br />
|Add &quot;auto&quot; expanded mode that outputs in expanded format if &quot;wrapped&quot; mode can't wrap the output to the screen width<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00417.php <nowiki>Re: psql wrapped format default for backslash-d commands</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support the ReST table output format<br />
|Details about the ReST format: http://docutils.sourceforge.net/rst.html#reference-documentation<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-08/msg01007.php <nowiki>Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00518.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-01/msg00609.php <nowiki>Re: Proposal: new border setting in psql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add option to print advice for people familiar with other databases<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-01/msg01845.php <nowiki>MySQL-ism help patch for psql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider showing TOAST and index sizes in \dt+<br />
* [http://archives.postgresql.org/pgsql-general/2010-01/msg00912.php <nowiki>\dt+ sizes don't include TOAST data</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow \dd to show constraint comments<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00436.php <nowiki>Re: More robust pg_hba.conf parsing/error logging</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-general/2009-09/msg00199.php <nowiki>comment on constraint</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add ability to edit views with \ev<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg00023.php <nowiki>Adding \ev view editor?</nowiki>]<br />
}}<br />
{{TodoItem<br />
|Add \dL to show languages<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-07/msg00915.php <nowiki>Re: [PATCH] Psql List Languages</nowiki>]<br />
}}<br />
<br />
{{TodoItemDone<br />
|Distinguish between unique indexes and unique constraints in \d+<br />
* http://archives.postgresql.org/message-id/8780.1271187360@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Fix FETCH_COUNT to handle SELECT ... INTO and WITH queries<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01565.php<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00192.php<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent psql from sending remaining single-line multi-statement queries after reconnecting<br />
* http://archives.postgresql.org/pgsql-bugs/2010-05/msg00159.php<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01283.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== pg_dump / pg_restore ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|<nowiki>Add full object name to the tag field. eg. for operators we need '=(integer, integer)', instead of just '='.</nowiki>}}<br />
<br />
{{TodoItem<br />
|Add pg_dumpall custom format dumps?<br />
* [http://archives.postgresql.org/pgsql-general/2010-05/msg00509.php pg_dumpall custom format]<br />
|}}<br />
<br />
{{TodoItem<br />
|Avoid using platform-dependent locale names in pg_dumpall output<br />
|Using native locale names puts roadblocks in the way of porting a dump to another platform. One possible solution is to get<br />
CREATE DATABASE to accept some agreed-on set of locale names and fix them up to meet the platform's requirements.<br />
* http://archives.postgresql.org/message-id/21396.1241716688@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Allow selection of individual object(s) of all types, not just tables}}<br />
<br />
{{TodoItem<br />
|In a selective dump, allow dumping of an object and all its dependencies}}<br />
<br />
{{TodoItem<br />
|Add options like pg_restore -l and -L to pg_dump}}<br />
<br />
{{TodoItem<br />
|Add support for multiple pg_restore -t options, like pg_dump<br />
|pg_restore's -t switch is less useful than pg_dump's in quite a few ways: no multiple switches, no pattern matching, no ability to pick up indexes and other dependent items for a selected table. It should be made to handle this switch just like pg_dump does.}}<br />
<br />
{{TodoItem<br />
|Stop dumping CASCADE on DROP TYPE commands in clean mode}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump --clean to drop roles that own objects or have privileges<br />
|tgl says: if this is about pg_dumpall, it's done as of 8.4. If it's really about pg_dump, what does it mean? pg_dump has no business dropping roles.}}<br />
<br />
{{TodoItem<br />
|Remove unnecessary function pointer abstractions in pg_dump source code}}<br />
<br />
{{TodoItem<br />
|Allow pg_dump to utilize multiple CPUs and I/O channels by dumping multiple objects simultaneously<br />
|The difficulty with this is getting multiple dump processes to produce a single dump output file. It also would require several sessions to share the same snapshot. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php <nowiki>pg_dump additional options for performance</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow pg_restore to load different parts of the COPY data for a single table simultaneously}}<br />
<br />
{{TodoItem<br />
|Remove support for dumping from pre-7.3 servers<br />
|In 7.3 and later, we can get accurate dependency information from the server. pg_dump still contains a lot of crufty code<br />
to try to deal with the lack of dependency info in older servers, but the usefulness of maintaining that code grows small.}}<br />
<br />
{{TodoItem<br />
|Allow pre/data/post files when schema and data are dumped separately, for performance reasons<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php <nowiki>pg_dump additional options for performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00185.php <nowiki>Re: pg_dump additional options for performance</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Refactor handling of database attributes between pg_dump and pg_dumpall<br />
|Currently only pg_dumpall emits database attributes, such as ALTER DATABASE SET commands and database-level GRANTs.<br />
Many people wish that pg_dump would do that. One proposal is to let pg_dump issue such commands if the -C switch was used,<br />
but it's unclear whether that will satisfy the demand.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01031.php <nowiki>ALTER DATABASE vs pg_dump</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-bugs/2010-05/msg00010.php summary of the issues]<br />
}}<br />
<br />
{{TodoItem<br />
|Change pg_dump so that a comment on the dumped database is applied to the loaded database, even if the database has a different name.<br />
|This will require new backend syntax, perhaps COMMENT ON CURRENT DATABASE. This is related to the previous item.}}<br />
<br />
{{TodoItem<br />
|Allow parallel restore of tar dumps<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-02/msg01154.php <nowiki>Re: parallel restore</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== ecpg ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Docs<br />
|Document differences between ecpg and the SQL standard and information about the Informix-compatibility module.}}<br />
<br />
{{TodoItem<br />
|Solve cardinality &gt; 1 for input descriptors / variables?}}<br />
<br />
{{TodoItem<br />
|Add a semantic check level, e.g. check if a table really exists}}<br />
<br />
{{TodoItem<br />
|fix handling of DB attributes that are arrays}}<br />
<br />
{{TodoItem<br />
|Fix nested C comments}}<br />
<br />
{{TodoItemEasy<br />
|sqlwarn[6] should be 'W' if the PRECISION or SCALE value specified}}<br />
<br />
{{TodoItem<br />
|Make SET CONNECTION thread-aware, non-standard?}}<br />
<br />
{{TodoItem<br />
|Allow multidimensional arrays}}<br />
<br />
{{TodoItem<br />
|Implement COPY FROM STDIN}} <br />
<br />
{{TodoItem<br />
|Provide a way to specify size of a bytea parameter<br />
* [http://archives.postgresql.org/message-id/200906192131.n5JLVoMo044178@wwwmaster.postgresql.org <nowiki>BUG #4866: ECPG and BYTEA</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
|Fix small memory leaks in ecpg<br />
|Memory leaks in a short running application like ecpg are not really a problem, but make debugging more complicated}} <br />
<br />
{{TodoItem<br />
|Allow re-usage of cursor name variables<br />
* [http://archives.postgresql.org/message-id/20100329113435.GA3430@feivel.credativ.lan <nowiki>Problems with variable cursorname in ecpg</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== libpq ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Prevent PQfnumber() from lowercasing unquoted column names<br />
|PQfnumber() should never have been doing lowercasing, but historically it has so we need a way to prevent it}}<br />
<br />
{{TodoItem<br />
|Allow statement results to be automatically batched to the client<br />
|Currently all statement results are transferred to the libpq client before libpq makes the results available to the application. This feature would allow the application to make use of the first result rows while the rest are transferred, or held on the server waiting for them to be requested by libpq. One complexity is that a statement like SELECT 1/col could error out mid-way through the result set.}}<br />
<br />
{{TodoItem<br />
|Consider disallowing multiple queries in PQexec() as an additional barrier to SQL injection attacks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00184.php <nowiki>Re: InitPostgres and flatfiles question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add PQexecf() that allows complex parameter substitution<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01803.php <nowiki>Last minute mini-proposal (I know, know) for PQexecf()</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add SQLSTATE and severity to errors generated within libpq itself<br />
* [http://archives.postgresql.org/pgsql-interfaces/2007-11/msg00015.php <nowiki>v8.1: Error severity on libpq PGconn*</nowiki>]<br />
* http://archives.postgresql.org/pgsql-hackers/2010-08/msg01425.php<br />
}}<br />
<br />
{{TodoItem<br />
|Add code to detect client encoding and locale from the operating system environment<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01040.php <nowiki>Determining client_encoding from client locale</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add support for interface/ipaddress binding to libpq<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01811.php <nowiki>SR/libpq - outbound interface/ipaddress binding</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Triggers ==<br />
<br />
{{TodoItem<br />
|Improve storage of deferred trigger queue<br />
|Right now all deferred trigger information is stored in backend memory. This could exhaust memory for very large trigger queues. This item involves dumping large queues into files, or doing some kind of join to process all the triggers, some bulk operation, or a bitmap. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00876.php <nowiki>Re: BUG #4204: COPY to table with FK has memory leak</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-10/msg00464.php <nowiki>Scaling up deferred unique checks and the after trigger queue</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow triggers to be disabled in only the current session.<br />
|This is currently possible by starting a multi-statement transaction, modifying the system tables, performing the desired SQL, restoring the system tables, and committing the transaction. ALTER TABLE ... TRIGGER requires a table lock so it is not ideal for this usage.}}<br />
<br />
{{TodoItem<br />
|With disabled triggers, allow pg_dump to use ALTER TABLE ADD FOREIGN KEY<br />
|If the dump is known to be valid, allow foreign keys to be added without revalidating the data.}}<br />
<br />
{{TodoItem<br />
|Allow statement-level triggers to access modified rows}}<br />
<br />
{{TodoItem<br />
|When statement-level triggers are defined on a parent table, have them fire only on the parent table, and fire child table triggers only where appropriate<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg01883.php <nowiki>Statement-level triggers and inheritance</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow AFTER triggers on system tables<br />
|System tables are modified in many places in the backend without going through the executor and therefore not causing triggers to fire. To complete this item, the functions that modify system tables will have to fire triggers.}}<br />
<br />
{{TodoItem<br />
|Tighten trigger permission checks<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00564.php <nowiki>Security leak with trigger functions?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow BEFORE INSERT triggers on views<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg01466.php <nowiki>Re: Why can't I put a BEFORE EACH ROW trigger on a view?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add database and transaction-level triggers<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00451.php <nowiki>Proposal for db level triggers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00620.php <nowiki>triggers on prepare, commit, rollback... ?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce locking requirements for creating a trigger<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg00635.php <nowiki>Re: Change lock requirements for adding a trigger</nowiki>]<br />
}}<br />
<br />
== Inheritance ==<br />
<br />
{{TodoItem<br />
|Allow inherited tables to inherit indexes, UNIQUE constraints, and primary/foreign keys<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-05/msg00285.php <nowiki>Partitioning/inherited tables vs FKs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Honor UNIQUE INDEX on base column in INSERTs/UPDATEs on inherited table, e.g. INSERT INTO inherit_table (unique_index_col) VALUES (dup) should fail<br />
|The main difficulty with this item is the problem of creating an index that can span multiple tables.}}<br />
<br />
{{TodoItem<br />
|Determine whether ALTER TABLE / SET SCHEMA should work on inheritance hierarchies (and thus support ONLY). If yes, implement it.}}<br />
<br />
{{TodoItem<br />
|ALTER TABLE variants sometimes support recursion and sometimes not, but this is poorly/not documented, and the ONLY marker would then be silently ignored. Clarify the documentation, and reject ONLY if it is not supported.}}<br />
<br />
== Indexes ==<br />
<br />
{{TodoItem<br />
|Prevent index uniqueness checks when UPDATE does not modify the column<br />
|Uniqueness (index) checks are done when updating a column even if the column is not modified by the UPDATE.<br />
However, HOT already short-circuits this in common cases, so more work might not be helpful.}}<br />
<br />
{{TodoItem<br />
|Allow the creation of on-disk bitmap indexes which can be quickly combined with other bitmap indexes<br />
|Such indexes could be more compact if there are only a few distinct values. Such indexes can also be compressed. Keeping such indexes updated can be costly.<br />
* [http://archives.postgresql.org/pgsql-patches/2005-07/msg00512.php <nowiki>Re: Bitmap index AM</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01107.php <nowiki>Bitmap index thoughts</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00265.php <nowiki>Stream bitmaps</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01214.php <nowiki>Re: Bitmapscan changes - Requesting further feedback</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00013.php <nowiki>Updated bitmap index patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00741.php <nowiki>Reviewing new index types (was Re: [PATCHES] Updated bitmap indexpatch)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01023.php <nowiki>Bitmap Indexes: request for feedback</nowiki>]<br />
* http://archives.postgresql.org/message-id/800923.27831.qm@web29010.mail.ird.yahoo.com <br />
}}<br />
<br />
{{TodoItem<br />
|Allow accurate statistics to be collected on indexes with more than one column or expression indexes, perhaps using per-index statistics<br />
* [http://archives.postgresql.org/pgsql-performance/2006-10/msg00222.php <nowiki>Re: Simple join optimized badly?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01131.php <nowiki>Stats for multi-column indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00741.php <nowiki>Cross-column statistics revisited</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg01431.php <nowiki>Multi-Dimensional Histograms</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having a larger statistics target for indexed columns and expression indexes. <br />
}}<br />
<br />
{{TodoItem<br />
|Consider smaller indexes that record a range of values per heap page, rather than having one index entry for every heap row<br />
|This is useful if the heap is clustered by the indexed values. <br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00341.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg01264.php <nowiki>Grouped Index Tuples</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg00465.php <nowiki>Grouped Index Tuples / Clustered Indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-03/msg00163.php <nowiki>Bitmapscan changes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00014.php <nowiki>Re: GIT patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00487.php <nowiki>Re: Index Tuple Compression Approach?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01589.php <nowiki>Re: Index AM change proposals, redux</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add REINDEX CONCURRENTLY, like CREATE INDEX CONCURRENTLY<br />
|This is difficult because you must upgrade to an exclusive table lock to replace the existing index file. CREATE INDEX CONCURRENTLY does not have this complication. This would allow index compaction without downtime. <br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00289.php <nowiki>Re: When/if to Reindex</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow multiple indexes to be created concurrently, ideally via a single heap scan<br />
|pg_restore allows parallel index builds, but it is done via subprocesses, and there is no SQL interface for this.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider sorting entries before inserting into btree index<br />
* [http://archives.postgresql.org/pgsql-general/2008-01/msg01010.php <nowiki>Re: ATTN: Clodaldo was Performance problem. Could it be related to 8.3-beta4?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow index scans to return matching index keys, not just the matching heap locations<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg01657.php <nowiki>Re: Is this TODO item done?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01477.php <nowiki>Index-only quals</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow creation of an index that can do comparisons to test if a value is between two column values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00757.php <nowiki>Proposal: temporal extension &quot;period&quot; data type</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using "effective_io_concurrency" for index scans<br />
* Currently only bitmap scans use this, which might be fine because most multi-row index scans use bitmap scans.<br />
}}<br />
<br />
=== GIST ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add more GIST index support for geometric data types}}<br />
<br />
{{TodoItem<br />
|Allow GIST indexes to create certain complex index types, like digital trees (see Aoki)}}<br />
<br />
{{TodoItem<br />
|Fix performance issues in contrib/seg and contrib/cube GiST support<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904161633160.4053@aragorn.flymine.org GiST index performance]<br />
* [http://archives.postgresql.org/message-id/alpine.DEB.2.00.0904221704470.22330@aragorn.flymine.org draft patch]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-05/msg00069.php <nowiki>Re: GiST index performance</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2009-06/msg00068.php <nowiki>GiST index performance</nowiki>]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== GIN ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Support empty indexed values (such as zero-element arrays) properly<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00237.php contrib/intarray vs empty arrays]<br />
* [http://archives.postgresql.org/pgsql-bugs/2009-05/msg00118.php BUG #4806: Bug with GiST index and empty integer array]<br />
}}<br />
<br />
{{TodoItem<br />
|Behave correctly for cases where some elements of an indexed value are NULL<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-03/msg01003.php <nowiki>GIN versus zero-key queries</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support queries that require a full scan<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg00402.php Issue report]<br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01132.php Older issue report]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg01581.php Original discussion of issue and proposed resolution]<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Hash ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Add UNIQUE capability to hash indexes}}<br />
<br />
{{TodoItem<br />
|Add hash WAL logging for crash recovery}}<br />
<br />
{{TodoItem<br />
|Allow multi-column hash indexes}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Sorting ==<br />
<br />
{{TodoItem<br />
|Consider whether duplicate keys should be sorted by block/offset<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00558.php <nowiki>Remove hacks for old bad qsort() implementations?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider being smarter about memory and external files used during sorts<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg01101.php <nowiki>Sorting Improvements for 8.4</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00045.php <nowiki>Re: Sorting Improvements for 8.4</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider detoasting keys before sorting}}<br />
<br />
== Fsync ==<br />
<br />
{{TodoItem<br />
|Determine optimal fdatasync/fsync, O_SYNC/O_DSYNC options<br />
|Ideally this requires a separate test program that can be run at initdb time or optionally later. Consider O_SYNC when O_DIRECT exists.}}<br />
<br />
{{TodoItem<br />
|Add program to test if fsync has a delay compared to non-fsync}}<br />
<br />
{{TodoItem<br />
|Consider sorting writes during checkpoint<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php <nowiki>Sorted writes in checkpoint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00050.php <nowiki>Re: Sorting writes during checkpoint</nowiki>]<br />
}}<br />
<br />
== Cache Usage ==<br />
<br />
{{TodoItem<br />
|Speed up COUNT(*)<br />
|We could use a fixed row count and a +/- count to follow MVCC visibility rules, or a single cached value could be used and invalidated if anyone modifies the table. Another idea is to get a count directly from a unique index, but for this to be faster than a sequential scan it must avoid access to the heap to obtain tuple visibility information.}}<br />
<br />
{{TodoItem<br />
|Provide a way to calculate an &quot;estimated COUNT(*)&quot;<br />
|Perhaps by using the optimizer's cardinality estimates or random sampling.<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-11/msg00943.php <nowiki>Re: Improving count(*)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow data to be pulled directly from indexes<br />
|Currently indexes do not have enough tuple visibility information to allow data to be pulled from the index without also accessing the heap. One way to allow this is to set a bit on index tuples to indicate if a tuple is currently visible to all transactions when the first valid heap lookup happens. This bit would have to be cleared when a heap tuple is expired.<br />
Another idea is to maintain a bitmap of heap pages where all rows are visible to all backends, and allow index lookups to reference that bitmap to avoid heap lookups, perhaps the same bitmap we might add someday to determine which heap pages need vacuuming. Frequently accessed bitmaps would have to be stored in shared memory. One 8k page of bitmaps could track 512MB of heap pages.<br />
A third idea would be for a heap scan to check if all rows are visible and if so set a per-table flag which can be checked by index scans. Any change to the table would have to clear the flag. To detect changes during the heap scan a counter could be set at the start and checked at the end --- if it is the same, the table has not been modified --- any table change would increment the counter.<br />
* [http://archives.postgresql.org/pgsql-patches/2007-10/msg00166.php <nowiki>Re: [HACKERS] Including Snapshot Info with Indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-01/msg00049.php <nowiki>Re: [HACKERS] Including Snapshot Info with Indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01094.php <nowiki>TODO item: Allow data to be pulled directly from indexes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00003.php <nowiki>Re: [PATCHES] VACUUM Improvements - WIP Patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider automatic caching of statements at various levels:<br />
* Parsed query tree<br />
* Query execute plan<br />
* Query results <br />
<br />
:<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00823.php <nowiki>Cached Query Plans (was: global prepared statements)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing internal areas (NUM_CLOG_BUFFERS) when shared buffers is increased<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-10/msg01419.php <nowiki>Re: slru.c race condition (was Re: TRAP: FailedAssertion(&quot;!((itemid)-&gt;lp_flags &amp; 0x01)&quot;,)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00030.php <nowiki>clog_buffers to 64 in 8.3?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-08/msg00024.php <nowiki>CLOG Patch</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the amount of memory used by PrivateRefCount<br />
|<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg00797.php <nowiki>PrivateRefCount (for 8.3)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-01/msg00752.php <nowiki>Re: PrivateRefCount (for 8.3)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing higher priority queries to have referenced buffer cache pages stay in memory longer<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00562.php <nowiki>Re: How to keep a table in memory?</nowiki>]<br />
}}<br />
<br />
== Vacuum ==<br />
<br />
{{TodoItem<br />
|Auto-fill the free space map by scanning the buffer cache or by checking pages written by the background writer<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-02/msg01125.php <nowiki>Dead Space Map</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-03/msg00011.php <nowiki>Re: Automatic free space map filling</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider having single-page pruning update the visibility map<br />
* <nowiki>https://commitfest.postgresql.org/action/patch_view?id=75</nowiki><br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg02344.php <nowiki>Re: visibility maps and heap_prune</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve tracking of total relation tuple counts now that vacuum doesn't always scan the whole heap<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00531.php Partial vacuum versus pg_class.reltuples]<br />
}}<br />
<br />
{{TodoItem<br />
|Bias FSM towards returning free space near the beginning of the heap file, in hopes that empty pages at the end can be truncated by VACUUM}}<br />
<br />
{{TodoItem<br />
|Make FSM return free space based on table clustering, to assist in maintaining clustering?}}<br />
<br />
{{TodoItem<br />
|Consider a more compact data representation for dead tuple locations within VACUUM<br />
* [http://archives.postgresql.org/pgsql-patches/2007-05/msg00143.php <nowiki>Re: Have vacuum emit a warning when it runs out of maintenance_work_mem</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide more information in order to improve user-side estimates of dead space bloat in relations<br />
* [http://archives.postgresql.org/pgsql-general/2009-05/msg01039.php <nowiki>Re: Bloated Table</nowiki>]<br />
}}<br />
<br />
=== Auto-vacuum ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItemEasy<br />
|Issue log message to suggest VACUUM FULL if a table is nearly empty?}}<br />
<br />
{{TodoItem<br />
|Prevent long-lived temporary tables from causing frozen-xid advancement starvation<br />
|The problem is that autovacuum cannot vacuum them to set frozen xids; only the session that created them can do that. <br />
* [http://archives.postgresql.org/pgsql-general/2007-06/msg01645.php <nowiki>Re: AutoVacuum Behaviour Question</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Prevent autovacuum from running if an old transaction is still running from the last vacuum<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00899.php <nowiki>Re: Autovacuum and OldestXmin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have free space allocation bias away from using trailing table pages<br />
|This improves the chances of truncating the table during vacuum<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-09/msg01124.php <nowiki>FSM search modes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have autoanalyze of parent tables occur when child tables are modified<br />
* http://archives.postgresql.org/message-id/AANLkTinx8lLTEKWcyEQ1rxVz6WMJVKNezfXW5TKnNAU6@mail.gmail.com<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Locking ==<br />
<br />
{{TodoItem<br />
|Fix priority ordering of read and write light-weight locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00893.php <nowiki>lwlocks and starvation</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-11/msg00905.php <nowiki>Re: lwlocks and starvation</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix problem when multiple subtransactions of the same outer transaction hold different types of locks, and one subtransaction aborts<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-11/msg01011.php <nowiki>FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg00001.php <nowiki>Re: FOR SHARE vs FOR UPDATE locks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00435.php <nowiki>Re: [PATCHES] [pgsql-patches] Phantom Command IDs, updated patch</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00773.php <nowiki>Re: savepoints and upgrading locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow UPDATEs on only non-referential integrity columns not to conflict with referential integrity locks<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00073.php <nowiki>Referential Integrity and SHARE locks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add idle_in_transaction_timeout GUC so locks are not held for long periods of time}}<br />
<br />
{{TodoItem<br />
|Improve deadlock detection when a page cleaning lock conflicts with a shared buffer that is pinned<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-01/msg00138.php <nowiki>BUG #3883: Autovacuum deadlock with truncate?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-committers/2008-01/msg00365.php <nowiki>Re: pgsql: Add checks to TRUNCATE, CLUSTER, and REINDEX to prevent</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Detect deadlocks involving LockBufferForCleanup()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00873.php <nowiki>Thoughts about bug #3883</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a lock timeout parameter<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00485.php <nowiki>SELECT ... FOR UPDATE [WAIT integer | NOWAIT] for 8.5</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider improving serialized transaction behavior to avoid anomalies<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00217.php <nowiki>Serializable Isolation without blocking</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01136.php <nowiki>User-facing aspects of serializable transactions</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00035.php <nowiki>Re: User-facing aspects of serializable transactions</nowiki>]<br />
}}<br />
<br />
== Startup Time Improvements ==<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for backend creation<br />
|This would prevent the overhead associated with process creation. Most operating systems have trivial process creation time compared to database startup overhead, but a few operating systems (Win32, Solaris) might benefit from threading. Also explore the idea of a single session using multiple threads to execute a statement faster.}}<br />
<br />
== Write-Ahead Log ==<br />
<br />
{{TodoItem<br />
|Eliminate need to write full pages to WAL before page modification<br />
|Currently, to protect against partial disk page writes, we write full page images to WAL before they are modified so we can correct any partial page writes during recovery. These pages can also be eliminated from point-in-time archive files. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-06/msg00655.php <nowiki>Re: Index Scans become Seq Scans after VACUUM ANALYSE</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|When full page writes are off, write CRC to WAL and check file system blocks on recovery<br />
|If CRC check fails during recovery, remember the page in case a later CRC for that page properly matches.}}<br />
<br />
{{TodoItem<br />
|Write full pages during file system write and not when the page is modified in the buffer cache<br />
|This allows most full page writes to happen in the background writer. It might cause problems for applying WAL on recovery into a partially-written page, but later the full page will be replaced from WAL.}}<br />
<br />
{{TodoItem<br />
|Reduce WAL traffic so only modified values are written rather than entire rows<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-03/msg01589.php <nowiki>Reduction in WAL for UPDATEs</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow WAL information to recover corrupted pg_controldata<br />
* [http://archives.postgresql.org/pgsql-patches/2006-06/msg00025.php <nowiki>Re: [HACKERS] pg_resetxlog -r flag</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a way to reduce rotational delay when repeatedly writing last WAL page<br />
|Currently fsync of WAL requires the disk platter to perform a full rotation to fsync again. One idea is to write the WAL to different offsets that might reduce the rotational delay. <br />
* [http://archives.postgresql.org/pgsql-hackers/2002-11/msg00483.php <nowiki>500 tpsQL + WAL log implementation</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow WAL logging to be turned off for a table, but the table might be dropped or truncated during crash recovery<br />
|Allow tables to bypass WAL writes and just fsync() dirty pages on commit. This should be implemented using ALTER TABLE, e.g. <nowiki>ALTER TABLE PERSISTENCE [ DROP | TRUNCATE | DEFAULT ]</nowiki>. Tables using non-default logging should not use referential integrity with default-logging tables. A table without dirty buffers during a crash could perhaps avoid the drop/truncate. <br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01016.php <nowiki>Re: [Bizgres-general] WAL bypass for INSERT, UPDATE and</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow WAL logging to be turned off for a table, but the table would avoid being truncated/dropped<br />
|To do this, only a single writer can modify the table, and writes must happen only on new pages so the new pages can be removed during crash recovery. Readers can continue accessing the table. Such tables probably cannot have indexes. One complexity is the handling of indexes on TOAST tables. <br />
* [http://archives.postgresql.org/pgsql-hackers/2005-12/msg01016.php <nowiki>Re: [Bizgres-general] WAL bypass for INSERT, UPDATE and</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Speed WAL recovery by allowing more than one page to be prefetched<br />
|This should be done utilizing the same infrastructure used for prefetching in general to avoid introducing complex error-prone code in WAL replay. <br />
* [http://archives.postgresql.org/pgsql-general/2007-12/msg00683.php <nowiki>Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00497.php <nowiki>Re: [GENERAL] Slow PITR restore</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg01279.php <nowiki>Read-ahead and parallelism in redo recovery</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve WAL concurrency by increasing lock granularity<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00556.php <nowiki>Reworking WAL locking</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Be more aggressive about creating WAL files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01325.php <nowiki>Re: PANIC caused by open_sync on Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-07/msg01075.php <nowiki>PreallocXlogFiles</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2005-04/msg00556.php <nowiki>WAL/PITR additional items</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Have resource managers report the duration of their status changes<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg01468.php <nowiki>Recovery of Multi-stage WAL actions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Move pgfoundry's xlogdump to /contrib and have it rely more closely on the WAL backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00035.php <nowiki>xlogdump</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Close deleted WAL files held open in *nix by long-lived read-only backends<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg01754.php <nowiki>Deleted WAL files held open by backends in Linux</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00060.php <nowiki>Re: Deleted WAL files held open by backends in Linux</nowiki>]<br />
}}<br />
<br />
== Optimizer / Executor ==<br />
<br />
{{TodoItem<br />
|Improve selectivity functions for geometric operators}}<br />
<br />
{{TodoItem<br />
|Precompile SQL functions to avoid overhead}}<br />
<br />
{{TodoItem<br />
|Create utility to compute accurate random_page_cost value}}<br />
<br />
{{TodoItem<br />
|Consider increasing the default values of from_collapse_limit, join_collapse_limit, and/or geqo_threshold<br />
* [http://archives.postgresql.org/message-id/4136ffa0905210551u22eeb31bn5655dbe7c9a3aed5@mail.gmail.com from_collapse_limit vs. geqo_threshold]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve ability to display optimizer analysis using OPTIMIZER_DEBUG}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE issue NOTICE messages when the estimated and actual row counts differ by a specified percentage}}<br />
<br />
{{TodoItem<br />
|Have EXPLAIN ANALYZE report rows as floating-point numbers<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg01363.php <nowiki>explain analyze rows=%.0f</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-06/msg00108.php <nowiki>Re: explain analyze rows=%.0f</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Log statements where the optimizer row estimates were dramatically different from the number of rows actually found?}}<br />
<br />
{{TodoItem<br />
|Improve how ANALYZE computes in-doubt tuples<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-11/msg00771.php <nowiki>VACUUM/ANALYZE counting of in-doubt tuples</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider compressed annealing to search for query plans<br />
|This might replace GEQO.<br />
* http://archives.postgresql.org/message-id/15658.1241278636%40sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using a hash for joining to a large IN (VALUES ...) list<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-05/msg00450.php <nowiki>Planning large IN lists</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow single batch hash joins to preserve outer pathkeys<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-09/msg00806.php Re: Potential Join Performance Issue]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|"lazy" hash tables - look up only the tuples that are actually requested<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid building the same hash table more than once during the same query<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid hashing for distinct and then re-hashing for hash join<br />
* [http://archives.postgresql.org/message-id/4136ffa0902191346g62081081v8607f0b92c206f0a@mail.gmail.com Re: Fixing Grittner's planner issues]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-04/msg00153.php a few crazy ideas about hash joins]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow hashing to be used on arrays, if the element type is hashable<br />
* http://archives.postgresql.org/message-id/11087.1244905821@sss.pgh.pa.us<br />
}}<br />
<br />
{{TodoItem<br />
|Improve use of expression indexes for ORDER BY <br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01553.php <nowiki>Resjunk sort columns, Heikki's index-only quals patch, and bug #5000</nowiki>]<br />
}}<br />
<br />
== Background Writer ==<br />
<br />
{{TodoItem<br />
|Consider having the background writer update the transaction status hint bits before writing out the page<br />
|Implementing this requires the background writer to have access to system catalogs and the transaction status log.}}<br />
<br />
{{TodoItem<br />
|Consider adding buffers the background writer finds reusable to the free list <br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Automatically tune bgwriter_delay based on activity rather then using a fixed interval<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php <nowiki>Background LRU Writer/free list</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider whether increasing BM_MAX_USAGE_COUNT improves performance<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-06/msg01007.php <nowiki>Bgwriter LRU cleaning: we've been going at this all wrong</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Test to see if calling PreallocXlogFiles() from the background writer will help with WAL segment creation latency<br />
* [http://archives.postgresql.org/pgsql-patches/2007-06/msg00340.php <nowiki>Re: Load Distributed Checkpoints, final patch</nowiki>]<br />
}}<br />
<br />
== Concurrent Use of Resources ==<br />
<br />
{{TodoItem<br />
|Do async I/O for faster random read-ahead of data<br />
|Async I/O allows multiple I/O requests to be sent to the disk with results coming back asynchronously.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00820.php <nowiki>Asynchronous I/O Support</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-performance/2007-09/msg00255.php <nowiki>Re: random_page_costs - are defaults of 4.0 realistic for SCSI RAID 1</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00027.php <nowiki>There's random access and then there's random access</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-01/msg00170.php <nowiki>Bitmap index scan preread using posix_fadvise (Was: There's random access and then there's random access)</nowiki>]<br />
The above patch is already applied as of 8.4, but it still remains to figure out how to handle plain indexscans effectively.<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-01/msg00806.php Problems with the patch submitted for posix_fadvise in index scans]<br />
}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better I/O utilization<br />
|This would allow a single query to make use of multiple I/O channels simultaneously. One idea is to create a background reader that can pre-fetch sequential and index scan pages needed by other backends. This could be expanded to allow concurrent reads from multiple devices in a partitioned table.}}<br />
<br />
{{TodoItem<br />
|Experiment with multi-threaded backend for better CPU utilization<br />
|This would allow several CPUs to be used for a single query, such as for sorting or query execution.<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00945.php <nowiki>Multi CPU Queries - Feedback and/or suggestions wanted!</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|SMP scalability improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00439.php <nowiki>Straightforward changes for increased SMP scalability</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00206.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
== TOAST ==<br />
<br />
{{TodoItem<br />
|Allow user configuration of TOAST thresholds<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-02/msg00213.php <nowiki>Re: Proposed adjustments in MaxTupleSize and toastthresholds</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00082.php <nowiki>pg_lzcompress strategy parameters</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce unnecessary cases of deTOASTing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-09/msg00895.php <nowiki>Re: [PATCHES] Eliminate more detoast copies for packed varlenas</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce costs of repeat de-TOASTing of values<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-06/msg01096.php <nowiki>WIP patch: reducing overhead for repeat de-TOASTing</nowiki>]<br />
}}<br />
<br />
== Miscellaneous Performance ==<br />
<br />
{{TodoItem<br />
|Use mmap() rather than SYSV shared memory or to write WAL files?<br />
|This would remove the requirement for SYSV SHM but would introduce portability issues. Anonymous mmap (or mmap to /dev/zero) is required to prevent I/O overhead.}}<br />
<br />
{{TodoItem<br />
|Consider mmap()'ing files into a backend?<br />
|Doing I/O to large tables would consume a lot of address space or require frequent mapping/unmapping. Extending the file also causes mapping problems that might require mapping only individual pages, leading to thousands of mappings. Another problem is that there is no way to _prevent_ I/O to disk from the dirty shared buffers so changes could hit disk before WAL is written.}}<br />
<br />
{{TodoItem<br />
|Add a script to ask system configuration questions and tune postgresql.conf}}<br />
<br />
{{TodoItem<br />
|Consider ways of storing rows more compactly on disk:<br />
* Reduce the row header size?<br />
* Consider reducing on-disk varlena length from four bytes to two because a heap row cannot be more than 64k in length}}<br />
<br />
{{TodoItem<br />
|Consider transaction start/end performance improvements<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00948.php <nowiki>Reducing Transaction Start/End Contention</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00361.php <nowiki>Re: Reducing Transaction Start/End Contention</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow configuration of backend priorities via the operating system<br />
|Though backend priorities make priority inversion during lock waits possible, research shows that this is not a huge problem.<br />
* [http://archives.postgresql.org/pgsql-general/2007-02/msg00493.php <nowiki>Priorities for users or queries?</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider increasing the minimum allowed number of shared buffers<br />
* [http://archives.postgresql.org/pgsql-bugs/2008-02/msg00157.php <nowiki>Re: [PATCH] Don't bail with legitimate -N/-B options</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider if CommandCounterIncrement() can avoid its AcceptInvalidationMessages() call<br />
* [http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php <nowiki>pgsql: Avoid incrementing the CommandCounter when</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider Cartesian joins when both relations are needed to form an indexscan qualification for a third relation<br />
* [http://archives.postgresql.org/pgsql-performance/2007-12/msg00090.php <nowiki>Re: TB-sized databases</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider not storing a NULL bitmap on disk if all the NULLs are trailing<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-12/msg00624.php <nowiki>Proposal for Null Bitmap Optimization(for Trailing NULLs)</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2007-12/msg00109.php <nowiki>Re: [HACKERS] Proposal for Null Bitmap Optimization(for TrailingNULLs)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Sort large UPDATE/DELETEs so it is done in heap order<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg01119.php <nowiki>Possible future performance improvement: sort updates/deletes by ctid</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow one transaction to see tuples using the snapshot of another transaction<br />
|This would assist multiple backends in working together. <br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00400.php <nowiki>Transaction Snapshot Cloning</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider decreasing the I/O caused by updating tuple hint bits<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00847.php <nowiki>Hint Bits and Write I/O</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-07/msg00199.php <nowiki>Re: [HACKERS] Hint Bits and Write I/O</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid the requirement of freezing pages that are infrequently modified <br />
|If all rows on a page are visible, it is possible to set a bit in the visibility map (once the visibility map is 100% reliable) and not need to freeze the page, avoiding a page rewrite<br />
* http://archives.postgresql.org/message-id/4BF701CF.2090205@agliodbs.com<br />
* http://archives.postgresql.org/pgsql-hackers/2010-06/msg00082.php<br />
}}<br />
<br />
{{TodoItem<br />
|Avoid reading in b-tree pages when replaying vacuum records in hot standby mode<br />
* [http://archives.postgresql.org/message-id/1272571938.4161.14739.camel@ebony <nowiki>Hot Standby tuning for btree_xlog_vacuum()</nowiki>]<br />
}}<br />
<br />
== Miscellaneous Other ==<br />
<br />
{{TodoItem<br />
|Deal with encoding issues for filenames in the server filesystem<br />
* {{MessageLink|20090413184335.39BE.52131E4D@oss.ntt.co.jp|a proposed patch here}}<br />
* {{MessageLink|8484.1244655656@sss.pgh.pa.us|some issues about it here}}<br />
* {{MessageLink|20100107103740.97A5.52131E4D@oss.ntt.co.jp|Windows-specific patch here}}<br />
}}<br />
<br />
{{TodoItem<br />
|Deal with encoding issues in the output of localeconv()<br />
* [http://archives.postgresql.org/message-id/40c6d9160904210658y590377cfw6dbbecb53d2b8be0@mail.gmail.com bug report]<br />
* [http://archives.postgresql.org/message-id/49EF8DA0.90008@tpf.co.jp draft patch]<br />
* [http://archives.postgresql.org/message-id/21710.1243620986@sss.pgh.pa.us review of patch]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide schema name and other fields available from SQL GET DIAGNOSTICS in error reports<br />
* [http://archives.postgresql.org/message-id/dcc563d10810211907n3c59a920ia9eb7cd2a6d5ea58@mail.gmail.com <nowiki>How to get schema name which violates fk constraint</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-11/msg00846.php <nowiki>patch - Report the schema along table name in a referential failure error message</nowiki>]<br />
* {{MessageLink|3191.1263306359@sss.pgh.pa.us|Re: NOT NULL violation and error-message}}<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-08/msg00213.php <nowiki>the case for machine-readable error fields</nowiki>]<br />
}}<br />
<br />
{{TodoItemEasy<br />
| Provide [http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html#LIBPQ-CONNECT-FALLBACK-APPLICATION-NAME fallback_application_name] in contrib/pgbench, oid2name, and dblink.<br />
* {{MessageLink|w2g9837222c1004070216u3bc46b3ahbddfdffdbfb46212@mail.gmail.com|fallback_application_name and pgbench}}<br />
}}<br />
<br />
== Source Code ==<br />
<br />
{{TodoItem<br />
|Add use of 'const' for variables in source tree}}<br />
<br />
{{TodoItemEasy<br />
|Remove warnings created by -Wcast-align}}<br />
<br />
{{TodoItem<br />
|Move platform-specific ps status display info from ps_status.c to ports}}<br />
<br />
{{TodoItem<br />
|Add optional CRC checksum to heap and index pages<br />
|One difficulty is how to prevent hint bit changes from affecting the computed CRC checksum.<br />
* http://archives.postgresql.org/message-id/19934.1226601952%40sss.pgh.pa.us<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00002.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg01028.php <nowiki>double-buffering page writes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00524.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01101.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-12/msg00011.php <nowiki>Re: Block-level CRC checks</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider a faster CRC32 algorithm<br />
* http://archives.postgresql.org/pgsql-hackers/2010-05/msg01112.php<br />
}}<br />
<br />
{{TodoItem<br />
|Allow cross-compiling by generating the zic database on the target system}}<br />
<br />
{{TodoItem<br />
|Improve NLS maintenance of libpgport messages linked onto applications}}<br />
<br />
{{TodoItem<br />
|Improve the module installation experience (/contrib, etc)<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-04/msg00132.php <nowiki>modules</nowiki>]<br />
* {{messageLink|ca33c0a30807231640n6fb4035dod8121a18aa1fa29c@mail.gmail.com|Re: PostgreSQL extensions packaging}}<br />
* {{messageLink|ca33c0a30804061349s41b4d8fcsa9c579454b27ecd2@mail.gmail.com|Database owner installable modules patch}}<br />
* [http://archives.postgresql.org//pgsql-hackers/2009-03/msg00855.php <nowiki>Re: contrib function naming, and upgrade issues</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2009-05/msg00912.php <nowiki>search_path vs extensions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Use UTF8 encoding for NLS messages so all server encodings can read them properly}}<br />
<br />
{{TodoItem<br />
|Allow creation of universal binaries for Darwin<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00884.php <nowiki>Getting to universal binaries for Darwin</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider GnuTLS if OpenSSL license becomes a problem<br />
* [http://archives.postgresql.org/pgsql-patches/2006-05/msg00040.php <nowiki>[PATCH] Add support for GnuTLS</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-12/msg01213.php <nowiki>TODO: GNU TLS</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider making NAMEDATALEN more configurable in future releases}}<br />
<br />
{{TodoItem<br />
|Research use of signals and sleep wake ups<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-07/msg00003.php <nowiki>Restartable signals 'n all that</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Allow C++ code to more easily access backend code<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg00302.php <nowiki>Mostly Harmless: Welcoming our C++ friends</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider simplifying how memory context resets handle child contexts<br />
* [http://archives.postgresql.org/pgsql-patches/2007-08/msg00067.php <nowiki>Re: Memory leak in nodeAgg</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Create three versions of libpgport to simplify client code<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-10/msg00154.php <nowiki>8.4 TODO item: make src/port support libpq and ecpg directly</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve detection of shared memory segments being used by others by checking the SysV shared memory field 'nattch'<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00656.php <nowiki>postgresql in FreeBSD jails: proposal</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00673.php <nowiki>Re: postgresql in FreeBSD jails: proposal</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Implement the non-threaded Avahi service discovery protocol<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00939.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00097.php <nowiki>Re: Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg01211.php <nowiki>Re: [PATCHES] Avahi support for Postgresql</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-04/msg00001.php <nowiki>Re: [HACKERS] Avahi support for Postgresql</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Reduce data row alignment requirements on some 64-bit systems<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-10/msg00369.php <nowiki>[WIP] Reduce alignment requirements on 64-bit systems.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Restructure TOAST internal storage format for greater flexibility<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-11/msg00049.php <nowiki>Re: PG_PAGE_LAYOUT_VERSION 5 - time for change</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
| Add regression tests for pg_dump/restore<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-02/msg01967.php <nowiki>"make install-check-pg_dump" target in src/regress]</nowiki>]<br />
}}<br />
<br />
=== Documentation ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Convert single quotes to apostrophes in the PDF documentation<br />
* [http://archives.postgresql.org/pgsql-docs/2007-12/msg00059.php <nowiki>SGML docs and pdf single-quotes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Provide a manpage for postgresql.conf<br />
* {{messageLink|20080819194311.GH4428@alvh.no-ip.org|A smaller default postgresql.conf}}<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Change the manpage-generating toolchain to use the new XML-based docbook2x tools<br />
* {{messageLink|200808211910.37524.peter_e@gmx.net|A smaller default postgresql.conf}}<br />
}}<br />
<br />
{{TodoItem<br />
|Consider changing documentation format from SGML to XML<br />
* [http://archives.postgresql.org/pgsql-docs/2006-12/msg00152.php <nowiki>Re: Authoring Tools WAS: Switching to XML</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Document support for N<nowiki>' '</nowiki> national character string literals, if it matches the SQL standard<br />
* http://archives.postgresql.org/message-id/1275895438.1849.1.camel@fsopti579.F-Secure.com<br />
}}<br />
<br />
{{TodoItem<br />
|Add diagrams to the documentation<br />
* http://archives.postgresql.org/pgsql-docs/2010-07/msg00001.php<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== /contrib/pg_upgrade ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Remove copy_dir() code, or use it<br />
}}<br />
<br />
{{TodoItem<br />
|Handle large object comments<br />
|This is difficult to do because the large object doesn't exist when --schema-only is loaded.<br />
}}<br />
<br />
{{TodoItem<br />
|Consider using pg_depend for checking object usage in version.c<br />
}}<br />
<br />
{{TodoItem<br />
|If reindex is necessary, allow it to be done in parallel with pg_dump custom format<br />
}}<br />
<br />
{{TodoItem<br />
|Migrate pg_statistic by dumping it out as a flat file, so analyze is not necessary<br />
|pg_class.oid is not preserved so schema.tablename must be used.<br />
}}<br />
<br />
{{TodoItem<br />
|Improve testing, perhaps using the buildfarm<br />
|The buildfarm has access to multiple versions of PostgreSQL.<br />
}}<br />
<br />
{{TodoItem<br />
|Create machine-readable output of pg_controldata<br />
|This would avoid parsing its output.<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Windows ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Remove configure.in check for link failure when cause is found}}<br />
<br />
{{TodoItem<br />
|Remove readdir() errno patch when runtime/mingwex/dirent.c rev 1.4 is released}}<br />
<br />
{{TodoItem<br />
|Allow psql to use readline once non-US code pages work with backslashes}}<br />
<br />
{{TodoItem<br />
|Fix problem with shared memory on the Win32 Terminal Server}}<br />
<br />
{{TodoItem<br />
|Improve signal handling<br />
* [http://archives.postgresql.org/pgsql-patches/2005-06/msg00027.php <nowiki>Simplify Win32 Signaling code</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Convert MSVC build system to remove most batch files<br />
* [http://archives.postgresql.org/pgsql-hackers/2007-08/msg00961.php <nowiki>MSVC build system</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Support pgxs when using MSVC}}<br />
<br />
{{TodoItem<br />
|Fix MSVC NLS support, like for to_char()<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-02/msg00485.php <nowiki>NLS on MSVC strikes back!</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php <nowiki>Fix for 8.3 MSVC locale (Was [HACKERS] NLS on MSVC strikes back!)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Find a correct rint() substitute on Windows<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00808.php <nowiki>Minor bug in src/port/rint.c</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Fix global namespace issues when using multiple terminal server sessions<br />
* [http://archives.postgresql.org/message-id/48F3BFCC.8030107@dunslane.net problems with Windows global namespace]}}<br />
<br />
{{TodoItem<br />
|Change from the current autoconf/gmake build system to cmake<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-12/msg01869.php <nowiki>About CMake (was Re: [COMMITTERS] pgsql: Append major version number and for libraries soname major)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Improve consistency of path separator usage<br />
* http://archives.postgresql.org/message-id/49C0BDC5.4010002@hagander.net<br />
}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
=== Wire Protocol Changes ===<br />
{{TodoSubsection}}<br />
<br />
{{TodoItem<br />
|Allow dynamic character set handling}}<br />
<br />
{{TodoItem<br />
|Add decoded type, length, precision}}<br />
<br />
{{TodoItem<br />
|Use compression?}}<br />
<br />
{{TodoItem<br />
|Update clients to use data types, typmod, schema.table.column names of result sets using new statement protocol}}<br />
<br />
{{TodoEndSubsection}}<br />
<br />
== Exotic Features ==<br />
<br />
{{TodoItem<br />
|Add pre-parsing phase that converts non-ISO syntax to supported syntax<br />
|This could allow SQL written for other databases to run without modification.}}<br />
<br />
{{TodoItem<br />
|Allow plug-in modules to emulate features from other databases}}<br />
<br />
{{TodoItem<br />
|Add features of Oracle-style packages<br />
|A package would be a schema with session-local variables, public/private functions, and initialization functions. It is also possible to implement these capabilities in any schema and not use a separate &quot;packages&quot; syntax at all.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00384.php <nowiki>proposal for PL packages for 8.3.</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Consider allowing control of upper/lower case folding of unquoted identifiers<br />
* [http://archives.postgresql.org/pgsql-hackers/2004-04/msg00818.php <nowiki>Bringing PostgreSQL torwards the standard regarding case folding</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg01527.php <nowiki>Re: [SQL] Case Preservation disregarding case sensitivity?</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-03/msg00849.php <nowiki>TODO Item: Consider allowing control of upper/lower case folding of unquoted, identifiers</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-07/msg00415.php <nowiki>Identifier case folding notes</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Add autonomous transactions<br />
* [http://archives.postgresql.org/pgsql-hackers/2008-01/msg00893.php <nowiki>autonomous transactions</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Give query progress indication<br />
* [[Query progress indication]]<br />
}}<br />
<br />
{{TodoItem<br />
|Rethink our type system<br />
* [[Rethinking datatypes]]<br />
}}<br />
<br />
== Features We Do ''Not'' Want ==<br />
<br />
{{TodoItem<br />
|All backends running as threads in a single process (not wanted)<br />
|This eliminates the process protection we get from the current setup. Thread creation is usually the same overhead as process creation on modern systems, so it seems unwise to use a pure threaded model, and MySQL and DB2 have demonstrated that threads introduce as many issues as they solve. Threading specific operations such as I/O, seq scans, and connection management has been discussed and will probably be implemented to enable specific performance features. Moving to a threaded engine would also require halting all other work on PostgreSQL for one to two years.}}<br />
<br />
{{TodoItem<br />
|Optimizer hints (not wanted)<br />
|Optimizer hints are used to work around problems in the optimizer and introduce upgrade and maintenance issues. We would rather have the problems reported and fixed. We have discussed a more sophisticated system of per-class cost adjustment instead, but a specification remains to be developed.<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-08/msg00506.php <nowiki>Re: An Idea for planner hints</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00517.php <nowiki>Index Tuning Features</nowiki>]<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-10/msg00663.php <nowiki>Re: [PERFORM] Hints proposal</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Embedded server (not wanted)<br />
|While PostgreSQL clients runs fine in limited-resource environments, the server requires multiple processes and a stable pool of resources to run reliably and efficiently. Stripping down the PostgreSQL server to run in the same process address space as the client application would add too much complexity and failure cases. Besides, there are several very mature embedded SQL databases already available.}}<br />
<br />
{{TodoItem<br />
|Obfuscated function source code (not wanted)<br />
|Obfuscating function source code has minimal protective benefits because anyone with super-user access can find a way to view the code. At the same time, it would greatly complicate backups and other administrative tasks. To prevent non-super-users from viewing function source code, remove SELECT permission on pg_proc.<br />
* [http://archives.postgresql.org/pgsql-general/2008-09/msg00668.php <nowiki>Obfuscated stored procedures (was Re: Oracle and Postgresql)</nowiki>]<br />
}}<br />
<br />
{{TodoItem<br />
|Indeterminate behavior for the GROUP BY clause (not wanted)<br />
|At least one other database product allows specification of a subset of the result columns which GROUP BY would need to be able to provide predictable results; the server is free to return any value from the group. This is not viewed as a desirable feature. PostgreSQL 9.1 will allow result columns that are not referenced by GROUP BY if a primary key for the same table is referenced in GROUP BY.<br />
* [http://archives.postgresql.org/pgsql-hackers/2010-03/msg00297.php <nowiki>Re: SQL compatibility reminder: MySQL vs PostgreSQL</nowiki>]<br />
}}<br />
<br />
</div><br />
<br />
[[Category:Todo]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Submitting_a_Patch&diff=12046Submitting a Patch2010-09-24T00:59:24Z<p>Schmiddy: /* Patch submission */ replace info about diff generation with cvs to git</p>
<hr />
<div>== Initial patch design ==<br />
<br />
If you have a trivial patch that serves an obvious need, you may be able to write the patch and submit it directly to the [http://archives.postgresql.org/pgsql-hackers/ pgsql-hackers] mailing list without having its design reviewed first. But in general, a non-trivial change should be discussed (potentially before the code is even written) on the [http://archives.postgresql.org/pgsql-hackers/ pgsql-hackers] list before being submitted as a patch.<br />
<br />
For general coding style guidelines, see the [[Developer FAQ]] and the [http://developer.postgresql.org/pgdocs/postgres/source.html PostgreSQL Coding Conventions].<br />
<br />
=== Design your interface first ===<br />
<br />
Ask yourself these questions: <br />
<br />
* Will the user interact with this new feature? if so, how? <br />
* What is the syntax? Have ideas, and the ability to defend technical decisions you believe strongly in.<br />
* What are the exact semantics/behaviors?<br />
* Are there any backward compatibility issues? <br />
* Get community buy-in at this level of detail before you start coding. But not necessarily consensus.<br />
* Write an opening paragraph to your email to the -hackers list that answers these questions:<br />
**This is the kind of problem I'm trying to solve<br />
**This is what it is doing right now<br />
**This is what it will do.<br />
<br />
Mostly, get someone from the community involved in your ideas as early as possible so that you can even get half-baked ideas vetted early, rather than creating something in a vacuum. Similarly, it's easier to make progress and keep patches focused if you concentrate on the smallest portion of the idea you can execute perfectly. Resist the temptation to build a giant patch all at once, as those are much less likely to be reviewed usefully and therefore committed. You should take a look at how your patch will eventually be [[Reviewing a Patch|reviewed]], so you can make sure that review is likely to succeed.<br />
<br />
=== Save us the trouble of reformatting your code ===<br />
<br />
Please read [http://developer.postgresql.org/pgdocs/postgres/source.html PostgreSQL Coding Conventions]. Also, follow the style of the adjacent code! [http://archives.postgresql.org/pgsql-hackers/2009-08/msg01001.php Suggestions from Tom] clarify some of the trickier situations you might run into.<br />
<br />
Naming things is important, and when in doubt, ask someone else to help you with names. We tend to use [http://en.wikipedia.org/wiki/CamelCase CamelCase] or underscores: thisStyleIsOkay or this_is_okay_too.<br />
<br />
Generally, try to blend in with the surrounding code. Do not use #ifdef's to enable your changes. Comments are for clarification not for delineating your code from the surroundings.<br />
<br />
Please remove any spurious whitespace. "git diff --color" makes them stand out like a sore thumb, in red.<br />
<br />
== Patch submission==<br />
<br />
Once you believe your patch is complete, you can submit it via e-mail to the [http://archives.postgresql.org/pgsql-hackers/ pgsql-hackers] mailing list. At that time, or after you wait for initial feedback, you should also add it to the page for the next [https://commitfest.postgresql.org/ CommitFest]. <br />
<br />
Normally changes should be submitted as a single patch that includes every file touched. If the patch is large and can be logically separated into distinct and separately commit-able sections for easier review, with a clear order they get applied in described when applicable, that can be more straightforward for reviewers to work with for more complicated patches. Patches in [http://en.wikipedia.org/wiki/Diff#Context_format Context Diff] format are preferred. See [[Working_with_Git#Context_diffs_with_Git|Working with git]] for ways to do this. [http://petereisentraut.blogspot.com/2009/09/how-to-submit-patch-by-email.html How to submit a patch by email] for more details about mailing in patches. If you're a new submitter, the suggestion there about using your judgment on patch formatting is not a recommended practice however--you should be using the standard context diff format.<br />
<br />
Please note that PostgreSQL is licensed under a BSD license. By posting a patch to the public PostgreSQL mailing lists, you are giving the PostgreSQL Global Development Group the non-revocable right to distribute your patch under the BSD license.<br />
<br />
Please include all of the following information with each patch submitted<br />
<br />
* Project name.<br />
* Uniquely identifiable file name, so we can tell difference between your v1 and v24.<br />
* What the patch does in a short paragraph.<br />
* Whether the patch is for discussion or for application (see WIP notes below)<br />
* Which CVS branch the patch is against (ordinarily this will be HEAD). For more on branches in PostgreSQL, see [[CVS Branch Management]].<br />
* Whether it compiles and tests successfully, so we know nothing obvious is broken.<br />
* Whether it contains any platform-specific items and if so, has it been tested on other platforms.<br />
* Confirm that the patch includes [[Regression test authoring|regression tests]] to check the new feature actually works as described.<br />
* Include some docs on how to use the new feature, including examples.<br />
* Describe the effect your patch has on performance, if any. If the patch is intended to improve performance, it's a good idea to include some reproducible tests to demonstrate the improvement. If a reviewer cannot duplicate your claimed performance improvement in a short period of time, it's very likely your patch will be bounced. Do not expect that a reviewer is going to find your performance feature so interesting that they will build an entire test suite to prove it works. You should have done that as part of patch validation, and included the necessary framework for testing with the submission.<br />
* Try to include a few lines about why you chose to do things particular ways, rather than let your reviewer guess what was happening. This can be done as code comments, but it might also be an additional reviewers' guide, or additions to a README file in one of the code directories.<br />
* If your patch addresses a [[Todo]] item, please give the name of the Todo item in your email. This is so that the reviewers will know that the item needs to be marked as done if your patch is applied.<br />
<br />
The objective of all of these suggested items is to ensure that the reviewer's time is not wasted. You spent time writing the code, but that does '''not''' mean you can demand time, energy and interest from a reviewer. Make it easy on yourself and others so that your patch is accepted quickly, easily and with good humor on all sides.<br />
<br />
It is helpful for early patches, ones not intended to be of commit quality, to be labelled clearly as such so that the appropriate style of review is done. The abbreviation WIP ("Work in Progress") is the standard shorthand to attach to patches intended for review not as a commit candidate, but for design feedback. Labelling your patch as a WIP on your e-mail subject line and on the matching description in the CF application will advise reviewers to focus more on the general approach, rather than on things like coding style that can normally be ignored in the early portion of a patch's lifecycle.<br />
<br />
Submitting the patch is just the first step towards getting it committed. Very few patches are committed exactly as originally submitted, even those submitted by experienced professional developers. For any non-trivial patch you should plan for at least 3 versions before final acceptance.<br />
<br />
The easiest way to get your patch rejected is to make lots of unrelated changes, like reformatting lines, correcting comments you felt were poorly worded etc. Each patch should have the minimum set of changes required to fulfil the single stated objective.<br />
<br />
=== Submission timing ===<br />
<br />
You need to pay attention to what the community work cycle is. If you're sending in a brand new idea in the beta phase, don't be surprised if no one is paying attention because they are focused on release work. Come back when the beta is done, please!<br />
<br />
PostgreSQL development is also organized with periodic [[CommitFest|CommitFests]], which are periods where new development halts in order to focus on patch review and committing. It's best if you can avoid sending in a new patch during the occasional weeks when there is an active CommitFest; you can check the schedule via the [https://commitfest.postgresql.org/ CommitFest application]. If your schedule doesn't allow waiting until an active CommitFest is over, you should explicitly label your submission as intended for the next CommitFest, not the current one, so that it's clear it's not intended to be part of the active review process.<br />
<br />
== Patch review and commit ==<br />
<br />
There's basically three different workflows a patch can follow after it's been submitted that lead to it being commited:<br />
<br />
Workflow A:<br />
# You post patch to pgsql-hackers<br />
# A committer picks it up immediately and commits it.<br />
<br />
Workflow B:<br />
# You post a patch to pgsql-hackers<br />
# You add the patch to the [http://commitfest.postgresql.org/action/commitfest_view/open open commitfest] queue<br />
# A committer picks up the patch from the queue, and commits it<br />
<br />
Workflow C:<br />
# You post a patch to pgsql-hackers<br />
# Bruce adds the patch to a list of unapplied patches<br />
# At the beginning of the next commit fest, Alvaro (with the help from others, I hope) goes through the list, and adds the patch to the [http://commitfest.postgresql.org/action/commitfest_view/open open commitfest] queue<br />
# A [[Committers|committer]] picks up the patch from the queue, and commits it<br />
<br />
At any of these stages, your patch might instead be rejected for technical, style, or other reasons. These rejections will normally come with feedback on whether an improved version of that patch would be more acceptable. In those cases, you should consider updating your patch based on that feedback and re-submit.<br />
<br />
== Followup on submissions ==<br />
<br />
=== How do you get someone to respond to you? ===<br />
<br />
You've sent an email to -hackers and no one has responded. What do you do?<br />
<br />
* Make sure you've added your patch to the [http://commitfest.postgresql.org/action/commitfest_view/open open commitfest] queue.<br />
* Start out by reviewing a patch or responding to email on the lists. Even if it is unrelated to what you're doing.<br />
* Start with submitting a patch that is small and uncontroversial to help them understand you, and to get you familiar with the overall process.<br />
* People are more willing to listen and work with someone who is already contributing.<br />
* Also, in our community -- if no one objects, then there is implicit approval. Within reason!<br />
<br />
Participating in community is a process, not a single event.<br />
<br />
=== Submitting patch updates ===<br />
<br />
When submitting a new version of a previously submitted patch, you should do a few additional things:<br />
<br />
* Uniquely identify the new version, usually done via an incremented suffix on the name of the patch<br />
* Make sure it's easy to find any earlier discussion of the patch. Don't expect that everyone will still be able to find previous submissions on their own. Either fully duplicate the information about the patch from your original messages, or provide a clear link to the earlier message via the [http://archives.postgresql.org/pgsql-hackers/ archives]. Note that you can link to your earlier post using the e-mail message ID of what you sent earlier, perhaps by checking your sent e-mail for it. That type of link is preferred because links to the archive by message number might sometimes get renumbered. See [[Template:MessageLink]] for more details.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=PostgreSQL_for_Oracle_DBAs&diff=10673PostgreSQL for Oracle DBAs2010-05-03T17:39:45Z<p>Schmiddy: /* REDO and Archiving */</p>
<hr />
<div>= Introduction =<br />
<br />
The following article contains information to help an Oracle DBA understand<br />
some terms and the management of a PostgreSQL database. This article is<br />
intended to be an introduction to PostgreSQL, not a tutorial or a complete<br />
definition of how to administer a PostgreSQL database. For complete<br />
documentation refer to the [http://www.postgresql.org/docs/manuals/ PostgreSQL manuals].<br />
<br />
= Oracle =<br />
<br />
== Brief description: ==<br />
<br />
* An Oracle database server consists of an Oracle instance and an Oracle database.<br />
* An Oracle instance consists of the Oracle background processes and the allocated memory within the shared global area (SGA) and the program global area (PGA).<br />
* The Oracle background processes consist of the following:<br />
** Database Writer Process (DBWn)<br />
** Log Writer Process (LGWR)<br />
** Checkpoint Process (CKPT)<br />
** System Monitor Process (SMON)<br />
** Process Monitor Process (PMON)<br />
** Recoverer Process (RECO)<br />
** Archiver Processes (ARCn)<br />
* An Oracle database consists of the database datafiles, control files, redo log files, archive log files, and parameter file.<br />
* To remotely access an Oracle database, there exists a separate process referred to as the Oracle listener.<br />
* In the Dedicated Server configuration (versus the Shared Server configuration) every established database session has its own process executing on the server.<br />
<br />
To keep things simple any comparisons with an Oracle database will always refer to a single instance managing a single database, RAC and Data Guard will not be mentioned. Note: PostgreSQL also has the concept of a warm standby (since 8.2) with the shipping of archive logs (introduced in 8.0).<br />
<br />
= PostgreSQL =<br />
<br />
== Database Server Processes ==<br />
<br />
The database server program postgres are all of the server processes. There are no separately named processes like in Oracle for the different duties within the database environment. If you were to look at the process list (ps) the name of the processes would be postgres. However, on most platforms, PostgreSQL modifies its command title so that individual server processes can readily be identified. You may need to adjust the parameters used for commands such as ps and top to show these updated titles in place of the process name ("postgres").<br />
<br />
The processes seen in a process list can be some of the following:<br />
<br />
* Master process - launches the other processes, background and session processes.<br />
* Writer process - background process that coordinates database writes, log writes and checkpoints.<br />
* Stats collector process - background process collecting information about server activity.<br />
* User session processes.<br />
<br />
The server processes communicate with each other using semaphores and shared memory to ensure data integrity throughout concurrent data access.<br />
<br />
== PostgreSQL Database Cluster ==<br />
<br />
Within a server, one or more Oracle instances can be built. The databases are separate from one another usually sharing only the Oracle listener process. PostgreSQL has the concept of a ''database cluster''. A database cluster is a collection of databases that is stored at a common file system location (the "data area"). It is possible to have multiple database clusters, so long as they use different data areas and different communication ports.<br />
<br />
The processes along with the file system components are all shared within the database cluster. All the data needed for a database cluster is stored within the cluster's data directory, commonly referred to as ''PGDATA'' (after the name of the environment variable that can be used to define it). The PGDATA directory contains several subdirectories and configuration files.<br />
<br />
The following are some of the cluster configuration files:<br />
<br />
* postgresql.conf - Parameter or main server configuration file.<br />
* pg_hba.conf - Client authentication configuration file.<br />
* pg_ident.conf - Map from OS account to PostgreSQL account file.<br />
<br />
The cluster subdirectories:<br />
<br />
* base - Subdirectorycontaining per-database subdirectories<br />
* global - Subdirectory containing cluster-wide tables<br />
** pg_auth - Authorization file containing user and role definitions.<br />
** pg_control - Control file.<br />
** pg_database - Information of databases within the cluster.<br />
* pg_clog - Subdirectory containing transaction commit status data<br />
* pg_multixact - Subdirectory containing multitransaction status data (used for shared row locks)<br />
* pg_subtrans - Subdirectory containing subtransaction status data<br />
* pg_tblspc - Subdirectory containing symbolic links to tablespaces<br />
* pg_twophase - Subdirectory containing state files for prepared transactions<br />
* pg_xlog - Subdirectory containing WAL (Write Ahead Log) files<br />
<br />
By default, for each database in the cluster there is a subdirectory within PGDATA/base, named after the database's OID (object identifier) in pg_database. This subdirectory is the default location for the database's files; in particular, its system catalogs are stored there. Each table and index is stored in a separate file, named after the table or index's filenode number, which can be found in pg_class.relfilenode.<br />
<br />
Several components that Oracle DBAs usually equate to one database are shared between databases within a PostgreSQL cluster, including the parameter file, control file, redo logs, tablespaces, accounts, roles, and background processes.<br />
<br />
== Tablespaces and Object Data Files ==<br />
<br />
PostgreSQL introduced tablespace management in version 8.0. The physical representation of a tablespace within PostgreSQL is simple: it is a directory on the file system, and the mapping is done via symbolic links.<br />
<br />
When a database is created, the default tablespace is where by default all of the database objects are stored. In Oracle this would be similar to the System, User, and Temporary tablespaces. If no default tablespace is defined during creation, the data files will go into a subdirectory of the PGDATA/base. Preferably the location of the system catalog information and the application data structures would reside in separately managed tablespaces. This is available.<br />
<br />
As in Oracle, the definition of a PostgreSQL table determines which tablespace the object resides. However, there exists no size limitation except physical boundaries placed on the device by the OS.<br />
<br />
The individual table's data is stored within a file within the tablespace (or directory). The database software will split the table across multiple datafiles in the event the table's data surpasses 1 GB.<br />
<br />
Since version 8.1, it's possible to partition a table over separate (or the same) tablespaces. This is based on PostgreSQL's table inheritance feature, using a capability of the query planner referred to as constraint exclusion.<br />
<br />
There exists no capacity for separating out specific columns (like LOBs) into separately defined tablespaces. However, in addition to the data files that represent the table (in multiples of 1 GB) there is a separation of data files for columns within a table that are TOASTed. The PostgreSQL storage system called TOAST (The Oversized-Attribute Storage Technique) automatically stores values larger than a single database page into a secondary storage area per table. The TOAST technique allows for data columns up to 1 GB in size.<br />
<br />
As in Oracle, the definition of an index determines which tablespace it resides within. Therefore, it is possible to gain the performance advantage of separating the disks that a table's data versus its indexing reside, relieving I/O contention during data manipulation.<br />
<br />
In Oracle there exists temporary tablespaces where sort information and temporary evaluation space needed for distinct statements and the like are used. PostgreSQL does not have this concept of a temporary tablespace; however it does require storage to be able to perform these activities as well. Within the "default" tablespace of the database (defined at database creation) there is a directory called pgsql_tmp. This directory holds the temporary storage needed for the evaluation. The files that get created within the directory exist only while the SQL statement is executing. They grow very fast, and are most likely not designed for space efficiency but rather speed. Be aware that disk fragmentation could result from this, and there needs to be sufficient space on the disk to support the user queries. With the release of 8.3, there are definitions of temporary tablespaces using the parameter ''temp_tablespaces''.<br />
<br />
== REDO and Archiving ==<br />
<br />
PostgreSQL uses ''Write-Ahead Logging'' (WAL) as its approach to transaction logging. WAL's central concept is that changes to data files (where tables and indexes reside) must be written only after those changes have been logged, that is, when log records describing the changes have been flushed to permanent storage. If we follow this procedure, we do not need to flush data pages to disk on every transaction commit, because we know that in the event of a crash we will be able to recover the database using the log: any changes that have not been applied to the data pages can be redone from the log records. (This is roll-forward recovery, also known as REDO.)<br />
<br />
PostgreSQL maintains its (WAL) in the ''pg_xlog'' subdirectory of the cluster's data directory.<br />
<br />
WAL was introduced into PostgreSQL in version 7.1. To maintain database consistency in case of a failure, previous releases forced all data modifications to disk before each transaction commit. With WAL, only one log file must be flushed to disk, greatly improving performance while adding capabilities like Point-In-Time Recovery and transaction archiving.<br />
<br />
A PostgreSQL system theoretically produces an indefinitely long sequence of WAL records. The system physically divides this sequence into WAL segment files, which are normally 16MB apiece. The system normally creates a few segment files and then "recycles" them by renaming no-longer-needed segment files to higher segment numbers. If you were to perform a listing of the pg_xlog directory there would always be a handful of files changing names over time.<br />
<br />
To add archiving of the WAL files there exists a parameter within the parameter file where a command is added to execute the archival process. Once this is done, Operation System "on-line" backups even become available by executing the ''pg_start_backup'' and the ''pg_stop_backup'' commands, which suspend and resume writing to the datafiles while continuing to write the transactions to the WAL files and executing the archival process.<br />
<br />
Inclusion of WAL archiving and the on-line backup commands were added in version 8.0.<br />
<br />
== Rollback or Undo ==<br />
<br />
It is interesting how the dynamic allocation of disk space is used for the storage and processing of records within tables. The files that represent the table grow as the table grows. It also grows with transactions that are performed against it. In Oracle there is a concept of rollback or undo segments that hold the information for rolling back a transaction. In PostgreSQL the data is stored within the file that represents the table. So when deletes and updates are performed on a table, the file that represents the object will contain the previous data. This space gets reused but to force recovery of used space, a maintenance process called ''vacuum'' must be executed.<br />
<br />
== Server Log File ==<br />
<br />
Oracle has the alert log file. PostgreSQL has the server log file. A configuration option would even have the connection information we normally see within the Oracle's listener.log appear in PostgreSQL's server log. The parameters within the server configuration file (postgresql.conf) determine the level, location, and name of the log file.<br />
<br />
To help with the maintenance of the server log file (it grows rapidly), there exists functionality for rotating the server log file. Parameters can be set to determine when to rotate the file based on the size or age of the file. Management of the old files is then left to the administrator.<br />
<br />
== Applications ==<br />
<br />
The command ''initdb'' creates a new PostgreSQL database cluster.<br />
<br />
The command ''psql'' starts the terminal-based front-end to PostgreSQL or SQL command prompt. Queries and commands can be executed interactively or through files. The psql command prompt has several attractive features:<br />
<br />
* Thorough on-line help for both the psql commands and the SQL syntax.<br />
* Command history and line editing.<br />
* SQL commands could exist on multiple lines and are executed only after the semi-colon (;).<br />
* Several SQL commands separated by semi-colons could be entered on a single line.<br />
* Flexible output formatting.<br />
* Multiple object description commands that are superior to Oracle's DESCRIBE.<br />
<br />
Depending on the security configurations of the environments, connections can be established locally or remotely through TCP/IP. Due to these separate security connections passwords may or may not be required to connect.<br />
<br />
The command ''pg_ctl'' is a utility for displaying status, starting, stopping, or restarting the PostgreSQL database server (postgres). Although the server can be started through the postgres executable, pg_ctl encapsulates tasks such as redirecting log output, properly detaching from the terminal and process group, and providing options for controlled shutdown.<br />
<br />
The commands ''pg_dump'' and ''pg_restore'' are utilities designed for exporting and importing the contents of a PostgreSQL database. Dumps can be output in either script or archive file formats. The script file format creates plain-text files containing the SQL commands required to reconstruct the database to the state it was at the time it was generated. The archive file format creates a file to be used with pg_restore to rebuild the database.<br />
<br />
The archive file formats are designed to be portable across architectures. Historically, any type of upgrade to the PostgreSQL software would require a pg_dump of the database prior to the upgrade. Then a pg_restore after the upgrade. Now, for minor releases (i.e., the third decimal – 8.2.x) upgrades can be done in place. However, changing versions at the first or second decimal still requires a pg_dump/pg_restore.<br />
<br />
There exists a graphical tool called [http://www.pgadmin.org/ ''pgAdmin III''] developed separately. It is distributed with the Linux and Windows versions of PostgreSQL. Connection to a database server can be established remotely to perform administrative duties. Because the tool is designed to manage all aspects of the database environment, connection to the database must be through a super user account.<br />
<br />
The pgAdmin III tool has the following standard attractive features:<br />
<br />
* Intuitive layout<br />
* Tree structure for creating and modifying database objects<br />
* Reviewing and saving of SQL when altering or creating objects<br />
<br />
[[Category:Oracle]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Warm_Standby&diff=10563Warm Standby2010-04-20T03:34:11Z<p>Schmiddy: copyediting: use 'standbyhost' consistently, 2ndQ section of "known bugs" no longer exists?</p>
<hr />
<div>__NOTOC__<br />
There are a couple available Projects available to help you setup a warm standby system: <br />
<br />
* Use the walmgr.py portion of Skype's [https://developer.skype.com/SkypeGarage/DbProjects/SkyTools SkyTools] package which will handle PITR backups from a primary to a single slave<br />
* Utilize Command Prompt's [https://projects.commandprompt.com/public/pitrtools PITR tools] to set everything up<br />
<br />
But to actually get a warm standby up manually is actually a pretty simple process. The following are notes only and intended to help your understanding. If you want to get this working correctly then please follow the manual, which is comprehensive and accurately maintained.<br />
<br />
[http://www.postgresql.org/docs/current/static/warm-standby.html Warm Standby Manual]<br />
<br />
== Pre-process recommendations ==<br />
*Use [http://www.postgresql.org/docs/current/static/pgstandby.html pg_standby] for your restore_command in the recovery.conf file on the standby. pg_standby is included in PostgreSQL 8.3, and you can copy the source from there to compile it for 8.2 yourself. It isn't compatible with 8.1. <br />
*Set up your standby host's environment and directory structure exactly the same as your primary. Otherwise you'll need to spend time changing any symlinks you've created on the primary for xlogs, tablespaces, or whatnot which is really just opportunity for error.<br />
*Pre-configure both the postgresql.conf and recovery.conf files for your standby. I usually keep all of my different config files for all of my different servers in a single, version-controlled directory that I can then check out and symlink to. Again, consistent environment & directory setups make symlinks your best friend.<br />
*Use ssh keys for simply, and safely, transferring files between hosts.<br />
*Follow all of the advice in the manual with respect to handling errors.<br />
<br />
== Outline of steps to get warm standby working ==<br />
*Set archive_command in your postgresql.conf, rysnc is a popular choice or you can just use one of the examples from the docs. I use: <br />
<code><pre><br />
rsync -a %p postgres@standbyhost:/path/to/wal_archive/%f<br />
</pre></code><br />
**You must use a command here that does atomic copies, meaning that the file will never appear under the destination filename until it has been completely copied over. This keeps the standby server from trying to read a partial file. rsync is known to work. A notable command that isn't atomic is scp. If you want to use scp for this purpose, you will need to transfer files into another directory on the secondary, then move them to where the restore command looks for them after the transfer is complete.<br />
***If you're using pg_standby, it will refuse to apply files unless they are the right length, which lowers the risk of non-atomic copies being applied. On Windows it even sleeps a bit after that to give time for things to settle. Performing the copy non-atomically is still a bad idea you should avoid.<br />
*Reload your config -- either: SELECT pg_reload_conf(); from psql or: pg_ctl reload -D data_dir/<br />
*Verify that the WALs are being shipped to their destination.<br />
*In psql, SELECT pg_start_backup('some_label');<br />
*Run your base backup. Again, rsync is good for this with something as simple as: <br />
<code><pre><br />
rsync -a --progress /path/to/data_dir/* postgres@standbyhost:/path/to/data_dir/ <br />
</pre></code><br />
*I'd suggest running this in a screen term window, the --progress flag will let you watch to see how far along the rsync is. The -a flag will preserve symlinks as well as all file permissions & ownership.<br />
*In psql, SELECT pg_stop_backup();<br />
**This drops a file to be archived that will have the same name as the first WAL shipped after the call to pg_start_backup() with a .backup suffix. Inside will be the start & stop WAL records defining the range of WAL files needed to be replayed before you can consider bringing the standby out of recovery.<br />
*Drop in, or symlink, your recovery.conf file in the standby's data_dir.<br />
**The restore command should use pg_standby (it's help/README are simple and to the point). I'd recommend redirecting all output from pg_standby to a log file that you can then watch to verify that everything is working correctly once you've started things.<br />
*Drop in, or symlink, your standby's postgresql.conf file.<br />
**If you don't symlink your pg_xlog directory to write WALs to a separate drive, you can safely delete everything under data_dir/pg_xlog on the standby host.<br />
*Start the standby db server with a normal: pg_ctl start -D /path/to/data_dir/<br />
*run a: tail -f on your standby log and watch to make sure that it's replaying logs. If everything's cool you'll see some info on each WAL file, in order, that the standby looks for along with 'success' messages. If it can't find the files for some reason, you'll see repeated messages like: 'WAL file not present yet. Checking for trigger file...' (assuming you set up pg_standby to look for a trigger file in your recovery_command).<br />
<br />
Execute this entire process at least a couple times, bringing up the standby into normal operations mode once it's played through all of the necessary WAL files (as noted in the .backup file) so that you can connect to it and verify that everything looks good, before doing all of this and leaving it running indefinitely. Once you do it a couple times, it becomes dirt simple. <br />
<br />
== Adjusting frequency of WAL updates in 8.1 ==<br />
<br />
Often people want to know that their secondary is never more than some amount behind the primary. The archive_timeout feature introduced into 8.2 allows doing that. If you're using WAL replication with 8.1, you can force 16MB worth of WAL activity that doesn't leave any changes behind with a hack like this:<br />
<br />
<code><pre><br />
create table xlog_switch as<br />
select '0123456789ABCDE' from generate_series(1,1000000);<br />
drop table xlog_switch;<br />
</pre></code><br />
<br />
If you put that into cron etc. to run via psql and you can make the window for log shipping as fine as you'd like even with no activity. <br />
If you do it too often you're increasing the odds it will interfere with real transactions though and it will use up more disk space; every couple of minutes is probably as often as you'd want to do this. Using archive_timeout doesn't have this issue, the manual suggests it can be set to only a few seconds if necessary.<br />
<br />
== Additional resources ==<br />
*[http://www.kennygorman.com/wordpress/?p=249 pg_standby lag monitoring]<br />
*[http://scale-out-blog.blogspot.com/2009/02/simple-ha-with-postgresql-point-in-time.html Simple HA with PITR]<br />
*[http://www.travishegner.com/2009/06/postgresql-83-warm-stand-by-replication.html PostgreSQL 8.3 Warm Stand-by Replication]: tutorial with Ubuntu specifics<br />
*[http://michsan.blogspot.com/2008/08/using-pgstandby-for-high-availability.html Using pg_standby for high availability of Postgresql]: tutorial that covers Debian, using 8.3 pg_standby on 8.2<br />
*Source material: <br />
** [http://archives.postgresql.org/pgsql-general/2008-01/msg01587.php warm standby examples] <br />
** [http://archives.postgresql.org/sydpug/2006-10/msg00001.php Creating an 8.2 warm-standby demo system]<br />
** [http://archives.postgresql.org/pgsql-general/2007-06/msg00015.php PITR Base Backup on an idle 8.1 server]<br />
<br />
[[Category:Replication]][[Category:Backup]]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Query_progress_indication&diff=9709Query progress indication2010-01-29T20:18:44Z<p>Schmiddy: simple pg_stat_activity check for whether query is waiting on lock?</p>
<hr />
<div>Postgres currently doesn't give any meaningful feedback about the query execution process to the user. This would be valuable for:<br />
# reporting whether the execution of a query is blocked on a lock; many users in #postgresql are confused about why their query takes so long to execute, when in fact it is blocked on a lock -- ''Isn't this easily gleaned from the "waiting" boolean in pg_stat_activity?''<br />
# reporting the progress of a long-running analysis query. When interactively executing complex, long-running queries, providing feedback about how long they will take to execute (and approximate answers in the mean-time) would be cool<br />
# reporting the progress of long-running utility queries. For example, it can be difficult to predict whether the runtime of a large CREATE INDEX will be minutes or hours; similarly, the progress of manual VACUUMs can be difficult to estimate accurately<br />
<br />
This has been discussed in the DBMS literature (see below). Offhand, I think implementing #1 and #3 would be pretty doable (perhaps with FE/BE changes), but #2 would take some Thought.<br />
<br />
== References ==<br />
Relevant papers:<br />
* [http://www.cs.toronto.edu/~cmishra/ProgressBars.pdf A Lightweight Online Framework For Query Progress Indicators]<br />
* [http://www.cs.wisc.edu/~gangluo/workload_final.pdf Multi-query SQL Progress Indicators]<br />
* [http://www.cs.wisc.edu/~gangluo/interface.pdf Toward a Progress Indicator for Database Queries]<br />
* [http://www.cs.wisc.edu/~gangluo/PI.pdf Increasing the Accuracy and Coverage of SQL Progress Indicators]<br />
<br />
Relevant pgsql-hackers threads:<br />
* [http://archives.postgresql.org/pgsql-hackers/2006-07/msg00719.php Progress bar updates]</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Installation_and_Administration_Best_practices&diff=8173Installation and Administration Best practices2009-09-25T18:23:06Z<p>Schmiddy: /* Directories Location Recommended */ cleaning up this whole section</p>
<hr />
<div>=== Proposal ===<br />
<br />
This page is under construction.<br />
<br />
This page will contain information about the best way to install and maintain a Postgresql database, including environment variables, paths and other relevant stuff.<br />
<br />
Fell free to add your suggestions and knowledge to make it a 'hands on' resorce.<br />
<br />
=== Self compiled vs. package distributed ===<br />
<br />
Before installing, consider whether to use packages distributed with your operating system or whether to roll your own. (Compiling PostgreSQL on Windows might be a difficult task. It's quite easy on Linux/Unix boxes.)<br />
<br />
Let's compare using a pre-built package and compiling PostgreSQL yourself. (Note: This is a rather Linux-centric view. For Windows, you'll likely want to use the binary packages provided for each release.)<br />
<br />
{| border="1" cellpadding="5" cellspacing="0"<br />
!Using pre-built package from distribution<br />
!Compiling yourself.<br />
|-<br />
|Very easy to install - just use your package manager.<br />
|You might need to install gcc and some development packages just for building PostgreSQL.<br />
|-<br />
|Installation is dependent on distribution (location of config files, initial tablespace).<br />
|You may install everything in one place, just where you want it.<br />
|-<br />
|Startup-scripts are included and supposed to work.<br />
|You need to provide your own system startup scripts.<br />
|-<br />
|The packages might be out of date or new minor versions might not become available frequently.<br />
|You are free to use the latest stable version and perform upgrades at your will.<br />
|-<br />
|The package management knows about the PostgreSQL installation and will update it.<br />
|Your package management doesn't know anything about the installation. Dependent libraries might get uninstalled or replaced by newer, incompatible versions. ('''Note:''' This is rather unlikely. I've never seen it happen. PostgreSQL doesn't depend on any strange or fast-evolving packages.)<br />
|}<br />
<br />
=== Compiling and installing in Solaris ===<br />
<br />
TODO: Add a workaround for the most common issues.<br />
<br />
<br />
== Compiling in Solaris using Sun Studio 12 ==<br />
<br />
<code><br />
./configure --prefix=/usr/local/pgsql84 CC=/opt/SUNWspro/bin/cc 'CFLAGS=-xO3 -xarch=native -xspace -W0,-Lt -W2,-Rcond_elim -Xa -xildoff -xc99=none -xCC' --datadir=/usr/local/pgsql84/data84 --enable-dtrace --enable-cassert --with-perl --with-python --with-libxml --with-libxslt --with-ossp-uuid --without-readline<br />
</code><br />
<br />
Notes: <br />
* OSol don't have readline library<br />
* Maybe you don't need --enable-cassert option.<br />
<br />
== Issues ==<br />
<br />
UUID: There is a problem with UUID library in Open Solaris 200911. If some one have a workaround in this, please post it.<br />
<br />
=== Multiple Versions on the same host ===<br />
<br />
If you have to install multiple PostgreSQL versions at the same host, compile from source and call configure like this:<br />
<br />
./configure --prefix=/opt/postgresql-8.2.11 --with-pgport=8200<br />
<br />
That way, you never need to worry what version you are talking with - you just look at the port number.<br />
<br />
Other way is changing port in postgresql.conf. Beware of that if you have am own init script, remeber to change values of PGDATA and PGUSER.<br />
<br />
=== Making sure it starts up at system boot time ===<br />
<br />
TODO: Provide a default init script (if there's not already one in contrib/).<br />
<br />
<br />
=== Recommended values to be changed in big servers ===<br />
<br />
In Linux, the SHMMAX value can often be set rather low, especially in older 32-bit distributions. Depending on your PostgreSQL configuration, you might need to tweak the values of SHMMAX and/or SHMALL.<br />
<br />
Example of a high configuration:<br />
<code><br />
# /etc/sysctl.conf<br />
fs.file-max = 32768<br />
kernel.shmmax = 1073741824<br />
kernel.shmall = 536870912<br />
</code><br />
<br />
How to calculate? One of the equations is: (FIXME: does this formula refer to SHMMAX?)<br />
<br />
<code><br />
250kb + 8.2kb * shared_buffers +14.2kb * max_connections<br />
</code><br />
<br />
The SHMMAX variable [http://ps-ax.com/shared-mem.html controls] the maximum amount of memory to be allocated for shared memory use. If you try to assign high values for e.g. the shared_buffers GUC in PostgreSQL without adjusting SHMMAX, you might see an [http://www.caktusgroup.com/blog/2009/08/13/setting-postgresqls-shmmax-in-mac-os-x-105-leopard/ error message] in Postgres' log like " ... Failed system call was shmget ... usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter", and you'll have to adjust SHMMAX upward accordingly.<br />
<br />
More information about SHMMAX [http://vista.intersystems.com/csp/docbook/DocBook.UI.Page.cls?KEY=GCI_unixparms can be found here].<br />
<br />
=== Directory Locations Recommended ===<br />
==== WAL Directory ====<br />
Write Ahead Logging (WAL) is a critical part of Postgres' operation. Due to Postgres' [http://www.postgresql.org/docs/current/static/wal-intro.html use of the WAL] to ensure data consistency, the WAL will receive significant I/O activity. Especially if your PGDATA directory isn't located on a capable RAID array, you might consider relocating the WAL (pg_xlog directory) to a separate disk to ease I/O load on the rest of the database. The WAL should be [http://archives.postgresql.org/pgsql-novice/2003-09/msg00259.php written in perfectly sequential fashion], so the I/O savings from relocating this directory can be substantial. However, be aware of additional [http://archives.postgresql.org/pgsql-sql/2006-12/msg00039.php data consistency risks] you will take on by relocating the WAL. <br />
<br />
(FIXME: awkward and vague paragraph) Configuration: If you have the PGDATA in very different location, you maybe want to have a /etc/postgresql/data<number>. Or, if you have several versions for testing purposes you maybe want to have 'debian' like tree (/etc/postgresql/<version>/<pgdata_number>). The default way is have config files inside the PGDATA.<br />
<br />
See [http://www.postgresql.org/files/documentation/books/aw_pgsql/hw_performance/node10.html here] for information on offloading various PostgreSQL data onto different drives.<br />
<br />
=== Versioning sql scripts and configuration files ===<br />
<br />
As you are doing right now (versioning the sql scripts), other best practice is to version the configuration files. Not only a simple versioning, remember that you have several envoronments (Development, Test, Production).<br />
<br />
Other good practice, is to have versioned the DBA modifications in a separated script (SET STORAGE modifications, special indexes and rules, etc).<br />
<br />
TODO: Paste a example.<br />
<br />
<br />
=== Backup and Recovery strategies ===<br />
<br />
TODO: How to perform those tasks.<br />
<br />
<br />
=== Users athentications ===<br />
<br />
Please for more information, read article [http://wiki.postgresql.org/wiki/Client_Authentication]. <br />
<br />
One recommended installation for access by host mode is the md5 method for encrypted password. We can draw something like this:<br />
<br />
<code><pre><br />
host all all 192.168.1.0/24 md5<br />
</pre></code><br />
<br />
This mode, reduces the the access to the IP addreses that are included in 192.168.1.x and using the correct password for the user.<br />
Adding to this restriction, you must remember that in the postgres.conf you should modify listen_addresses variable [http://www.postgresql.org/docs/current/static/runtime-config-connection.html listen_addresses].<br />
<br />
Remember that you can create users wiht '''VALID UNTIL''' option. When you create a user you can calculate the timestamp with something like this:<br />
<br />
<code><br />
SELECT (CURRENT_DATE+1)::timestamp;<br />
</code><br />
<br />
You might replace 1 with the number of days that you want to enable the user(7=1 week, 30=month, etc.). Then, copy result in the ALTER or CREATE statement.<br />
<br />
Remember: The first step before change '''trust''' for other method based in passwords, you should assign one to the user. <br />
<br />
Changes in the pg_hba.conf only need '''reload''' signal. So, there is no need downtime.<br />
<br />
You can create superusers for the databases, but remember that if youwant to restrict the access to the databases, the superusers still have permissions to drop the other BD's and other mainteneance tasks. Try to reduce the number of superusers or almost one.<br />
<br />
=== LDAP Auth ===<br />
<br />
TODO: explain the best ways to put on work this method<br />
[http://wiki.postgresql.org/wiki/Client_Authentication]<br />
<br />
=== Monitoring Index and table accesing and use ===<br />
<br />
TODO: Explain how to monitor the use of tables and indexes to apply modification to the way to storage them.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Installation_and_Administration_Best_practices&diff=8168Installation and Administration Best practices2009-09-25T17:59:39Z<p>Schmiddy: /* Recommended values to be changed in big servers */ Adjusting awkward phrasing, improving links</p>
<hr />
<div>=== Proposal ===<br />
<br />
This page is under construction.<br />
<br />
This page will contain information about the best way to install and maintain a Postgresql database, including environment variables, paths and other relevant stuff.<br />
<br />
Fell free to add your suggestions and knowledge to make it a 'hands on' resorce.<br />
<br />
=== Self compiled vs. package distributed ===<br />
<br />
Before installing, consider whether to use packages distributed with your operating system or whether to roll your own. (Compiling PostgreSQL on Windows might be a difficult task. It's quite easy on Linux/Unix boxes.)<br />
<br />
Let's compare using a pre-built package and compiling PostgreSQL yourself. (Note: This is a rather Linux-centric view. For Windows, you'll likely want to use the binary packages provided for each release.)<br />
<br />
{| border="1" cellpadding="5" cellspacing="0"<br />
!Using pre-built package from distribution<br />
!Compiling yourself.<br />
|-<br />
|Very easy to install - just use your package manager.<br />
|You might need to install gcc and some development packages just for building PostgreSQL.<br />
|-<br />
|Installation is dependent on distribution (location of config files, initial tablespace).<br />
|You may install everything in one place, just where you want it.<br />
|-<br />
|Startup-scripts are included and supposed to work.<br />
|You need to provide your own system startup scripts.<br />
|-<br />
|The packages might be out of date or new minor versions might not become available frequently.<br />
|You are free to use the latest stable version and perform upgrades at your will.<br />
|-<br />
|The package management knows about the PostgreSQL installation and will update it.<br />
|Your package management doesn't know anything about the installation. Dependent libraries might get uninstalled or replaced by newer, incompatible versions. ('''Note:''' This is rather unlikely. I've never seen it happen. PostgreSQL doesn't depend on any strange or fast-evolving packages.)<br />
|}<br />
<br />
=== Compiling and installing in Solaris ===<br />
<br />
TODO: Add a workaround for the most common issues.<br />
<br />
<br />
== Compiling in Solaris using Sun Studio 12 ==<br />
<br />
<code><br />
./configure --prefix=/usr/local/pgsql84 CC=/opt/SUNWspro/bin/cc 'CFLAGS=-xO3 -xarch=native -xspace -W0,-Lt -W2,-Rcond_elim -Xa -xildoff -xc99=none -xCC' --datadir=/usr/local/pgsql84/data84 --enable-dtrace --enable-cassert --with-perl --with-python --with-libxml --with-libxslt --with-ossp-uuid --without-readline<br />
</code><br />
<br />
Notes: <br />
* OSol don't have readline library<br />
* Maybe you don't need --enable-cassert option.<br />
<br />
== Issues ==<br />
<br />
UUID: There is a problem with UUID library in Open Solaris 200911. If some one have a workaround in this, please post it.<br />
<br />
=== Multiple Versions on the same host ===<br />
<br />
If you have to install multiple PostgreSQL versions at the same host, compile from source and call configure like this:<br />
<br />
./configure --prefix=/opt/postgresql-8.2.11 --with-pgport=8200<br />
<br />
That way, you never need to worry what version you are talking with - you just look at the port number.<br />
<br />
Other way is changing port in postgresql.conf. Beware of that if you have am own init script, remeber to change values of PGDATA and PGUSER.<br />
<br />
=== Making sure it starts up at system boot time ===<br />
<br />
TODO: Provide a default init script (if there's not already one in contrib/).<br />
<br />
<br />
=== Recommended values to be changed in big servers ===<br />
<br />
In Linux, the SHMMAX value can often be set rather low, especially in older 32-bit distributions. Depending on your PostgreSQL configuration, you might need to tweak the values of SHMMAX and/or SHMALL.<br />
<br />
Example of a high configuration:<br />
<code><br />
# /etc/sysctl.conf<br />
fs.file-max = 32768<br />
kernel.shmmax = 1073741824<br />
kernel.shmall = 536870912<br />
</code><br />
<br />
How to calculate? One of the equations is: (FIXME: does this formula refer to SHMMAX?)<br />
<br />
<code><br />
250kb + 8.2kb * shared_buffers +14.2kb * max_connections<br />
</code><br />
<br />
The SHMMAX variable [http://ps-ax.com/shared-mem.html controls] the maximum amount of memory to be allocated for shared memory use. If you try to assign high values for e.g. the shared_buffers GUC in PostgreSQL without adjusting SHMMAX, you might see an [http://www.caktusgroup.com/blog/2009/08/13/setting-postgresqls-shmmax-in-mac-os-x-105-leopard/ error message] in Postgres' log like " ... Failed system call was shmget ... usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter", and you'll have to adjust SHMMAX upward accordingly.<br />
<br />
More information about SHMMAX [http://vista.intersystems.com/csp/docbook/DocBook.UI.Page.cls?KEY=GCI_unixparms can be found here].<br />
<br />
=== Directories Location Recommended ===<br />
<br />
Logs: Sometimes logs could be your best wy to determine an issue, but sometimes could add some overhead to your servers. If your RAID isn't enough powerful, you could have a separated storage for them (a disc or something else). Usually , Postgresql logs are in pg_log directory inside the PGDATA. You can have a soft link to this storage or directly an absolute path in postgresql.conf.<br />
<br />
WAL: WAL is an important part of our databases, so it need the same level of attention. If you don't have RAID, maybe the first thing to think about is to store WAL files in other discs. <br />
<br />
Configuration: If you have the PGDATA in very different location, you maybe want to have a /etc/postgresql/data<number>. Or, if you have several versions for test porpuses you maybe want to have 'debian' like tree (/etc/postgresql/<version>/<pgdata_number>). The default way is have config files inside the PGDATA.<br />
<br />
=== Versioning sql scripts and configuration files ===<br />
<br />
As you are doing right now (versioning the sql scripts), other best practice is to version the configuration files. Not only a simple versioning, remember that you have several envoronments (Development, Test, Production).<br />
<br />
Other good practice, is to have versioned the DBA modifications in a separated script (SET STORAGE modifications, special indexes and rules, etc).<br />
<br />
TODO: Paste a example.<br />
<br />
<br />
=== Backup and Recovery strategies ===<br />
<br />
TODO: How to perform those tasks.<br />
<br />
<br />
=== Users athentications ===<br />
<br />
Please for more information, read article [http://wiki.postgresql.org/wiki/Client_Authentication]. <br />
<br />
One recommended installation for access by host mode is the md5 method for encrypted password. We can draw something like this:<br />
<br />
<code><pre><br />
host all all 192.168.1.0/24 md5<br />
</pre></code><br />
<br />
This mode, reduces the the access to the IP addreses that are included in 192.168.1.x and using the correct password for the user.<br />
Adding to this restriction, you must remember that in the postgres.conf you should modify listen_addresses variable [http://www.postgresql.org/docs/current/static/runtime-config-connection.html listen_addresses].<br />
<br />
Remember that you can create users wiht '''VALID UNTIL''' option. When you create a user you can calculate the timestamp with something like this:<br />
<br />
<code><br />
SELECT (CURRENT_DATE+1)::timestamp;<br />
</code><br />
<br />
You might replace 1 with the number of days that you want to enable the user(7=1 week, 30=month, etc.). Then, copy result in the ALTER or CREATE statement.<br />
<br />
Remember: The first step before change '''trust''' for other method based in passwords, you should assign one to the user. <br />
<br />
Changes in the pg_hba.conf only need '''reload''' signal. So, there is no need downtime.<br />
<br />
You can create superusers for the databases, but remember that if youwant to restrict the access to the databases, the superusers still have permissions to drop the other BD's and other mainteneance tasks. Try to reduce the number of superusers or almost one.<br />
<br />
=== LDAP Auth ===<br />
<br />
TODO: explain the best ways to put on work this method<br />
[http://wiki.postgresql.org/wiki/Client_Authentication]<br />
<br />
=== Monitoring Index and table accesing and use ===<br />
<br />
TODO: Explain how to monitor the use of tables and indexes to apply modification to the way to storage them.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Installation_and_Administration_Best_practices&diff=8167Installation and Administration Best practices2009-09-25T17:30:54Z<p>Schmiddy: /* Self compiled vs. package distributed */</p>
<hr />
<div>=== Proposal ===<br />
<br />
This page is under construction.<br />
<br />
This page will contain information about the best way to install and maintain a Postgresql database, including environment variables, paths and other relevant stuff.<br />
<br />
Fell free to add your suggestions and knowledge to make it a 'hands on' resorce.<br />
<br />
=== Self compiled vs. package distributed ===<br />
<br />
Before installing, consider whether to use packages distributed with your operating system or whether to roll your own. (Compiling PostgreSQL on Windows might be a difficult task. It's quite easy on Linux/Unix boxes.)<br />
<br />
Let's compare using a pre-built package and compiling PostgreSQL yourself. (Note: This is a rather Linux-centric view. For Windows, you'll likely want to use the binary packages provided for each release.)<br />
<br />
{| border="1" cellpadding="5" cellspacing="0"<br />
!Using pre-built package from distribution<br />
!Compiling yourself.<br />
|-<br />
|Very easy to install - just use your package manager.<br />
|You might need to install gcc and some development packages just for building PostgreSQL.<br />
|-<br />
|Installation is dependent on distribution (location of config files, initial tablespace).<br />
|You may install everything in one place, just where you want it.<br />
|-<br />
|Startup-scripts are included and supposed to work.<br />
|You need to provide your own system startup scripts.<br />
|-<br />
|The packages might be out of date or new minor versions might not become available frequently.<br />
|You are free to use the latest stable version and perform upgrades at your will.<br />
|-<br />
|The package management knows about the PostgreSQL installation and will update it.<br />
|Your package management doesn't know anything about the installation. Dependent libraries might get uninstalled or replaced by newer, incompatible versions. ('''Note:''' This is rather unlikely. I've never seen it happen. PostgreSQL doesn't depend on any strange or fast-evolving packages.)<br />
|}<br />
<br />
=== Compiling and installing in Solaris ===<br />
<br />
TODO: Add a workaround for the most common issues.<br />
<br />
<br />
== Compiling in Solaris using Sun Studio 12 ==<br />
<br />
<code><br />
./configure --prefix=/usr/local/pgsql84 CC=/opt/SUNWspro/bin/cc 'CFLAGS=-xO3 -xarch=native -xspace -W0,-Lt -W2,-Rcond_elim -Xa -xildoff -xc99=none -xCC' --datadir=/usr/local/pgsql84/data84 --enable-dtrace --enable-cassert --with-perl --with-python --with-libxml --with-libxslt --with-ossp-uuid --without-readline<br />
</code><br />
<br />
Notes: <br />
* OSol don't have readline library<br />
* Maybe you don't need --enable-cassert option.<br />
<br />
== Issues ==<br />
<br />
UUID: There is a problem with UUID library in Open Solaris 200911. If some one have a workaround in this, please post it.<br />
<br />
=== Multiple Versions on the same host ===<br />
<br />
If you have to install multiple PostgreSQL versions at the same host, compile from source and call configure like this:<br />
<br />
./configure --prefix=/opt/postgresql-8.2.11 --with-pgport=8200<br />
<br />
That way, you never need to worry what version you are talking with - you just look at the port number.<br />
<br />
Other way is changing port in postgresql.conf. Beware of that if you have am own init script, remeber to change values of PGDATA and PGUSER.<br />
<br />
=== Making sure it starts up at system boot time ===<br />
<br />
TODO: Provide a default init script (if there's not already one in contrib/).<br />
<br />
<br />
=== Recommended values to be changed in big servers ===<br />
<br />
In Linux, the SHMMAX value is setted in a historical value. So, depending on your server, the first value to be changed is SHMMAX and SHMALL.<br />
<br />
Example of a high configuration:<br />
<code><br />
# /etc/sysctl.conf<br />
fs.file-max = 32768<br />
kernel.shmmax = 1073741824<br />
kernel.shmall = 536870912<br />
</code><br />
<br />
How to calculate? One of the equations is:<br />
<code><br />
250kb + 8.2kb * shared_buffers +14.2kb * max_connections<br />
</code><br />
<br />
Why this value is important? This value govern shared memory allocation with other values. If you try to assign high values in Postgresql but you don't touch this values, Postgresql may not run.<br />
<br />
One interest link could be visited [http://vista.intersystems.com/csp/docbook/DocBook.UI.Page.cls?KEY=GCI_unixparms here.]<br />
<br />
=== Directories Location Recommended ===<br />
<br />
Logs: Sometimes logs could be your best wy to determine an issue, but sometimes could add some overhead to your servers. If your RAID isn't enough powerful, you could have a separated storage for them (a disc or something else). Usually , Postgresql logs are in pg_log directory inside the PGDATA. You can have a soft link to this storage or directly an absolute path in postgresql.conf.<br />
<br />
WAL: WAL is an important part of our databases, so it need the same level of attention. If you don't have RAID, maybe the first thing to think about is to store WAL files in other discs. <br />
<br />
Configuration: If you have the PGDATA in very different location, you maybe want to have a /etc/postgresql/data<number>. Or, if you have several versions for test porpuses you maybe want to have 'debian' like tree (/etc/postgresql/<version>/<pgdata_number>). The default way is have config files inside the PGDATA.<br />
<br />
=== Versioning sql scripts and configuration files ===<br />
<br />
As you are doing right now (versioning the sql scripts), other best practice is to version the configuration files. Not only a simple versioning, remember that you have several envoronments (Development, Test, Production).<br />
<br />
Other good practice, is to have versioned the DBA modifications in a separated script (SET STORAGE modifications, special indexes and rules, etc).<br />
<br />
TODO: Paste a example.<br />
<br />
<br />
=== Backup and Recovery strategies ===<br />
<br />
TODO: How to perform those tasks.<br />
<br />
<br />
=== Users athentications ===<br />
<br />
Please for more information, read article [http://wiki.postgresql.org/wiki/Client_Authentication]. <br />
<br />
One recommended installation for access by host mode is the md5 method for encrypted password. We can draw something like this:<br />
<br />
<code><pre><br />
host all all 192.168.1.0/24 md5<br />
</pre></code><br />
<br />
This mode, reduces the the access to the IP addreses that are included in 192.168.1.x and using the correct password for the user.<br />
Adding to this restriction, you must remember that in the postgres.conf you should modify listen_addresses variable [http://www.postgresql.org/docs/current/static/runtime-config-connection.html listen_addresses].<br />
<br />
Remember that you can create users wiht '''VALID UNTIL''' option. When you create a user you can calculate the timestamp with something like this:<br />
<br />
<code><br />
SELECT (CURRENT_DATE+1)::timestamp;<br />
</code><br />
<br />
You might replace 1 with the number of days that you want to enable the user(7=1 week, 30=month, etc.). Then, copy result in the ALTER or CREATE statement.<br />
<br />
Remember: The first step before change '''trust''' for other method based in passwords, you should assign one to the user. <br />
<br />
Changes in the pg_hba.conf only need '''reload''' signal. So, there is no need downtime.<br />
<br />
You can create superusers for the databases, but remember that if youwant to restrict the access to the databases, the superusers still have permissions to drop the other BD's and other mainteneance tasks. Try to reduce the number of superusers or almost one.<br />
<br />
=== LDAP Auth ===<br />
<br />
TODO: explain the best ways to put on work this method<br />
[http://wiki.postgresql.org/wiki/Client_Authentication]<br />
<br />
=== Monitoring Index and table accesing and use ===<br />
<br />
TODO: Explain how to monitor the use of tables and indexes to apply modification to the way to storage them.</div>Schmiddyhttps://wiki.postgresql.org/index.php?title=Talk:Why_PostgreSQL_Instead_of_MySQL_2009&diff=8152Talk:Why PostgreSQL Instead of MySQL 20092009-09-23T20:19:08Z<p>Schmiddy: /* Notes for updates to include */ note on mysqldump</p>
<hr />
<div>==Notes for updates to include==<br />
<br />
* InnoDB doesn't even allow a consistent backup for free? http://krow.livejournal.com/594927.html<br />
** I haven't tried out the 3rd party "Hot Backup" tool mentioned on that page, but AIUI its main advantage (in addition to speed) over mysqldump seems to be in handling mixes of InnoDB + MyISAM tables, and dumping them together in a consistent state. Using the --single-transaction flag of [http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html#option_mysqldump_single-transaction mysqldump] should dump all '''InnoDB tables only''' in a single MySQL database in a consistent state, similar to pg_dump's operation.</div>Schmiddy