PostgreSQL wiki - User contributions [en]

Replication, Clustering, and Connection Pooling

2017-05-04T01:04:44Z

Craigkerstiens: /* Comparison matrix */

==Introduction==
There are many approaches available to scale PostgreSQL beyond running on a single server. An outline of the terminology and basic technologies involved is at [http://www.postgresql.org/docs/current/interactive/high-availability.html High Availability and Load Balancing]. There is a [http://momjian.us/main/writings/pgsql/replication.pdf presentation] covering some of these solutions.

There is no one-size fits all replication software. You have to understand your requirements and how various approaches fit into that. For example, here are two extremes in the replication problem space:

* You have a few servers connected to a local network you want to always keep current for failover and load-balancing purposes. Here you would be considering solutions that are synchronous, eager, and therefore conflict-free.
* Your users take a local copy of the database with them on laptops when they leave the office, make changes while they are away, and need to merge those with the main database when they return. Here you'd want an asynchronous, lazy replication approach, and will be forced to consider how to handle conflicts in cases where the same record has been modified both on the master server and on a local copy.

These are both database replication problems, but the best way to solve them is very different. And as you can see from these examples, replication has a lot of specific terminology that you'll have to understand to figure out what class of solution makes sense for your requirements. A great source for this background is in the
[http://www.postgres-r.org/documentation/terms Postgres-R Terms and Definitions for Database Replication]. The main theoretical topic it doesn't mention is how to resolve conflict resolution in lazy replication cases like the laptop situation, which involves voting and similar schemes.

== Features in the Core of PostgreSQL ==

*[[Hot Standby]]/[[Streaming Replication]] is available as of PostgreSQL 9.0 and provides asynchronous binary replication to one or more standbys. Standbys may also become hot standbys meaning they can be queried as a read-only database. This is the fastest type of replication available as WAL data is sent immediately rather than waiting for a whole segment to be produced and shipped.

*[[Warm Standby]]/Log Shipping is a HA solution which 'replicates' a database cluster to an archive or a warm (can be brought up quickly, but not available for querying) standby server. Overhead is very low and it's easy to set up. This is a simple and appropriate solution if all you care about is continuous backup and short failover times.

* PostgreSQL 9.4's [[Logical Changeset Extraction]] forms the foundation of the [[Bi-Directional Replication]] and [[Logical Log Streaming Replication]] features being added to PostgreSQL.

Historically, the PostgreSQL core team considered replication and clustering technology outside the scope of the main project's focus but this changed in 2008, see the [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00913.php Core Team's statement]. Replication is now a significant focus of ongoing PostgreSQL development.

==Comparison matrix==

This page is being overhauled at [[Clustering]]

{| border="1" cellpadding="1" cellspacing="0" style="font-size: 85%; border: gray solid 1px; border-collapse: collapse; text-align: center; width: 100%; table-layout: fixed;"
|- style="background: #ececec"
! Program
! License
! Maturity
! Replication Method
! Sync
! Connection Pooling
! Load Balancing
! Query Partitioning
|-
! style="text-align:block;" bgcolor="#ececec" | [[PgCluster]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Not production ready
| bgcolor="#ffffaa" | Master-Master
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ddffdd" | Yes
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | '''pgpool-I'''
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Statement-Based Middleware
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Pgpool-II]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | Statement-Based Middleware
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
|-
! style="text-align:block;" bgcolor="#ececec" | [[slony]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Bucardo]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Master, Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Londiste]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Mammoth]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | No longer maintained
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[rubyrep]]
| bgcolor="#ffffaa" | MIT
| bgcolor="#ffffaa" | No longer maintained
| bgcolor="#ffffaa" | Master-Master, Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [https://2ndquadrant.com/en/resources/bdr/ Bi-Directional Replication]
| bgcolor="#ffffaa" | PostgreSQL (BSD)
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | Master-Master<br/>(no triggers needed)
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [https://www.citusdata.com/citus-products/pg-shard pg_shard]
| bgcolor="#ffffaa" | LGPL
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | Statement-based Middleware (as an extension)
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
|-
! style="text-align:block;" bgcolor="#ececec" | [http://2ndquadrant.com/resources/pglogical pglogical]
| bgcolor="#ffffaa" | PostgreSQL
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [http://postgres-xl.org/ Postgres-XL]
| bgcolor="#ffffaa" | PostgreSQL
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | MPP Postgres, scalable writes & reads
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
|-
! style="text-align:block;" bgcolor="#ececec" | [https://www.citusdata.com/ Citus]
| bgcolor="#ffffaa" | AGPL
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | MPP Postgres, scalable writes & reads
| bgcolor="#ffffaa" | Asynchronous or Synchronous
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
|}

==Replication==

Aside from the in-core streaming replication, mentioned above...

*Slony-I: Seems good, single master only, master is a single point of failure, no good failover system for electing a new master or having a failed master rejoin the cluster. Slave databases are mostly for safety or for parallelizing queries for performance. Suffers from O(N^2) communications (N = cluster size). with reasonable sysadmin you can implement failover system yourself. regarding communications, you can cascade the replication to reduce load on the master. If you were implementing a large replication cluster, this would probably be a good idea. Slony is powerful, trigger based, and highly configurable.

* PGCluster: PGCluster (which, incidentally, is not the same as PGCluster-II, a shared-disk solution), which does synchronous multimaster replication. Two single-points failure spots, load balancer and the data replicator. The project has historically looked a bit dead, but they just released a new version and http://pgfoundry.org/projects/pgcluster is up to date (at least downloads page) One major downside to PGCluster is that it uses a modified version of PostgreSQL, and it usually lags a few releases behind.

* http://www.pgpool.net/ pgpool 1/2 is a reasonable solution. it's statement level replication, which has some downsides, but is good for certain things. pgpool 2 has a neat distributed table mechanism which is interesting. You might want to be looking here if you have extremely high ratios of read to write but need to service a huge transaction volume. Supports load-balancing and replication by implementing a proxy that duplicates all updates to all slaves. It can partition data by doing this, and it can semi-intelligently route queries to the appropriate servers.

* "Mammoth Replicator" - BSD - http://www.commandprompt.com/products/mammothreplicator/ - Former proprietary solution, now open source. Uses a central logging process to distribute data changes amongst nodes. Essentially a fork of Postgres, as the changes are written directly into the backend.

* "Bucardo" - BSD License - http://bucardo.org/ - Trigger-based, asynchronous, multi-master or master-slave, written using plperl.

* Cybertec, an Austrian company, offers a proprietary packaging of PGCluster. They simply call it PostgreSQL Multimaster-Replication, see http://www.cybertec.at.

* [[Londiste_Tutorial|Londiste]], a part of [[Skytools]] (https://developer.skype.com/SkypeGarage/DbProjects/SkyTools) which is a collection of replication tools from the Skype people. Purports to be simpler to use than Slony.

* [https://2ndquadrant.com/en/resources/bdr/ BDR (Bi-Directional Replication for PostgreSQL)] - multi-master replication based on log streaming logical replication.

* [https://2ndquadrant.com/en/resources/pglogical/ pglogical] - pglogical is a logical replication system implemented entirely as a PostgreSQL extension. Fully integrated, it requires no triggers or external programs. This alternative to physical replication is a highly efficient method of replicating data using a publish/subscribe model for selective replication.

* [http://www.continuent.com/index.php?option=com_content&task=view&id=212&Itemid=169 Continuent uni/cluster], proprietary and the related Sequoia (jdbc, formerly known as c-jdbc)

* [http://www.postgres-r.org Postgres-R] is still in development. It features eager and thus conflict-free, but async multi-master replication.

* [http://symmetricds.codehaus.org/ SymmetricDS] is an open-source, web-enabled, database independent, data synchronization software application. It uses web and database technologies to replicate tables between relational databases in near real time. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outages. Supports several relational databases, including PostgreSQL. Licensed under Lesser GPL (LGPL).

* DRBD (http://www.drbd.org/), a device driver that replicates disk blocks to other nodes. This works for failover only, not for scaling reads. Easy migration of devices if combined with an NFS export.

* [http://sourceforge.net/projects/daffodilreplica/ Daffodil Replicator]. Supports several relational databases, including PostgreSQL. Licensed under GPL.

* "RubyRep" - MIT License - http://www.rubyrep.org/ - Ruby based, asynchronous, multi-master replication system, which supports Postgres and MySQL.

* "pg_comparator" - BSD License - http://pgfoundry.org/projects/pg-comparator/ - Perl-based, table-level async master-slave "diff" and "patch" method of replication. Low configuration overhead.

* [https://www.citusdata.com/ Citus Data] develops the [https://www.citusdata.com/citus-products/pg-shard pg_shard extension] which can transparently shard a PostgreSQL table across many servers and replicate the shards, and [https://www.citusdata.com/product/citus Citus] which parallelizes queries across many servers and cores for real-time analytics and supports bulk loading, distributed joins, columnar storage and more.

* [http://www.enterprisedb.com/products-services-training/products-overview/tools/ EDB Replication Server]: Provides single and multi-master solutions for read/write scalability, availability, performance, and data integration with Oracle, SQL Server, and Postgres. EDB Replication Server uses Postgres' fast Logical Decoding technology to support large multi-master clusters and single master configurations for a wide variety of data center solutions.

===Inactive projects===
* Slony-II
* PGReplication

==Clustering==

* [https://www.citusdata.com/product/citus Citus] - shards and replicates tables across a scalable, high availability cluster of commodity PostgreSQL servers and parallelizes queries for real-time SQL on big data.

* [http://www.greenplum.org Greenplum Database] - Not so much a replication solution as a way to parallelize queries, and targeted at the data warehousing and big data crowd. Greenplum is tightly integrated with PostgreSQL.

* [http://www.enterprisedb.com/products/gridsql.do GridSQL for EnterpriseDB Advanced Server] (formerly ExtenDB)

* [http://db.cs.yale.edu/hadoopdb/hadoopdb.html HadoopDB] - A MapReduce layer put in front of a cluster of postgres back end servers. Shared-nothing clustering.

* [[PL/Proxy]] - database partitioning system implemented as PL language.

* [https://www.citusdata.com/citus-products/pg-shard pg_shard] - extension that shards and replicates table across many servers, can also [https://www.citusdata.com/blog/14-marco/178-scaling-out-postgresql-on-amazon-rds-using-masterless-pg-shard scale out Amazon RDS]

* sequoia (jdbc, formerly known as c-jdbc)

* [http://www.postgres-xl.org/ Postgres-XL] is a shared nothing, multi-master clustering solution which can transparently distribute a table on a set of nodes and execute queries in parallel of those nodes. It has a additional component called Global Transaction Manager (GTM) for providing globally consistent view of the cluster. The project is based on the latest available 9.5 release of PostgreSQL. Some companies, such as [http://www.2ndquadrant.com 2ndQuadrant] provides commercial support for the product.

==Connection Pooling and Acceleration==

Connection pooling programs let you reduce database-related overhead when it's the sheer number of physical connections dragging performance down. This is particularly important on Windows, where system limitations prevent large number of connections; see "I cannot run with more than about 125 connections at once" in [[Running & Installing PostgreSQL On Native Windows]]. It's also vital for web applications where the number of connections can get very large.

Some programs that implement connection pooling are:
* [[PgBouncer]]
* [http://www.pgpool.net/ pgpool]

Some people also or alternately use [http://www.danga.com/memcached/ memcached] in various ways to reduce the work the database handles directly by caching popular data. [https://github.com/ohmu/pgmemcache/ pgmemcache] is a PostgreSQL extension for interfacing with memcached servers.

== Cluster management ==

* [https://github.com/ohmu/pglookout/ pglookout] - PostgreSQL replication monitoring and failover daemon
* [[repmgr]] - replication manager for PostgreSQL clusters
* [https://github.com/MasahikoSawada/pg_keeper/tree/REL1_0_STABLE pg_keeper] - Background worker based simple failover daemon.

== Other Resources ==

Sources for more information located but not yet integrated into here.

=== Articles ===

* <strike>[http://bristlecone.continuent.org/uploads/bristlecone/HomePage/PG_East-Scale-Out-Benchmarks_FINAL2.pdf Portable Scale-Out Benchmarks for PostgreSQL] by Robert Hodges</strike> not available (2012-10-11)
* <strike>[http://www.fastware.com.au/docs/PostgreSQL_HighAvailability.pdf High Availability and PostgreSQL] by Gavin Sherry</strike> not available (2012-10-11)
* [http://www.palominodb.com/blog/2012/08/01/cascade-replication-and-delayed-servers-postgresql-92 Cascade Replication and Delayed servers on PostgreSQL 9.2]
* [http://prongs.org/blog/postgresql-replication Streaming Replication on PostgreSQL 9.3 by Afra Ahmad]

* [http://burger-ag.de/postgresql_replikation.whtml Replikation - Lösungen für PostgreSQL] page covering this topic in German. It translates pretty well through [http://babelfish.altavista.com/ Babelfish].

=== Video tutorials ===

* [http://youtu.be/qTPNB0y37EI Alex Alexander presenting Managing Postgres HA using DRBD, Pacemaker and Corosync] youtube video

==Credits==

Sources for the initial information on this page include:
*[http://archives.postgresql.org/pgsql-performance/2007-06/msg00264.php replication thread]
*[http://archives.postgresql.org/pgsql-general/2007-08/msg00085.php pgpool2 vs sequoia]
*[http://archives.postgresql.org/pgsql-hackers/2006-10/msg00810.php Postgresql Caching]

[[Category:Replication]][[Category:Administration]][[Category:Performance]][[Category:Clustering]]

Performance Optimization

2017-05-03T05:21:55Z

Craigkerstiens: /* General Setup and Optimization */

{{Languages}}

== General Setup and Optimization ==
* [[Tuning Your PostgreSQL Server]] by Greg Smith, Robert Treat, and Christopher Browne
* [http://www.revsys.com/writings/postgresql-performance.html Performance Tuning PostgreSQL] by Frank Wiles
* [http://www.pgcon.org/2008/schedule/events/104.en.html GUCs: A Three Hour Tour] by Josh Berkus. Also useful here is his [http://pgfoundry.org/docman/view.php/1000106/84/calcfactors.sxc tuning OpenOffice spreadsheet], which suggests tuning values for 5 different types of workloads.
* [http://linuxfinances.info/info/quickstart.html QuickStart Guide to Tuning PostgreSQL] by Christopher Browne
* [http://www.varlena.com/GeneralBits/Tidbits/annotated_conf_e.html Annotated postgresql.conf] by Josh Berkus and Shridhar Daithankar (older V7.4 targeted version of material covered in the GUC tour referenced above)
* [http://www.varlena.com/GeneralBits/Tidbits/perf.html Performance Tuning] by Josh Berkus and Shridhar Daithankar
* [http://www.zope.org/Members/pupq/pg_in_aggregates Replacing Slow Loops in PostgreSQL] by Joel Burton
* [http://momjian.us/main/writings/pgsql/hw_performance/ PostgreSQL Hardware Performance Tuning] by Bruce Momjian
* [http://www.targeted.org/articles/databases/fragmentation.html The effects of data fragmentation in a mixed load database] by Dmitry Dvoinikov
* [https://wiki.postgresql.org/images/a/a8/Postgres_8_3_Performance.pdf PostgreSQL Performance Features in 8.3] by Simon Riggs
* [http://2ndquadrant.com/media/pdfs/talks/Postgres_Performance_Update84.pdf PostgreSQL Performance Features in 8.4] by Simon Riggs
* [http://www.craigkerstiens.com/2012/10/01/understanding-postgres-performance/ Understanding Postgres Performance] by Craig Kerstiens
* [http://www.craigkerstiens.com/2013/01/10/more-on-postgres-performance/ More on Postgres Performance] by Craig Kerstiens
* [https://www.citusdata.com/blog/2016/10/12/count-performance/ Faster PostgreSQL counting]

Performance courses are available from a number of companies. Check [http://www.postgresql.org/about/eventarchive events and trainings] for further details.

==Critical maintenance for performance==
*[[Introduction to VACUUM, ANALYZE, EXPLAIN, and COUNT]] by Jim Nasby.
*[[VACUUM FULL]] and why you should avoid it
*[[Planner Statistics]]
*[[Using EXPLAIN]]
*[[Logging Difficult Queries]]
*[[Logging Checkpoints]]
*[http://www.westnet.com/~gsmith/content/postgresql/chkp-bgw-83.htm Checkpoints and the Background Writer: PostgreSQL 8.3 Improvements and Migration] by Greg Smith
*[[Bulk Loading and Restores]]
*[[Performance Analysis Tools]] by Craig Ringer

== Database architecture ==
* [[Priorities|Limiting and prioritizing user/query/database resource usage]] by Craig Ringer
* [[Prioritizing databases by separating into multiple clusters]] by Craig Ringer
* [[Clustering]]
* [[Shared Storage]]

==Database Hardware Selection and Setup==
* [[Database Hardware]]
* [[Reliable Writes]]

==Benchmark Workloads==
* [[:Category:Benchmarking]]

[[Category:Administration]][[Category:Performance]][[Category:Benchmarking]]
[[Category:General articles and guides]]

PgConfSV Talks 2016

2016-12-21T01:00:48Z

Craigkerstiens: /* Baden A */

= PGConf SV 2016 Talks =

== Conference Website ==

http://www.pgconfsv.com/

== Trainings: Monday, 14 Nov 2016 ==

* PostgreSQL When It's Not Your Day Job, Christophe Pettus
* SQL Analytics 101, Leon Tchikindas
* Multi-tenancy: Designing for Scale, Ozgun Ergodan
* A beginners guide to pl/pgsql, Jim Mlodgenski

== Sessions: Tuesday, 15 Nov 2016 ==

=== Salon A/B ===

* PGLogical - The Logical Replication for PostgreSQL, Petr Jelinek
* Multi-master replication for painless fault tolerance, Stas Kelvich
* How to use the DBT-3 test kit, Mark Wong
* Implementing Graph Database based on PostgreSQL, Kisung Kim
* Are NoSQL Databases more reliable?, Alan Jones
* Dynamic Database Credentials, Sean Chittenden

=== Salon C/D ===

* Postgresql as an Application Modernization and Consolidation Cornerstone, Charles Finley
* [https://www.youtube.com/watch?v=w2Lu3KMOG98&index=32&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Sort vs. Hash: A Duality], Peter Geoghegan
* [https://www.youtube.com/watch?v=jTsAlJuH5IQ&index=17&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes pg_paxos: Table Replication through Distributed Consensus], Marco Slot
* [https://www.youtube.com/watch?v=Iyfpf1cc4ds&index=27&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Gophers Riding Elephants: Writing PostgreSQL tools in Go], AJ Bahnken
* What is in these databases anyway? Seeing the relationships between diverse and disparate data new, Nathan Hemenway
* [https://www.youtube.com/watch?v=VRob84dQTYc&index=15&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Making Data Dance], Barrett Clark

=== Baden A ===

* [https://www.youtube.com/watch?v=k0Q3ARki0Ew&index=23&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Governor: Postgres HA orchestrator, now with integrated RAFT], Chris Winslett
* [https://www.youtube.com/watch?v=x_XpPbfomso&index=22&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Connection pooling, routing, and queuing with PgBouncer], Peter Eisentraut
* [https://www.youtube.com/watch?v=v0MAAdo4RIs&index=21&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Database Hardware Selection Guidelines], Bruce Momjian
* [https://www.youtube.com/watch?v=GqrBp0gyNHs&index=16&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Understanding Autovacuum], Dan Robinson
* [https://www.youtube.com/watch?v=oY-X5HFu_A4&index=26&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes CCP: Containerized Clustered Postgres], Josh Berkus

== Sessions: Wednesday, 16 Nov 2016 ==

=== Salon A/B ===

* [https://www.youtube.com/watch?v=Ugf6IAsaJqk&index=30&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes PostgreSQL's secret NoSQL superpowers], Amanda Gilmore
* [https://www.youtube.com/watch?v=s-BvKkVSyTA&index=31&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Securing PostgreSQL], Christophe Pettus
* Using Postgres XL, Richard Silva
* Debugging Your PL/pgSQL Code, Jim Mlodgenski
* [https://www.youtube.com/watch?v=_0q_VWrIQUE&index=2&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Simplify your API with s2x (SQL to Anything)], Manuel Zahariev
* [https://www.youtube.com/watch?v=a0SDogoPzss&index=5&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes PgBouncer - Connection Pooling and Routing], Petr Jelinek

=== Salon C/D ===

* How does PostgreSQL actually work?, Peter Eisentraut
* Becoming a SQL Guru, Stella Nisenbaum
* [http://www.joeconway.com/presentations/fun_with_functions-pgconf.sv-2016.pdf Fun with Functions], Joe Conway
* Durability in Postgres, Andres Freund
* PostgreSQL Top 10 Features, Simon Riggs
* [https://www.youtube.com/watch?v=h0eEJ7uX13Q&index=1&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Understanding Advanced Data Types in PostgreSQL], Peter Van Hardenberg

=== Baden A ===

* PL/CUDA - The Fusion of HPC grade power with In-Database Analytics, Kohei KaiGai
* Designing The Right Schema To Power Heap, Dan Robinson
* In-core compression: how to shrink your database size in several times, Anastasia Lubennikova
* PostgreSQL High Availability in a Containerized World, Jignesh Shah
* Distributed In-Database Machine Learning with MADlib, Frank McQuillan
* Postgres 9.4 The Hard Way, James Miller

PgConfSV Talks 2016

2016-12-21T00:58:46Z

Craigkerstiens: /* Salon C/D */

= PGConf SV 2016 Talks =

== Conference Website ==

http://www.pgconfsv.com/

== Trainings: Monday, 14 Nov 2016 ==

* PostgreSQL When It's Not Your Day Job, Christophe Pettus
* SQL Analytics 101, Leon Tchikindas
* Multi-tenancy: Designing for Scale, Ozgun Ergodan
* A beginners guide to pl/pgsql, Jim Mlodgenski

== Sessions: Tuesday, 15 Nov 2016 ==

=== Salon A/B ===

* PGLogical - The Logical Replication for PostgreSQL, Petr Jelinek
* Multi-master replication for painless fault tolerance, Stas Kelvich
* How to use the DBT-3 test kit, Mark Wong
* Implementing Graph Database based on PostgreSQL, Kisung Kim
* Are NoSQL Databases more reliable?, Alan Jones
* Dynamic Database Credentials, Sean Chittenden

=== Salon C/D ===

* Postgresql as an Application Modernization and Consolidation Cornerstone, Charles Finley
* [https://www.youtube.com/watch?v=w2Lu3KMOG98&index=32&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Sort vs. Hash: A Duality], Peter Geoghegan
* [https://www.youtube.com/watch?v=jTsAlJuH5IQ&index=17&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes pg_paxos: Table Replication through Distributed Consensus], Marco Slot
* [https://www.youtube.com/watch?v=Iyfpf1cc4ds&index=27&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Gophers Riding Elephants: Writing PostgreSQL tools in Go], AJ Bahnken
* What is in these databases anyway? Seeing the relationships between diverse and disparate data new, Nathan Hemenway
* [https://www.youtube.com/watch?v=VRob84dQTYc&index=15&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Making Data Dance], Barrett Clark

=== Baden A ===

* Governor: Postgres HA orchestrator, now with integrated RAFT, Chris Winslett
* Connection pooling, routing, and queuing with PgBouncer, Peter Eisentraut
* Database Hardware Selection Guidelines, Bruce Momjian
* Understanding Autovacuum, Dan Robinson
* CCP: Containerized Clustered Postgres, Josh Berkus

== Sessions: Wednesday, 16 Nov 2016 ==

=== Salon A/B ===

* [https://www.youtube.com/watch?v=Ugf6IAsaJqk&index=30&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes PostgreSQL's secret NoSQL superpowers], Amanda Gilmore
* [https://www.youtube.com/watch?v=s-BvKkVSyTA&index=31&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Securing PostgreSQL], Christophe Pettus
* Using Postgres XL, Richard Silva
* Debugging Your PL/pgSQL Code, Jim Mlodgenski
* [https://www.youtube.com/watch?v=_0q_VWrIQUE&index=2&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Simplify your API with s2x (SQL to Anything)], Manuel Zahariev
* [https://www.youtube.com/watch?v=a0SDogoPzss&index=5&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes PgBouncer - Connection Pooling and Routing], Petr Jelinek

=== Salon C/D ===

* How does PostgreSQL actually work?, Peter Eisentraut
* Becoming a SQL Guru, Stella Nisenbaum
* [http://www.joeconway.com/presentations/fun_with_functions-pgconf.sv-2016.pdf Fun with Functions], Joe Conway
* Durability in Postgres, Andres Freund
* PostgreSQL Top 10 Features, Simon Riggs
* [https://www.youtube.com/watch?v=h0eEJ7uX13Q&index=1&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Understanding Advanced Data Types in PostgreSQL], Peter Van Hardenberg

=== Baden A ===

* PL/CUDA - The Fusion of HPC grade power with In-Database Analytics, Kohei KaiGai
* Designing The Right Schema To Power Heap, Dan Robinson
* In-core compression: how to shrink your database size in several times, Anastasia Lubennikova
* PostgreSQL High Availability in a Containerized World, Jignesh Shah
* Distributed In-Database Machine Learning with MADlib, Frank McQuillan
* Postgres 9.4 The Hard Way, James Miller

PgConfSV Talks 2016

2016-12-21T00:57:32Z

Craigkerstiens: /* Salon A/B */

= PGConf SV 2016 Talks =

== Conference Website ==

http://www.pgconfsv.com/

== Trainings: Monday, 14 Nov 2016 ==

* PostgreSQL When It's Not Your Day Job, Christophe Pettus
* SQL Analytics 101, Leon Tchikindas
* Multi-tenancy: Designing for Scale, Ozgun Ergodan
* A beginners guide to pl/pgsql, Jim Mlodgenski

== Sessions: Tuesday, 15 Nov 2016 ==

=== Salon A/B ===

* PGLogical - The Logical Replication for PostgreSQL, Petr Jelinek
* Multi-master replication for painless fault tolerance, Stas Kelvich
* How to use the DBT-3 test kit, Mark Wong
* Implementing Graph Database based on PostgreSQL, Kisung Kim
* Are NoSQL Databases more reliable?, Alan Jones
* Dynamic Database Credentials, Sean Chittenden

=== Salon C/D ===

* Postgresql as an Application Modernization and Consolidation Cornerstone, Charles Finley
* Sort vs. Hash: A Duality, Peter Geoghegan
* pg_paxos: Table Replication through Distributed Consensus, Marco Slot
* Gophers Riding Elephants: Writing PostgreSQL tools in Go, AJ Bahnken
* What is in these databases anyway? Seeing the relationships between diverse and disparate data new, Nathan Hemenway
* Making Data Dance, Barrett Clark

=== Baden A ===

* Governor: Postgres HA orchestrator, now with integrated RAFT, Chris Winslett
* Connection pooling, routing, and queuing with PgBouncer, Peter Eisentraut
* Database Hardware Selection Guidelines, Bruce Momjian
* Understanding Autovacuum, Dan Robinson
* CCP: Containerized Clustered Postgres, Josh Berkus

== Sessions: Wednesday, 16 Nov 2016 ==

=== Salon A/B ===

* [https://www.youtube.com/watch?v=Ugf6IAsaJqk&index=30&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes PostgreSQL's secret NoSQL superpowers], Amanda Gilmore
* [https://www.youtube.com/watch?v=s-BvKkVSyTA&index=31&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Securing PostgreSQL], Christophe Pettus
* Using Postgres XL, Richard Silva
* Debugging Your PL/pgSQL Code, Jim Mlodgenski
* [https://www.youtube.com/watch?v=_0q_VWrIQUE&index=2&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Simplify your API with s2x (SQL to Anything)], Manuel Zahariev
* [https://www.youtube.com/watch?v=a0SDogoPzss&index=5&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes PgBouncer - Connection Pooling and Routing], Petr Jelinek

=== Salon C/D ===

* How does PostgreSQL actually work?, Peter Eisentraut
* Becoming a SQL Guru, Stella Nisenbaum
* [http://www.joeconway.com/presentations/fun_with_functions-pgconf.sv-2016.pdf Fun with Functions], Joe Conway
* Durability in Postgres, Andres Freund
* PostgreSQL Top 10 Features, Simon Riggs
* [https://www.youtube.com/watch?v=h0eEJ7uX13Q&index=1&list=PLixnExCn6lRoaqtaIarHjRTDOyxEVIPes Understanding Advanced Data Types in PostgreSQL], Peter Van Hardenberg

=== Baden A ===

* PL/CUDA - The Fusion of HPC grade power with In-Database Analytics, Kohei KaiGai
* Designing The Right Schema To Power Heap, Dan Robinson
* In-core compression: how to shrink your database size in several times, Anastasia Lubennikova
* PostgreSQL High Availability in a Containerized World, Jignesh Shah
* Distributed In-Database Machine Learning with MADlib, Frank McQuillan
* Postgres 9.4 The Hard Way, James Miller

PgConfSV Talks 2016

2016-12-21T00:54:49Z

Craigkerstiens: /* Salon C/D */

Clustering

2016-04-05T23:29:21Z

Craigkerstiens:

== Projects ==

(alphabetical order)

* [[Bucardo]]
* [[BDR User Guide|BDR (Bi-Directional Replication for PostgreSQL)]]
* [https://www.citusdata.com/product/citus Citus]
* [[GridSQL]]
* [[HadoopDB]]
* [https://www.citusdata.com/citus-products/pg-shard pg_shard]
* [[Pgpool-II]]
* [[PL/Proxy]]
* [[Postgres-XC|Postgres-XC]]
* [http://www.postgres-xl.org/ Postgres-XL], a Postgres-XC derivative/enhancement.
* [[PostgresForest]]
* [[SkyTools]] ([[Londiste]])
* [[Slony]]
* [[Stado]]
* [[Tungsten]]

Work in progress is being tracked at [[Clustering Development Projects]]

== Stalled ==
Projects which seem to have stalled and have not had updates in 1+ year.

* [[Mammoth]]
* [[PgCluster]]
* [[Postgres-R]]
* [[rubyrep]]

== Existing Overview Docs ==
* [[Replication, Clustering, and Connection Pooling]]

== Template for information ==

To include information about a clustering project, please create a new page with the project's name as the page title, and use the template '''[[Template:ClusteringProject]]''' to describe the project.

[[Category:Replication]]
[[Category:Clustering]]

Replication, Clustering, and Connection Pooling

2016-04-05T23:28:38Z

Craigkerstiens:

==Introduction==
There are many approaches available to scale PostgreSQL beyond running on a single server. An outline of the terminology and basic technologies involved is at [http://www.postgresql.org/docs/current/interactive/high-availability.html High Availability and Load Balancing]. There is a [http://momjian.us/main/writings/pgsql/replication.pdf presentation] covering some of these solutions.

There is no one-size fits all replication software. You have to understand your requirements and how various approaches fit into that. For example, here are two extremes in the replication problem space:

* You have a few servers connected to a local network you want to always keep current for failover and load-balancing purposes. Here you would be considering solutions that are synchronous, eager, and therefore conflict-free.
* Your users take a local copy of the database with them on laptops when they leave the office, make changes while they are away, and need to merge those with the main database when they return. Here you'd want an asynchronous, lazy replication approach, and will be forced to consider how to handle conflicts in cases where the same record has been modified both on the master server and on a local copy.

These are both database replication problems, but the best way to solve them is very different. And as you can see from these examples, replication has a lot of specific terminology that you'll have to understand to figure out what class of solution makes sense for your requirements. A great source for this background is in the
[http://www.postgres-r.org/documentation/terms Postgres-R Terms and Definitions for Database Replication]. The main theoretical topic it doesn't mention is how to resolve conflict resolution in lazy replication cases like the laptop situation, which involves voting and similar schemes.

== Features in the Core of PostgreSQL ==

*[[Hot Standby]]/[[Streaming Replication]] is available as of PostgreSQL 9.0 and provides asynchronous binary replication to one or more standbys. Standbys may also become hot standbys meaning they can be queried as a read-only database. This is the fastest type of replication available as WAL data is sent immediately rather than waiting for a whole segment to be produced and shipped.

*[[Warm Standby]]/Log Shipping is a HA solution which 'replicates' a database cluster to an archive or a warm (can be brought up quickly, but not available for querying) standby server. Overhead is very low and it's easy to set up. This is a simple and appropriate solution if all you care about is continuous backup and short failover times.

* PostgreSQL 9.4's [[Logical Changeset Extraction]] forms the foundation of the [[Bi-Directional Replication]] and [[Logical Log Streaming Replication]] features being added to PostgreSQL.

Historically, the PostgreSQL core team considered replication and clustering technology outside the scope of the main project's focus but this changed in 2008, see the [http://archives.postgresql.org/pgsql-hackers/2008-05/msg00913.php Core Team's statement]. Replication is now a significant focus of ongoing PostgreSQL development.

==Comparison matrix==

This page is being overhauled at [[Clustering]]

{| border="1" cellpadding="1" cellspacing="0" style="font-size: 85%; border: gray solid 1px; border-collapse: collapse; text-align: center; width: 100%; table-layout: fixed;"
|- style="background: #ececec"
! Program
! License
! Maturity
! Replication Method
! Sync
! Connection Pooling
! Load Balancing
! Query Partitioning
|-
! style="text-align:block;" bgcolor="#ececec" | [[PgCluster]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stalled
| bgcolor="#ffffaa" | Master-Master
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ddffdd" | Yes
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | '''pgpool-I'''
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Statement-Based Middleware
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Pgpool-II]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | Statement-Based Middleware
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
|-
! style="text-align:block;" bgcolor="#ececec" | [[slony]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Bucardo]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Master, Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Londiste]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[Mammoth]]
| bgcolor="#ffffaa" | BSD
| bgcolor="#ffffaa" | Stalled
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[rubyrep]]
| bgcolor="#ffffaa" | MIT
| bgcolor="#ffffaa" | Stalled
| bgcolor="#ffffaa" | Master-Master, Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [[BDR User Guide|BDR (Bi-Directional Replication)]]
| bgcolor="#ffffaa" | PostgreSQL (BSD)
| bgcolor="#ffffaa" | Stable
| bgcolor="#ffffaa" | Master-Master<br/>(no triggers needed)
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|-
! style="text-align:block;" bgcolor="#ececec" | [https://www.citusdata.com/citus-products/pg-shard pg_shard]
| bgcolor="#ffffaa" | LGPL
| bgcolor="#ffffaa" | Recent release
| bgcolor="#ffffaa" | Statement-based Middleware (as an extension)
| bgcolor="#ffffaa" | Synchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ddffdd" | Yes
| bgcolor="#ddffdd" | Yes
|-
! style="text-align:block;" bgcolor="#ececec" | [http://2ndquadrant.com/resources/pglogical pglogical]
| bgcolor="#ffffaa" | PostgreSQL
| bgcolor="#ffffaa" | Recent Release
| bgcolor="#ffffaa" | Master-Slave
| bgcolor="#ffffaa" | Asynchronous
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
| bgcolor="#ffaaaa" | No
|}

==Replication==

Aside from the in-core streaming replication, mentioned above...

*Slony-I: Seems good, single master only, master is a single point of failure, no good failover system for electing a new master or having a failed master rejoin the cluster. Slave databases are mostly for safety or for parallelizing queries for performance. Suffers from O(N^2) communications (N = cluster size). with reasonable sysadmin you can implement failover system yourself. regarding communications, you can cascade the replication to reduce load on the master. If you were implementing a large replication cluster, this would probably be a good idea. Slony is powerful, trigger based, and highly configurable.

* PGCluster: PGCluster (which, incidentally, is not the same as PGCluster-II, a shared-disk solution), which does synchronous multimaster replication. Two single-points failure spots, load balancer and the data replicator. The project has historically looked a bit dead, but they just released a new version and http://pgfoundry.org/projects/pgcluster is up to date (at least downloads page) One major downside to PGCluster is that it uses a modified version of PostgreSQL, and it usually lags a few releases behind.

* http://www.pgpool.net/ pgpool 1/2 is a reasonable solution. it's statement level replication, which has some downsides, but is good for certain things. pgpool 2 has a neat distributed table mechanism which is interesting. You might want to be looking here if you have extremely high ratios of read to write but need to service a huge transaction volume. Supports load-balancing and replication by implementing a proxy that duplicates all updates to all slaves. It can partition data by doing this, and it can semi-intelligently route queries to the appropriate servers.

* "Mammoth Replicator" - BSD - http://www.commandprompt.com/products/mammothreplicator/ - Former proprietary solution, now open source. Uses a central logging process to distribute data changes amongst nodes. Essentially a fork of Postgres, as the changes are written directly into the backend.

* "Bucardo" - BSD License - http://bucardo.org/ - Trigger-based, asynchronous, multi-master or master-slave, written using plperl.

* Cybertec, an Austrian company, offers a proprietary packaging of PGCluster. They simply call it PostgreSQL Multimaster-Replication, see http://www.cybertec.at.

* [[Londiste_Tutorial|Londiste]], a part of [[Skytools]] (https://developer.skype.com/SkypeGarage/DbProjects/SkyTools) which is a collection of replication tools from the Skype people. Purports to be simpler to use than Slony.

* [[BDR User Guide|BDR (Bi-Directional Replication for PostgreSQL)]] - multi-master replication based on log streaming logical replication.

* [http://www.continuent.com/index.php?option=com_content&task=view&id=212&Itemid=169 Continuent uni/cluster], proprietary and the related Sequoia (jdbc, formerly known as c-jdbc)

* [http://www.postgres-r.org Postgres-R] is still in development. It features eager and thus conflict-free, but async multi-master replication.

* [http://symmetricds.codehaus.org/ SymmetricDS] is an open-source, web-enabled, database independent, data synchronization software application. It uses web and database technologies to replicate tables between relational databases in near real time. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outages. Supports several relational databases, including PostgreSQL. Licensed under Lesser GPL (LGPL).

* DRBD (http://www.drbd.org/), a device driver that replicates disk blocks to other nodes. This works for failover only, not for scaling reads. Easy migration of devices if combined with an NFS export.

* [http://sourceforge.net/projects/daffodilreplica/ Daffodil Replicator]. Supports several relational databases, including PostgreSQL. Licensed under GPL.

* "RubyRep" - MIT License - http://www.rubyrep.org/ - Ruby based, asynchronous, multi-master replication system, which supports Postgres and MySQL.

* "pg_comparator" - BSD License - http://pgfoundry.org/projects/pg-comparator/ - Perl-based, table-level async master-slave "diff" and "patch" method of replication. Low configuration overhead.

* [https://www.citusdata.com/ Citus Data] develops the [https://www.citusdata.com/citus-products/pg-shard pg_shard extension] which can transparently shard a PostgreSQL table across many servers and replicate the shards, and [https://www.citusdata.com/product/citus Citus] which parallelizes queries across many servers and cores for real-time analytics and supports bulk loading, distributed joins, columnar storage and more.

===Inactive projects===
* Slony-II
* PGReplication

==Clustering==

* [https://www.citusdata.com/product/citus Citus] - shards and replicates tables across a scalable, high availability cluster of commodity PostgreSQL servers and parallelizes queries for real-time SQL on big data.

* [http://www.greenplum.com/index.php?page=greenplum-database Greenplum Database] (formerly Bizgres MPP), proprietary. Not so much a replication solution as a way to parallelize queries, and targeted at the data warehousing crowd. Similar to ExtenDB, but tightly integrated with PostgreSQL.

* [http://www.enterprisedb.com/products/gridsql.do GridSQL for EnterpriseDB Advanced Server] (formerly ExtenDB)

* [http://db.cs.yale.edu/hadoopdb/hadoopdb.html HadoopDB] - A MapReduce layer put in front of a cluster of postgres back end servers. Shared-nothing clustering.

* [[PL/Proxy]] - database partitioning system implemented as PL language.

* [https://www.citusdata.com/citus-products/pg-shard pg_shard] - extension that shards and replicates table across many servers, can also [https://www.citusdata.com/blog/14-marco/178-scaling-out-postgresql-on-amazon-rds-using-masterless-pg-shard scale out Amazon RDS]

* sequoia (jdbc, formerly known as c-jdbc)

==Connection Pooling and Acceleration==

Connection pooling programs let you reduce database-related overhead when it's the sheer number of physical connections dragging performance down. This is particularly important on Windows, where system limitations prevent large number of connections; see "I cannot run with more than about 125 connections at once" in [[Running & Installing PostgreSQL On Native Windows]]. It's also vital for web applications where the number of connections can get very large.

Some programs that implement connection pooling are:
* [[PgBouncer]]
* [http://www.pgpool.net/ pgpool]

Some people also or alternately use [http://www.danga.com/memcached/ memcached] in various ways to reduce the work the database handles directly by caching popular data. [https://github.com/ohmu/pgmemcache/ pgmemcache] is a PostgreSQL extension for interfacing with memcached servers.

== Cluster management ==

* [https://github.com/ohmu/pglookout/ pglookout] - PostgreSQL replication monitoring and failover daemon
* [[repmgr]] - replication manager for PostgreSQL clusters

== Other Resources ==

Sources for more information located but not yet integrated into here.

=== Articles ===

* <strike>[http://bristlecone.continuent.org/uploads/bristlecone/HomePage/PG_East-Scale-Out-Benchmarks_FINAL2.pdf Portable Scale-Out Benchmarks for PostgreSQL] by Robert Hodges</strike> not available (2012-10-11)
* <strike>[http://www.fastware.com.au/docs/PostgreSQL_HighAvailability.pdf High Availability and PostgreSQL] by Gavin Sherry</strike> not available (2012-10-11)
* [http://www.palominodb.com/blog/2012/08/01/cascade-replication-and-delayed-servers-postgresql-92 Cascade Replication and Delayed servers on PostgreSQL 9.2]
* [http://prongs.org/blog/postgresql-replication Streaming Replication on PostgreSQL 9.3 by Afra Ahmad]

* [http://burger-ag.de/postgresql_replikation.whtml Replikation - Lösungen für PostgreSQL] page covering this topic in German. It translates pretty well through [http://babelfish.altavista.com/ Babelfish].

=== Video tutorials ===

* [http://youtu.be/qTPNB0y37EI Alex Alexander presenting Managing Postgres HA using DRBD, Pacemaker and Corosync] youtube video

==Credits==

Sources for the initial information on this page include:
*[http://archives.postgresql.org/pgsql-performance/2007-06/msg00264.php replication thread]
*[http://archives.postgresql.org/pgsql-general/2007-08/msg00085.php pgpool2 vs sequoia]
*[http://archives.postgresql.org/pgsql-hackers/2006-10/msg00810.php Postgresql Caching]

[[Category:Replication]][[Category:Administration]][[Category:Performance]][[Category:Clustering]]

PostgreSQL derived databases

2016-04-05T21:01:22Z

Craigkerstiens:

A list of PostgreSQL derived forks and rebranded distributions in alphabetical order.

{| align="center" border="1" cellspacing="0" {{Prettytable}}
|-
!{{Hl2}} |Name
!{{Hl2}} |Vendor
!{{Hl2}} |License
!{{Hl2}} |Availability
!{{Hl2}} |Notes
|-
|Aster Data
|Teradata
|Proprietary
|2005-....
|PostgreSQL + Map/Reduce
|-
|BDR
|2ndQuadrant
|BSD
|2014-
|PostgreSQL Multi Master, contributed actively back to Core PG
|-
|Bizgres
|Greenplum
|BSD
|2005-2007
|PostgreSQL + BI features
|-
|[http://www.citusdata.com Citus]
|Citus Data
|AGPL-3.0
|2012-
|Sharding and replication across many servers with MPP [https://www.citusdata.com/citus-products/citusdb-software]
|-
|Cybercluster
|Cybertec
|BSD
|2007-2010
|Clustering (pgCluster fork)
|-
|[http://greenplum.org/ Greenplum Database]
|Greenplum
|Apache2
|2005-....
|PostgreSQL + BI features (formerly known as "Bizgres MPP") [https://github.com/greenplum-db/gpdb]
|-
|ExtenDB
|ExtenDB
|Proprietary
|2003-2007
|PostgreSQL + BI Features [http://www.stormdb.com/community/stado?destination=node%2F8]
|-
|FUJITSU Enterprise Postgres
|Fujitsu
|proprietary
|2006-....
|Full PostgreSQL compatibility with additional functionality [http://www.fujitsu.com/global/products/software/middleware/opensource/postgres/]
|-
|GresCube
|NTT DATA
|Proprietary
|2012-....
|Database appliance solution based on PostgreSQL [http://opensol.jp.nttdata.com/solutions/grescube/]
|-
|GridSQL
|EnterpriseDB
|GPL
|2007-2010
|PostgreSQL + BI Features (formerly ExtenDB) [http://www.stormdb.com/community/stado?destination=node%2F8]
|-
|Great Bridge PostgreSQL
|Great Bridge LLC
|BSD
|1999-2001
|PostgreSQL re-distribution
|-
|HadoopDB
|Yale University
|Apache License V2.0
|2009-....
|PostgreSQL + shared-nothing cluster + Hadoop [http://db.cs.yale.edu/hadoopdb/hadoopdb.html]
|-
|[http://hadapt.com/ Hadapt]
|Teradata
|Proprietary
|2011-....
|HadoopDB fork
|-
|[[Mammoth]]
|Command Prompt
|BSD
|2005-2010
|PostgreSQL + proprietary replication + extensions
|-
|Netezza
|IBM
|proprietary
|2002-....
|Appliance based on PostgreSQL SQL engine
|-
|NuSphere UltraSQL
|NuSphere
|proprietary
|2002-2003
|Native Win32 port of PostgreSQL
|-
|ParAccel
|Actian
|proprietary
|2005-....
|PostgreSQL + BI features [https://en.wikipedia.org/wiki/ParAccel]
|-
|Pervasive PostgreSQL
|Pervasive
|BSD
|2005-2006
|PostgreSQL re-distribution
|-
|pgCluster
|SRA
|BSD
|2002-2005
|Clustering (Share Nothing)
|-
|pgCluster-II
|SRA
|BSD
|2006-2007
|Clustering (Shared Disk)
|-
|[http://www.pgpool.net/ pgPool-II]
|pgPool GDG
|BSD
|2006-....
|Clustering (Connection Pooling / Replication / Load-Balancing)
|-
|[https://www.pipelinedb.com/ PipelineDB]
|PipelineDB
|GPL v3
|2015-....
|Streaming SQL
|-
|PostgresForest
|NTT DATA
|BSD
|2006-2010
|Clustering / PostgresForest is a fork of the JDBC driver, not from the backend code.
|-
|Postgres Plus
|EnterpriseDB
|OSS
|2008-....
|PostgreSQL + contrib + apps + drivers
|-
|Postgres Plus Advanced Server
|EnterpriseDB
|proprietary
|2008-....
|Postgres + Oracle compatibility + apps, formally EnterpriseDB AS [http://www.enterprisedb.com/products-services-training/products/postgres-plus-advanced-server]
|-
|Postgres-R
|PGDG
|BSD
|2006-2010
|Clustering
|-
|[http://postgres-x2.github.io/ Postgres-X2]
|PGX2DG
|BSD
|2015-
|Clustering (formerly Postgres-XC)
|-
|Postgres-XC
|PGXCDG
|BSD
|2010-2013
|Clustering [http://postgresxc.wikia.com/wiki/Postgres-XC_Wiki]
|-
|[http://www.postgres-xl.org/ Postgres-XL]
|PGXLDG
|BSD
|2014-....
|Clustering
|-
|PowerGres
|SRA OSS
|proprietary
|2003-....
|Native Win32 port of PostgreSQL and Linux RPM
|-
|PowerGres Plus
|SRA OSS
|proprietary
|2003-....
|PostgreSQL + custom storage engine, redundant WAL, encrypted database [http://powergres.sraoss.co.jp/s/ja/product/Plus.php]
|-
|PostgreSQL for Solaris
|Sun
|TPL
|2006-2009
|PostgreSQL re-distribution
|-
|RecDB
|umn.edu
|BSD
|2013-....
|Recommendation Engine [http://www-users.cs.umn.edu/~sarwat/RecDB/]
|-
|Red Hat Database
|Red Hat
|BSD
|2002-2003
|PostgreSQL re-distribution
|-
|Redshift
|Amazon
|Private/Cloud-based
|2013-....
|Data Warehouse on AWS (based on ParACCEL) [http://aws.amazon.com/ru/documentation/redshift] [https://en.wikipedia.org/wiki/Amazon_Redshift]
|-
|Stado
|Stado GDG
|GPL
|2011-2011
|PostgreSQL + BI Features (fork of GridSQL) [https://wiki.postgresql.org/wiki/Stado]
|-
|TelegraphCQ
|UC Berkeley
|BSD
|2000-2008
|Data Stream oriented fork of PostgreSQL
|-
|TruCQ
|Truviso
|proprietary
|2008-2012
|Fork of TelegraphCQ
|-
|Vertica
|HP
|proprietary
|2005-....
| Column-oriented DataWarehouse (created by Stonebraker),<br />may only be forking the psql client library.
|-
|Yahoo! Everest
|Yahoo!
|private
|2008-....
|multi-petabyte database / MPP [http://fr.scribd.com/doc/3159239/70-EverestPGCon-RT]
|}

== Related documents ==

* https://en.wikipedia.org/wiki/Template:Timeline_PostgreSQL
* https://github.com/rafaelma/postgresql-timeline
* http://de.slideshare.net/pgconf/elephant-roads-a-tour-of-postgres-forks
* https://github.com/daamien/artwork/tree/master/inkscape/PostgreSQL_timeline

[[Category:Derivative Products|!]]

FOSDEM 2014

2014-02-02T03:45:12Z

Craigkerstiens: /* PostgreSQL FOSDEM Devroom */

==Schedule==

Schedule is available at the following website: http://www.postgresql.eu/events/schedule/fosdem2014/

== PGDay FOSDEM 2014 ==

We are also organizing one-day event: http://fosdem2014.pgconf.eu/

== Talks: Friday, January 31st, 2014 ==

=== Radisson Blu ===

* Welcome (Magnus Hagander)
* Statistics in PostgreSQL (Heikki Linnakangas)
* Advanced Extension Use Cases (Dimitri Fontaine)
* Integrated cache invalidation for better hit ratios (Magnus Hagander)
* [http://thebuild.com/presentations/worst-day-fosdem-2014.pdf The Worst Day of Your Life] (Christophe Pettus)
* [https://wiki.postgresql.org/images/8/85/Materialised_Views_-_FOSDEM.pdf Materialised views now and in the future] (Thom Brown)
* [http://www.slideshare.net/MarkusWinand/indexes-neglectedperformanceallrounder Indexes: The neglected performance all-rounder] (Markus Winand)
* PostgreSQL - Community meets Business (Michael Meskes)

== Talks: Saturday, February 1st, 2014 ==

=== PostgreSQL FOSDEM Devroom ===

* Managing Postgres in a devops environment (Gabriele Bartolini, Marco Nenciarini)
* Real-Life PostgreSQL JSON (Christophe Pettus)
* A look at the Elephants trunk - PostgreSQL 9.4 (Magnus Hagander)
* [https://speakerdeck.com/craigkerstiens/postgres-performance-for-humans Postgres Performance for Humans] (Craig Kerstiens)
* Postgres for Application Developers (Will Leinweber)
* [http://www.slideshare.net/shanemcintosh/identifying-hotspots-in-the-postgresql-build-process Identifying Hotspots in the PostgreSQL Build Process] (Shane McIntosh)
* Announcements, Annual Report and Election Results (Magnus Hagander)

Speakers: please upload your slides to the wiki and add a link above.

[[Category:PostgreSQL Events]]

PostgreSQL Conference Europe Talks 2013

2013-11-01T13:35:59Z

Craigkerstiens:

= PostgreSQL Conference Europe 2013 Talks =

== Conference Website ==

http://2013.pgconf.eu/

== Trainings: Tuesday 29th October, 2013 ==

=== Dargle ===

* PostgreSQL & PostGIS (Gianni Ciolli)
* Writing & using Postgres Extensions (Dimitri Fontaine)

=== Dodder ===

* PostgreSQL indexing (Hans-Jürgen Schönig)

=== Liffey ===

* PostgreSQL Performance Tuning (Bruce Momjian)
* Data Processing Inside PostgreSQL (Bruce Momjian)

== Talks: Wednesday 30th October, 2013 ==

=== Earlsfort ===

* Keynote: Handling the Spotlight: How PostgreSQL can become the database of choice (Mark Taylor)
* What's New in PostgreSQL 9.3 (Robert Treat)
* PostgreSQL - One of the most important Open Source projects (Michael Meskes)
* Going spatial with PostGIS and more (Hugo Mercier)
* Nulls Make Things Easier? (Bruce Momjian)
* PostgreSQL Disaster Recovery with Barman (Gabriele Bartolini)

=== Leeson ===

* PostgreSQL for Rails (Roland Sonnenschein)
* Introduction of pg_statsinfo and pg_stats_reporter ~Statistics Reporting Tool for DBA~ (Mitsumasa KONDO)
* Writing a foreign data wrapper (Bernd Helmle) [http://de.slideshare.net/psoo1978/pg-fdw Slides]
* Success stories on migrating from Sybase ASE to PostgreSQL (Achim Eisele, Jens Wilke)
* Multicorn: writing Foreign Data Wrappers in python (Ronan Dunklau)

=== Pembroke ===

* Writing a user-defined datatype (Heikki Linnakangas)
* LATERAL querying in 9.3; keep up with the new features! (Gianni Ciolli)
* Detecting performance bottlenecks (Hans-Jürgen Schönig)
* Explaining the Index of PostgreSQL Indexes (Jonathan S. Katz)
* Conduct change from Oracle to PostgreSQL (Jean-Paul Argudo)

== Talks: Thursday 31th October, 2013 ==

=== Earlsfort ===

* Taking advantage of custom background workers (Michael Paquier)
* Doing PITR Right - There's more than just replication (Stephen Frost)
* Postgres what they really use (([http://www.craigkerstiens.com Craig Kerstiens]) [https://speakerdeck.com/craigkerstiens/postgres-what-they-really-use slides])
* Indexes: The neglected performance all-rounder (Markus Winand)
* Migrating from MySQL to PostgreSQL (Dimitri Fontaine)
* Lightning talks (Harald Armin Massa)

=== Leeson ===

* Gotcha! Finer points of the postgres query planner and how to use them in your queries (Atri Sharma)
* [https://wiki.postgresql.org/images/a/ad/Materialised_views_now_and_the_future-pgconfeu_2013.pdf Materialised views now and the future] (Thom Brown)
* [http://www.slideshare.net/davidfetter/federation-with-foreigndatawrapperspgconfeu20131031 Federation with Foreign Data Wrappers] (David Fetter)
* Switching Horses Overnight - a Software Manufacturer's View on Replacing an Existing Database Management System with PostgreSQL (Robert Lichtenberger, Stefan Andreatta)
* My experience with embedding PostgreSQL (Jignesh Shah)

=== Pembroke ===

* pgBadger v.4 (Jean-Paul Argudo)
* Business Intelligence & Performance (Simon Riggs)
* Concurrency in Postgres (Peter Geoghegan)
* Next generation of GIN (Alexander Korotkov, Oleg Bartunov)
* Visualizing Postgres ([http://bitfission.com Will Leinweber]) [https://speakerdeck.com/leinweber/visualizing-postgres slides]

== Lightning talks: Thursday 31st October, 2013 ==

* [http://www.slideshare.net/davidfetter/slides-pg-confeu20131031 Slides] (David Fetter)

== Talks: Friday 1st November, 2013 ==

=== Earlsfort ===

* Epic fails in the RDBMS world (Willem Leenen)
* Using Postgres FDW w/ a sharded PostgreSQL Database (Stephen Frost)
* Postgres versus an anonymous database from Redwood (Gregory Stark)
* Hierarchies Without Masters (Gianni Ciolli)
* PostgreSQL Futures (Simon Riggs)
* PostgreSQL in 5 years - Expectations from the market place (Keith Alsheimer)
* Closing (Dave Page, Magnus Hagander)

=== Leeson ===

* Pagination done the PostgreSQL way (Markus Winand)
* Postgres from Vision to Reality, ABNAMRO Client case (Michel Sijmons)
* PostgreSQL Backups, the good the bad and the ugly (Joshua D. Drake)
* Elephants in fashion (Valentine Gogichashvili)

=== Pembroke ===

* Agile Oracle to PostgreSQL migrations (Gabriele Bartolini)
* Useful PostgreSQL Extensions -- the extensions behind the scenes (Devrim GÜNDÜZ)
* Binary storage for nested data structures and application to hstore data type. (Oleg Bartunov)
* Wiretapping the Wire Protocol: automatic data visualization for psql with Cartographer and FEMEBE (Maciek Sakrejda)

Postgres Open 2013

2013-09-20T20:44:41Z

Craigkerstiens: /* Wednesday September 18 */

Slides for talks will be linked here!

Events shown for placeholding purposes. See the "Editing help" for wiki formatting tips.

== Monday September 16 ==
* PostgreSQL Data Type Safari (Jonathan Katz)
* Surviving Server Overload (Greg Smith)
* Database Normalization for Everybody (Brent Friedman)
* [https://speakerdeck.com/zzzeek/introduction-to-sqlalchemy-and-orms-pgopen-2013 Introduction to SQLAlchemy (Mike Bayer)]

== Tuesday September 17 ==
* [http://baumler.com/postgres/ Case Study: Converting the CATME application from MySQL to Postgres (Julie Baumler)]
* [http://speakerdeck.com/peterg/concurrency-in-postgres Concurrency in Postgres (Peter Geoghegan)]
* [http://phaedrusdeinus.org/talks/federate_fdw/federate_fdw.pdf Federating Queries Using postgres_fdw (John Melesky)]
* Keynote (Jesse Noller)
* [http://scanningpages.files.wordpress.com/2013/09/puppet_pgopen2013.pdf Managing PostgreSQL with Puppet (Steve Singer)]
* Middleware is Dead! (David Benoit)
* My experience with embedding PostgreSQL (Jignesh Shah)
* [https://docs.google.com/file/d/0B2hio52iFB7AaHB0VEhaYmtZMms/edit?pli=1 New approach to resolve Object-relational impedance mismatch (Henrietta Dombrovskaya)]
* Postgres what they really use (Craig Kersteins)
* [http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013 Postgres-XC dynamic cluster management (Koichi Suzuki)]
* [https://sites.google.com/site/robertmhaas/presentations/Query%20Planning%20Gone%20Wrong.odp?attredirects=0&d=1 Query Planning Gone Wrong (Robert Haas)]
* Recovering from a Damaged Cluster (Robert Bernier)
* [https://speakerdeck.com/selenamarie/sane-schema-management-with-alembic Sane Schema Management With Alembic (Selena Deckelmann)]
* [http://www.slideshare.net/denishpatel/scaling-postgres Scaling Postgres (Denish Patel)]
* [http://t.co/q0dQZJg2cJ Taking Advantage of Custom Background Workers (Michael Pacquier)]
* What's New in PostgreSQL 9.3 (Magnus Hagander)

=== Lightning Talks ===
* [http://tinyurl.com/pgopen2013-ipy-sql IPython Notebook: your new SQL client (Catherine Devlin)]
* [https://wiki.postgresql.org/images/d/d4/Json-speed-user-groups.pdf JSON, Speed, and User Groups (Jonathan Katz)]
* [http://www.slideshare.net/markwkm/ohai-my-name-is-chelnik-postgres-open-2013-report OHAI, my name is Chelnik!] Postgres Open 2013 Report (Mark Wong)
* [http://www.slideshare.net/denishpatel/choosing-the-d Choosing the "D"/Doctor (Denish Patel)]

== Wednesday September 18 ==
* A Comparison of PostgreSQL Encryption Options (Syed Faisal Akber, Dong Ye)
* Geographically Distributed Parallel Databases – A Reality! (Christine Normile)
* [https://wiki.postgresql.org/wiki/File:Postgres_Is_Different_From_Better_Than_Your_RDBMS.pdf How Postgres is different from (read BETTER than) your database (Gurjeet Singh)]
* [http://momjian.us/main/writings/pgsql/inside_shmem.pdf Inside PostgreSQL Shared Memory (Bruce Momjian)]
* JackDB - Database Development In Your Web Browser (Sehrope Sarkuni)
* Multi-Database Collaboration and Extract-Transform-Load with HTSQL (Clark C. Evans)
* NewSQL: Leveraging PostgreSQL (John S. Jones)
* Postgres Plus Cloud Database (Hans Hrasna (Tom Kincaid))
* [http://www.consistentstate.com/pdf/backup-and-recovery-methods.pdf PostgreSQL Backup and Recovery Methods (Kevin Kempter)]
* [https://wiki.postgresql.org/wiki/File:PostgresOnZFS-PGO.pdf PostgreSQL on ZFS: backup, replication, disaster recovery, and more (Keith Paskett)]
* Replication Refresh (Simon Riggs)
* [https://speakerdeck.com/craigkerstiens/scalable-uniques-in-postgres-with-hll Scalable Uniques in Postgres - HyperLogLog (Craig Kerstiens)]
* Secure your webapp passwords in PostgreSQL (Magnus Hagander)
* SQL database server Requests from the Browser (David Keeney)
* Using JSON with Postgres 9.3 (Andrew Dunstan)
* Using Postgres FDW w/ a sharded PostgreSQL Database (Stephen Frost)
* [http://www.keithf4.com/wp-content/uploads/2013/09/pg_open2013_when_pg_cant_you_can.pdf When PostgreSQL Can't, You Can (Keith Fiske)]
* [http://bit.ly/1eddml6 Writing portable postgreSQL-backed network applications for the open cloud (Ryan Jarvinen)]

Postgres Open 2013

2013-09-20T15:19:37Z

Craigkerstiens: /* Wednesday September 18 */

Slides for talks will be linked here!

Events shown for placeholding purposes. See the "Editing help" for wiki formatting tips.

== Monday September 16 ==
* PostgreSQL Data Type Safari
* Surviving Server Overload
* Database Normalization for Everybody
* [https://speakerdeck.com/zzzeek/introduction-to-sqlalchemy-and-orms-pgopen-2013 Introduction to SQLAlchemy]

== Tuesday September 17 ==
* [http://baumler.com/postgres/ Case Study: Converting the CATME application from MySQL to Postgres]
* [http://speakerdeck.com/peterg/concurrency-in-postgres-1 Concurrency in Postgres]
* [http://phaedrusdeinus.org/talks/federate_fdw/federate_fdw.pdf Federating Queries Using postgres_fdw]
* Keynote
* [http://scanningpages.files.wordpress.com/2013/09/puppet_pgopen2013.pdf Managing PostgreSQL with Puppet]
* Middleware is Dead
* My experience with embedding PostgreSQL
* [https://docs.google.com/file/d/0B2hio52iFB7AaHB0VEhaYmtZMms/edit?pli=1 New approach to resolve Object-relational impedance mismatch]
* Postgres what they really use
* [http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013 Postgres-XC dynamic cluster management]
* [https://sites.google.com/site/robertmhaas/presentations/Query%20Planning%20Gone%20Wrong.odp?attredirects=0&d=1 Query Planning Gone Wrong]
* Recovering from a Damaged Cluster
* [https://speakerdeck.com/selenamarie/sane-schema-management-with-alembic Sane Schema Management With Alembic]
* [http://www.slideshare.net/denishpatel/scaling-postgres Scaling Postgres]
* [http://t.co/q0dQZJg2cJ Taking Advantage of Custom Background Workers]
* What's New in PostgreSQL 9.3

=== Lightning Talks ===
* [http://tinyurl.com/pgopen2013-ipy-sql IPython Notebook: your new SQL client]
* [https://wiki.postgresql.org/images/d/d4/Json-speed-user-groups.pdf JSON, Speed, and User Groups]
* [http://www.slideshare.net/markwkm/ohai-my-name-is-chelnik-postgres-open-2013-report OHAI, my name is Chelnik!] Postgres Open 2013 Report
* [http://www.slideshare.net/denishpatel/choosing-the-d Choosing the "D"/Doctor]

== Wednesday September 18 ==
* A Comparison of PostgreSQL Encryption Options
* Geographically Distributed Parallel Databases – A Reality!
* [https://wiki.postgresql.org/wiki/File:Postgres_Is_Different_From_Better_Than_Your_RDBMS.pdf How Postgres is different from (read BETTER than) your database]
* [http://momjian.us/main/writings/pgsql/inside_shmem.pdf Inside PostgreSQL Shared Memory]
* JackDB - Database Development In Your Web Browser
* Multi-Database Collaboration and Extract-Transform-Load with HTSQL
* NewSQL: Leveraging PostgreSQL
* Postgres Plus Cloud Database
* [http://www.consistentstate.com/pdf/backup-and-recovery-methods.pdf PostgreSQL Backup and Recovery Methods]
* [https://wiki.postgresql.org/wiki/File:PostgresOnZFS-PGO.pdf PostgreSQL on ZFS: backup, replication, disaster recovery, and more]
* Replication Refresh
* [https://speakerdeck.com/craigkerstiens/scalable-uniques-in-postgres-with-hll Scalable Uniques in Postgres - HyperLogLog]
* Secure your webapp passwords in PostgreSQL
* SQL database server Requests from the Browser
* Using JSON with Postgres 9.3
* Using Postgres FDW w/ a sharded PostgreSQL Database
* [http://www.keithf4.com/wp-content/uploads/2013/09/pg_open2013_when_pg_cant_you_can.pdf When PostgreSQL Can't, You Can]
* [http://bit.ly/1eddml6 Writing portable postgreSQL-backed network applications for the open cloud]

Postgres Open 2013

2013-09-20T15:18:14Z

Craigkerstiens: /* Wednesday September 18 */

Slides for talks will be linked here!

Events shown for placeholding purposes. See the "Editing help" for wiki formatting tips.

== Monday September 16 ==
* PostgreSQL Data Type Safari
* Surviving Server Overload
* Database Normalization for Everybody
* [https://speakerdeck.com/zzzeek/introduction-to-sqlalchemy-and-orms-pgopen-2013 Introduction to SQLAlchemy]

== Tuesday September 17 ==
* [http://baumler.com/postgres/ Case Study: Converting the CATME application from MySQL to Postgres]
* [http://speakerdeck.com/peterg/concurrency-in-postgres-1 Concurrency in Postgres]
* [http://phaedrusdeinus.org/talks/federate_fdw/federate_fdw.pdf Federating Queries Using postgres_fdw]
* Keynote
* [http://scanningpages.files.wordpress.com/2013/09/puppet_pgopen2013.pdf Managing PostgreSQL with Puppet]
* Middleware is Dead
* My experience with embedding PostgreSQL
* [https://docs.google.com/file/d/0B2hio52iFB7AaHB0VEhaYmtZMms/edit?pli=1 New approach to resolve Object-relational impedance mismatch]
* Postgres what they really use
* [http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013 Postgres-XC dynamic cluster management]
* [https://sites.google.com/site/robertmhaas/presentations/Query%20Planning%20Gone%20Wrong.odp?attredirects=0&d=1 Query Planning Gone Wrong]
* Recovering from a Damaged Cluster
* [https://speakerdeck.com/selenamarie/sane-schema-management-with-alembic Sane Schema Management With Alembic]
* [http://www.slideshare.net/denishpatel/scaling-postgres Scaling Postgres]
* [http://t.co/q0dQZJg2cJ Taking Advantage of Custom Background Workers]
* What's New in PostgreSQL 9.3

=== Lightning Talks ===
* [http://tinyurl.com/pgopen2013-ipy-sql IPython Notebook: your new SQL client]
* [https://wiki.postgresql.org/images/d/d4/Json-speed-user-groups.pdf JSON, Speed, and User Groups]
* [http://www.slideshare.net/markwkm/ohai-my-name-is-chelnik-postgres-open-2013-report OHAI, my name is Chelnik!] Postgres Open 2013 Report
* [http://www.slideshare.net/denishpatel/choosing-the-d Choosing the "D"/Doctor]

== Wednesday September 18 ==
* A Comparison of PostgreSQL Encryption Options
* Geographically Distributed Parallel Databases – A Reality!
* [https://wiki.postgresql.org/wiki/File:Postgres_Is_Different_From_Better_Than_Your_RDBMS.pdf How Postgres is different from (read BETTER than) your database]
* [http://momjian.us/main/writings/pgsql/inside_shmem.pdf Inside PostgreSQL Shared Memory]
* JackDB - Database Development In Your Web Browser
* Multi-Database Collaboration and Extract-Transform-Load with HTSQL
* NewSQL: Leveraging PostgreSQL
* Postgres Plus Cloud Database
* [http://www.consistentstate.com/pdf/backup-and-recovery-methods.pdf PostgreSQL Backup and Recovery Methods]
* [https://wiki.postgresql.org/wiki/File:PostgresOnZFS-PGO.pdf PostgreSQL on ZFS: backup, replication, disaster recovery, and more]
* Replication Refresh
* Scalable Uniques in Postgres - HyperLogLog
* Secure your webapp passwords in PostgreSQL
* SQL database server Requests from the Browser
* [https://speakerdeck.com/craigkerstiens/scalable-uniques-in-postgres-with-hll Using JSON with Postgres 9.3]
* Using Postgres FDW w/ a sharded PostgreSQL Database
* [http://www.keithf4.com/wp-content/uploads/2013/09/pg_open2013_when_pg_cant_you_can.pdf When PostgreSQL Can't, You Can]
* [http://bit.ly/1eddml6 Writing portable postgreSQL-backed network applications for the open cloud]

FOSDEM 2013

2013-02-06T14:52:04Z

Craigkerstiens: /* PostgreSQL FOSDEM Devroom */

==Schedule==

Schedule is available at the FOSDEM website: https://fosdem.org/2013/schedule/track/postgresql/

== PGDay FOSDEM 2013 ==

We are also organizing one-day event: https://wiki.postgresql.org/wiki/PGDay_FOSDEM_2013

==Presentations==

Slides of the presentations:

== Talks: Friday, February 1st, 2013 ==

=== Radisson Blu ===

* Welcome (Magnus Hagander)
* Sqitch: VCS-enabled database change management (Ronan Dunklau)
* Postgresql 9.2 FTS Solutions (Emanuel Calvo) [http://de.slideshare.net/ecalvofranco/postgresql-fts-solutions?utm_source=slideshow&utm_medium=email&utm_campaign=upload_digest slideshare.net]
* Pagination done the PostgreSQL way (Markus Winand) [[File:Pagination_Done_the_PostgreSQL_Way.pdf]]
* Announcements FOSDEM PGDay 2013 (Dave Page, Jean-Paul Argudo)
* 3D an exact geometries for PostGIS (Hugo Mercier) [[File:Postgis 3d pgday2013 hm.pdf]]
* Understanding PostgreSQL timelines (Heikki Linnakangas)
* Maintaining Very Large Databases (VLDs) (Devrim GÜNDÜZ)

== Talks: Sunday, February 3rd, 2013 ==

=== PostgreSQL FOSDEM Devroom ===

* openbarter, a possible solution for ecological regulation (Olivier Chaussavoine)
* [http://tapoueh.org/images/confs/Fosdem2013_Event_Triggers.pdf Event Triggers] (Dimitri Fontaine)
* [https://github.com/Oslandia/presentations/blob/master/fosdem_2013/pgdevroom/fosdem_2013_pgdevroom_vincent_picavet_postgis20.pdf PostGIS 2.0 and beyond] (Vincent Picavet)
* Making apt.postgresql.org a reality (Christoph Berg)
* [https://speakerdeck.com/craigkerstiens/postgres-demystified Postgres Demystified] (Craig Kerstiens)
* PostgreSQL as a Schemaless Database (Christophe Pettus)
* [http://tapoueh.org/images/confs/Fosdem2013_High_Availability.pdf Implementing High Availability] (Dimitri Fontaine)
* Practical Tips for Better PostgreSQL Applications (Marc Balmer)

Speakers: please upload your slides to the wiki and add a link above.

[[Category:PostgreSQL Events]]

FOSDEM 2013

2013-02-06T14:51:42Z

Craigkerstiens: /* PostgreSQL FOSDEM Devroom */

==Schedule==

Schedule is available at the FOSDEM website: https://fosdem.org/2013/schedule/track/postgresql/

== PGDay FOSDEM 2013 ==

We are also organizing one-day event: https://wiki.postgresql.org/wiki/PGDay_FOSDEM_2013

==Presentations==

Slides of the presentations:

== Talks: Friday, February 1st, 2013 ==

=== Radisson Blu ===

* Welcome (Magnus Hagander)
* Sqitch: VCS-enabled database change management (Ronan Dunklau)
* Postgresql 9.2 FTS Solutions (Emanuel Calvo) [http://de.slideshare.net/ecalvofranco/postgresql-fts-solutions?utm_source=slideshow&utm_medium=email&utm_campaign=upload_digest slideshare.net]
* Pagination done the PostgreSQL way (Markus Winand) [[File:Pagination_Done_the_PostgreSQL_Way.pdf]]
* Announcements FOSDEM PGDay 2013 (Dave Page, Jean-Paul Argudo)
* 3D an exact geometries for PostGIS (Hugo Mercier) [[File:Postgis 3d pgday2013 hm.pdf]]
* Understanding PostgreSQL timelines (Heikki Linnakangas)
* Maintaining Very Large Databases (VLDs) (Devrim GÜNDÜZ)

== Talks: Sunday, February 3rd, 2013 ==

=== PostgreSQL FOSDEM Devroom ===

* openbarter, a possible solution for ecological regulation (Olivier Chaussavoine)
* [http://tapoueh.org/images/confs/Fosdem2013_Event_Triggers.pdf Event Triggers] (Dimitri Fontaine)
* [https://github.com/Oslandia/presentations/blob/master/fosdem_2013/pgdevroom/fosdem_2013_pgdevroom_vincent_picavet_postgis20.pdf PostGIS 2.0 and beyond] (Vincent Picavet)
* Making apt.postgresql.org a reality (Christoph Berg)
* [Postgres Demystified https://speakerdeck.com/craigkerstiens/postgres-demystified] (Craig Kerstiens)
* PostgreSQL as a Schemaless Database (Christophe Pettus)
* [http://tapoueh.org/images/confs/Fosdem2013_High_Availability.pdf Implementing High Availability] (Dimitri Fontaine)
* Practical Tips for Better PostgreSQL Applications (Marc Balmer)

Speakers: please upload your slides to the wiki and add a link above.

[[Category:PostgreSQL Events]]

FOSDEM 2013

2013-02-06T14:51:22Z

Craigkerstiens: /* PostgreSQL FOSDEM Devroom */

==Schedule==

Schedule is available at the FOSDEM website: https://fosdem.org/2013/schedule/track/postgresql/

== PGDay FOSDEM 2013 ==

We are also organizing one-day event: https://wiki.postgresql.org/wiki/PGDay_FOSDEM_2013

==Presentations==

Slides of the presentations:

== Talks: Friday, February 1st, 2013 ==

=== Radisson Blu ===

* Welcome (Magnus Hagander)
* Sqitch: VCS-enabled database change management (Ronan Dunklau)
* Postgresql 9.2 FTS Solutions (Emanuel Calvo) [http://de.slideshare.net/ecalvofranco/postgresql-fts-solutions?utm_source=slideshow&utm_medium=email&utm_campaign=upload_digest slideshare.net]
* Pagination done the PostgreSQL way (Markus Winand) [[File:Pagination_Done_the_PostgreSQL_Way.pdf]]
* Announcements FOSDEM PGDay 2013 (Dave Page, Jean-Paul Argudo)
* 3D an exact geometries for PostGIS (Hugo Mercier) [[File:Postgis 3d pgday2013 hm.pdf]]
* Understanding PostgreSQL timelines (Heikki Linnakangas)
* Maintaining Very Large Databases (VLDs) (Devrim GÜNDÜZ)

== Talks: Sunday, February 3rd, 2013 ==

=== PostgreSQL FOSDEM Devroom ===

* openbarter, a possible solution for ecological regulation (Olivier Chaussavoine)
* [http://tapoueh.org/images/confs/Fosdem2013_Event_Triggers.pdf Event Triggers] (Dimitri Fontaine)
* [https://github.com/Oslandia/presentations/blob/master/fosdem_2013/pgdevroom/fosdem_2013_pgdevroom_vincent_picavet_postgis20.pdf PostGIS 2.0 and beyond] (Vincent Picavet)
* Making apt.postgresql.org a reality (Christoph Berg)
* [Postgres Demystified](https://speakerdeck.com/craigkerstiens/postgres-demystified) (Craig Kerstiens)
* PostgreSQL as a Schemaless Database (Christophe Pettus)
* [http://tapoueh.org/images/confs/Fosdem2013_High_Availability.pdf Implementing High Availability] (Dimitri Fontaine)
* Practical Tips for Better PostgreSQL Applications (Marc Balmer)

Speakers: please upload your slides to the wiki and add a link above.

[[Category:PostgreSQL Events]]

Postgres Open 2012

2012-09-20T02:32:58Z

Craigkerstiens: /* Tuesday September 18 */

Slides for talks will be linked here!

== Monday September 17 ==

* [https://wiki.postgresql.org/wiki/File:Pg_security_20120917.odp PostgreSQL AuthN / AuthZ]

== Tuesday September 18 ==

* [https://wiki.postgresql.org/images/7/73/Range-types-pgopen-2012.pdf Range Types in PostgreSQL 9.2 - Your Life Will Never Be the Same]
* Scaling out by distributing and replicating data in Postgres-XC
* PostgreSQL When It's Not Your Job

* [http://tapoueh.org/images/fotolog.pdf Large Scale MySQL Migration to PostgreSQL]
* Choosing a logical replication system: Slony vs Bucardo
* Temporal Database Demo

* 12 Calm Years of PostgreSQL in Critical Messaging
* [https://speakerdeck.com/u/jacobian/p/the-first-postsql-database PostgreSQL in the cloud: Theory and Practice]
* [http://www.hagander.net/talks/Backup%20strategies.pdf PostgreSQL Backup Strategies]

* This Is PostGIS
* [http://plv8-pgopen.herokuapp.com Embracing the Web with JSON and PLV8]
* How Akiban Implemented a New Database Compatible with the PostgreSQL Protocol

* [[Media:Ha_postgres.pdf|High Availability with PostgreSQL and Pacemaker]]
* Logging: Not Just for Lumberjacks
* [http://stuff.coffeecode.net/2012/pgopen_fulltext/pgsql-fulltext-intro.html Full-text search - seek and ye shall find]

=== Lightning Talks ===

TBD!

== Wednesday September 19 ==

* [http://pgexperts.com/document.html?id=54 Super Jumbo Deluxe], plus [http://pgexperts.com/presentations.html more presentations here]
* [[Media:OO_approach.pdf|An object oriented approach to data driven software development]]
* TransLattice Elastic Database Architecture Deep Dive

* [http://www.pateldenish.com/2012/04/deploying-maximum-ha-architecture-with-postgresql.html Deploying maximum HA architecture with Postgres] , for pg_reorg [http://static.slidesharecdn.com/swf/ssplayer2.swf?doc=p90xyourdatabase-13010796913746-phpapp02&stripped_title=p90-x-your-database p90x talk]
* Scaling Postgres with some help from Redis
* Retail DDL

* Query Logging and Workload Analysis
* A Shared-nothing cluster system: Postgres-XC
* [http://jkshah.blogspot.com/2012/09/pgopen-2012-dvdstore-benchmark-and.html DVDStore Benchmark and PostgreSQL]

* Performance Improvements in PostgreSQL 9.2
* A Batch of Commit Batching
* [http://momjian.us/main/presentations/features.html#cte Programming the SQL Way with Common Table Expressions]

* Postgres is the new default – how we transitioned our platform at Engine Yard and why you should too
* Leveraging PLV8 in Javascript-heavy Web Applications
* PG Extractor - A smarter pg_dump