DockerizingPostgres

Dockerizing Postgres Unconference Session

This session will be to discuss the various technical changes required to make PostgreSQL on Docker a seamless experience. Looking here for anyone who is already running Postgres on Docker, or who wants to do so but hit technical obstacles.

Session Attendees: Josh Berkus

Session Notes

• Josh Berkus: shouldn't need SSH, but do need it today. Should be one host, one port, one service.
• Jeff Davis: What about data?
• Josh: ... volume management ...
• Jan Urbanski: "restart" should be "redeploy cluster"
• Josh: volume points from inside container to outside directory
• ... flocker uses ZFS instead of UnionFS ...
• Jeff: What is the advantage of Docker?
• Josh: pg_hba.conf can be in the container (improves or worsens LDAP config??)
  • replace Vagrant
  • replicate customer environment
  • packaging (extensions with dependencies, etc)
  • orchestration
  • public clouds
• Valentine Gogichasvili: Auditability
• Jan: Distribution, Handoff
• ?: Testing
• Jan: "parity"
• Josh: Node.js "cycle" where project doesn't handle packaging well, so a container is required, so fewer projects deal with packaging
• Greg Smith: flocker good for his use case
• ... so many use cases ...
• Greg: other projects will have similar persistence issues, so we can wait for Docker to solve them
• Josh: CoreOS conference panel first question was about persistence
• Ozgun: Issue of persistence similar to AWS local storage vs. EBS choice
• ... what happens when external storage is not available ...
• Josh: Freeze or EIO
• Ozgun: Logging?
• Jeff: emit_log_hook
• Jan: persistent config settings opposed to docker philosophy of immutability
• Josh: many apps take minutes to find that there's a new master to connect to
• Jan: PGBouncer reload
• Josh: software-defined networking
• Greg: graceful shutdown could take minutes
• Josh: and hard shutdown w/ async replication means data loss
• Alexander Kukushkin: pg_upgrade?
• Josh: ... explains pg_upgrade procedure involving 3 kinds of docker containers ....
• Valentine: Advantages of docker aren't directly related to production.
• Josh: orchestration doesn't strictly require containers, but problem not solved with puppet, e.g. master brought up before replica
• Josh: crystal ball says that in 4 years everyone will be using containers (not necessarily Docker)
• ... whether they want to or not ...
• ... other databases in the same boat (with some theoretical advantages for multi-master) ...
• ... lots of Docker functionality being built in linux
• Jan: nsspawn
• ?: how to distribute patches?
• Josh: new image
• Josh: we should have official images for Docker, CoreOS, etc.

Postgres on Docker Issues

Eliminating SSH

Postgres has a number of activities which still require shell access. Auto.conf elminates one of those, but there are more.

pg_hba.conf

There is currently no good way, short of using pgAdmin + Adminpack, to make changes to pg_hba.conf without ssh access.

Restart

We can reload via an SQL command, but not restart.

recovery.conf

Remastering etc. is not possible without editing recovery.conf. Alternate solutions: [1]

Volume Management

PostgreSQL needs to store its data on an external volume, which is generally not managed by Docker. How are people handling this? Flocker? Kubernetes?

Init

Do all the PostgreSQL parts work correctly if we run a container without a full init system, ala CoreOS? Has anyone tried this?

Auto-Configuration

Can we create an improved auto-configuration on startup, if we know that we're running in a container?