Slony-I Event Propagation

Christopher Browne

Afilias Canada

eRServer was an oversimplification

The naïve user: “But it's all about replicating data!!! Why this fixation on events???”

eRServer (and others) were too simplistic: change config and you must rebuild nodes from scratch

Slony-I supports dynamic reconfig

eRServer tried to bring replication completely up to date in one chunk

Slony-I is all about Event Propagation

Each node provides and consumes various types of events:

Configuration
Controlling creation/destruction of various sorts of Slony-I objects

Replication Control
Controlling subscriptions

SYNC
Telling subscribers about data to pull

Configuration Events

Nodes: STORE_NODE, ENABLE_NODE, DISABLE_NODE, DROP_NODE

Paths: STORE_PATH, DROP_PATH

Listen paths: STORE_LISTEN, DROP_LISTEN

Set operations: STORE_SET, DROP_SET

Operations within sets: SET_{ADD,MOVE,DROP}_TABLE, SET_{ADD,MOVE,DROP}_SEQUENCE

Subscription Events

SUBSCRIBE_SET, UNSUBSCRIBE_SET, COPY_SET, ENABLE_SUBSCRIPTION

All of these events are sent due to some need to change configuration of the cluster of Slony-I nodes.

These events all propagate throughout
the cluster

The SYNC Event

Normal operations = series of SYNC events

Nodes originating set = frequent SYNCs

A SYNC passes info about what XIDs
are active, telling the subscriber
what data to pull from sl_log_N

What's in a SYNC?

Min active XID, max active XID

List of O/S XIDs: Active Transactions

800701

800750

800772

800824

800925

801197

801165

MinXID

800701

MaxXID

801197

801025

SYNC 1, 48752

How Does Slony-I Use a SYNC?

It uses transaction visibility to determine what transactions are still under way

Once a transaction falls off the “in progress” list, it either needs to be applied in that SYNC, or was rolled back and is discarded

SYNC Grouping

If replication is way behind, efficiencies can be gained by grouping several SYNC events together and processing them together:
Query just once, Sort just once, COMMIT just once

slon “-g” and “-o” options allow tuning how Slony-I groups SYNCs together

Need SYNCs

If no slon is running, no SYNCs are issued until a slon restarts

A 4h slon outage would mean 4h of work must be applied in one BIG transaction

To keep updates more “bite-sized”, either keep slons running, or run generate_syncs.sh as a cron job

How Things Look

When everything is working right, you'll see, on origin nodes, a steady sequence of SYNCs and acknowledgements

DEBUG2 remoteWorkerThread_5: forward confirm 36,4403408 received by 5

DEBUG2 remoteWorkerThread_78: forward confirm 36,4403418 received by 78

DEBUG2 remoteWorkerThread_1: forward confirm 36,4403418 received by 1

DEBUG2 syncThread: new sl_action_seq 56254407 - SYNC 4403419

DEBUG2 localListenThread: Received event 36,4403419 SYNC

DEBUG2 remoteWorkerThread_1: forward confirm 36,4403419 received by 1

DEBUG2 remoteWorkerThread_78: forward confirm 36,4403419 received by 78

DEBUG2 syncThread: new sl_action_seq 56254422 - SYNC 4403420

DEBUG2 localListenThread: Received event 36,4403420 SYNC

DEBUG2 remoteWorkerThread_501: forward confirm 36,4403413 received by 501

DEBUG2 remoteWorkerThread_1: forward confirm 36,4403420 received by 1

DEBUG2 remoteWorkerThread_78: forward confirm 36,4403420 received by 78

DEBUG2 syncThread: new sl_action_seq 56254443 - SYNC 4403421

DEBUG2 localListenThread: Received event 36,4403421 SYNC

How Things Look – Subscriber

Subscribers see a bit more action...

DEBUG2 remoteWorkerThread_36: SYNC 4417864 processing

DEBUG2 remoteWorkerThread_36: syncing set 1 with 55 table(s) from provider 36

DEBUG2 remoteWorkerThread_36: syncing set 3 with 1 table(s) from provider 36

DEBUG2 remoteHelperThread_36_36: 0.006 seconds delay for first row

DEBUG2 remoteHelperThread_36_36: 0.008 seconds until close cursor

DEBUG2 remoteWorkerThread_36: new sl_rowid_seq value: 36000000000018051

DEBUG2 remoteWorkerThread_36: SYNC 4417864 done in 0.225 seconds

DEBUG2 remoteWorkerThread_1: forward confirm 36,4417864 received by 1

DEBUG2 remoteListenThread_36: queue event 36,4417865 SYNC

DEBUG2 remoteWorkerThread_36: Received event 36,4417865 SYNC

DEBUG2 remoteWorkerThread_36: SYNC 4417865 processing

DEBUG2 remoteWorkerThread_36: syncing set 1 with 55 table(s) from provider 36

DEBUG2 remoteWorkerThread_36: syncing set 3 with 1 table(s) from provider 36

DEBUG2 remoteHelperThread_36_36: 0.006 seconds delay for first row

DEBUG2 remoteHelperThread_36_36: 0.007 seconds until close cursor

DEBUG2 remoteWorkerThread_36: new sl_rowid_seq value: 36000000000018051

DEBUG2 remoteWorkerThread_36: SYNC 4417865 done in 0.186 seconds

DEBUG2 remoteWorkerThread_1: forward confirm 36,4417865 received by 1

Listen Paths

Table sl_listen controls which paths (sl_path) each node uses to get events from other nodes

Determines the “paths more travelled”

In 1.0.x, it was necessary to manually build the “listener network.” Blech...

Missing entries caused events not to propagate; part of the cluster falls into silence.

Listen Paths – Slony-I 1.1

In 1.1, the “listener” network is calculated automatically based on the data in sl_path and sl_subscribe.

2005-06-22T17:24:05