HP ProLiant DL380 G5 Tuning Guide
This hardware was generously donated by HP.
Upcoming tests
Linux Tunables:
- SLUB vs. SLAB (memory management)
- stats_targets (dbt3/tpc-h dependent)
- I/O Elevators
- SCSI controller tuning (might not be possible)
- physical disk partitioning
Postgres GUCs:
- number of connections
Memory
- shared_buffers
WAL
- checkpoint_segments
- checkpoint_completion_target
- turning archive_mode on and just a vanilla copy
Planner
- effective cache size
- auto vacuum
- random_page_cost
- should we look at these?
#cpu_tuple_cost = 0.01 # same scale as above #cpu_index_tuple_cost = 0.005 # same scale as above #cpu_operator_cost = 0.0025 # same scale as above
- default_statistics_target = 10 # range 1-1000
Hardware Details
- 2 x Quad Core Intel(R) Xeon(R) E5405 (2.00GHz, 1333MHz FSB, 80W) Processor
- HP 32GB Fully Buffered DIMM PC2-5300 8x4GB DR LP Memory
- HP Smart Array P800 Controller
- 8 x HP 72GB Hot Plug 2.5 SAS 15,000 rpm Hard Drive
- MSA70 with 25 x HP 72GB Hot Plug 2.5 SAS 15,000 rpm Hard Drive
Software Details
- 2.6.28-gentoo (Gentoo)
Hardware Baseline
fio
We will use fio v1.23 to get some simple system characteristics using the internal disks of the system on various file systems, software and hardware raid configurations, and volume management configurations. 5 simple tests will be run: a sequential read, sequential write, random read, random write, and read-write test.
sequential read
[seq-read] rw=read size=56g directory=/test fadvise_hint=0 blocksize=8k direct=0 numjobs=1 nrfiles=1 runtime=1h time_based exec_prerun=echo 3 > /proc/sys/vm/drop_caches
sequential write
[seq-write] rw=write size=56g directory=/test fadvise_hint=0 blocksize=8k direct=0 numjobs=1 nrfiles=1 time_based runtime=1h exec_prerun=echo 3 > /proc/sys/vm/drop_caches
random read
[random-read] rw=randread size=7g directory=/test fadvise_hint=0 blocksize=8k direct=0 numjobs=8 nrfiles=1 runtime=1h time_based exec_prerun=echo 3 > /proc/sys/vm/drop_caches
random write
[random-write] rw=randwrite size=7g directory=/test fadvise_hint=0 blocksize=8k direct=0 numjobs=8 ; this is number of files total per process nrfiles=1 runtime=1h time_based exec_prerun=echo 3 > /proc/sys/vm/drop_caches
read-write
[read-write] rw=rw rwmixread=50 size=7g directory=/test fadvise_hint=0 blocksize=8k direct=0 numjobs=8 nrfiles=1 runtime=1h time_based exec_prerun=echo 3 > /proc/sys/vm/drop_caches
Results
One Disk
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 57.9 | 89.0 | 2.3 | 6.6 | 8.0/5.6 |
ext2-noatime | 59.4 | 92.0 | 2.3 | 6.5 | 8.0/6.8 |
ext3 | 59.4 | 81.8 | 2.3 | 22.8 | 8.2/7.0 |
ext3 (noatime) | |||||
ext3 (data=journal) | 61.4 | 72.4 | 2.3 | 11.0 | 7.5/7.5 |
ext3 (data=journal,noatime} | 60.8 | 71.2 | 2.3 | 11.0 | 7.5/7.5 |
jfs | 65.4 | 97.0 | 2.3 | 4.8 | 8.2/7.0 |
jfs (noatime) | 65.5 | 95.0 | 2.3 | 4.8 | 6.8/8.0 |
xfs | 71.9 | 96.2 | * | * | * |
xfs (noatime) | 68.8 | 96.3 | 2.3 | * | * |
- - Test did not complete successfully.
Two Disks
Software Raid
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 87.0 | 177.0 | 4.2 | 11.9 | 13.6/13.6 |
ext2-noatime | 80.2 | 180.0 | 4.3 | 12.0 | 13.4/13.4 |
ext3 | 86.2 | 158 | 4.2 | 38.1 | 17.8/9.0 |
ext3 (noatime) | 88.2 | 161.0 | 4.3 | 39.6 | 13.4/13.4 |
ext3 (data=journal) | 85.8 | 83.0 | 4.2 | 17.1 | 7.9/13.9 |
ext3 (data=journal,noatime) | 87.6 | 83.4 | 4.2 | 17.1 | 10.3/14.4 |
jfs | 88.0 | 181 | 4.1 | 7.9 | 14.1/18.8 |
jfs (noatime) | 88.7 | 175 | 4.3 | 7.8 | 16.4/16.8 |
reiserfs | 96.2 | 116 | 4.3 | 64.5 | 13.6/18.2 |
reiserfs (noatime) | 92.6 | 114 | 4.3 | 64.5 | 16.0/11.4 |
Hardware Raid
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 78.8 | 166.0 | 4.3 | 12.0 | 17.8/13.4 |
ext2-noatime | 78.9 | 170.0 | 4.3 | 12.0 | 13.4/15.6 |
ext3 | 73.9 | 82.5 | 4.3 | 41.0 | 13.0/13.0 |
ext3 (noatime) | 74.5 | 77.5 | 4.3 | 48.0 | 15.2/11.0 |
ext3 (data=journal) | 78.9 | 75.9 | 4.3 | 16.6 | 15.8/15.8 |
ext3 (data=journal,noatime) | 78.0 | 75.9 | 4.3 | 16.4 | 13.8/9.9 |
jfs | 78.8 | 128.0 | 4.4 | 8.0 | 16.2/15.9 |
jfs (noatime) | 79.3 | 133.0 | 4.4 | 8.0 | 18.2/11.5 |
reiserfs | 75 | 115 | 4.32 | 61.2 | 13.1/10.1 |
reiserfs (noatime) | ND | ND | ND | ND | ND |
LVM
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 136.0 | 186.0 | 4.5 | 11.9 | 26.4/26.4 |
ext2 (noatime) | 137.0 | 190.0 | 4.5 | 12.1 | 23.2/23.2 |
ext3 | 129.0 | 161.0 | 4.5 | 38.9 | 12.8/22.8 |
ext3 (noatime) | 136.0 | 169.0 | 4.5 | 38.4 | 26.2/19.9 |
ext3 (data=journal) | 135.0 | 90.4 | 4.4 | 16.6 | 15.2/21.2 |
ext3 (data=journal,noatime) | 137.0 | 89.6 | 4.5 | 16.9 | 21.2/15.2 |
jfs | 148.0 | 180.0 | 4.5 | 7.9 | 17.9/25.0 |
jfs (noatime) | 147.0 | 180.0 | 4.5 | 8.1 | 24.9/21.4 |
Three Disks
Software RAID
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 200.0 | 43.1 | 5.8 | 4.2 | 8.2/9.9 |
ext2 (noatime) | 200.0 | 43.3 | 5.8 | 4.2 | 11.5/9.9 |
ext3 | 199.0 | 45.6 | 5.7 | 9.6 | 24.0/24.0 |
ext3 (noatime) | 201.0 | 46.3 | 5.7 | 9.5 | 4.7/12.6 |
ext3 (data=journal) | 199.0 | 24.3 | 5.8 | 4.7 | 4.7/7.3 |
ext3 (data=journal,noatime) | 200.0 | 24.4 | 5.8 | 4.6 | 6.1/8.5 |
jfs | 218.0 | 43.2 | 5.8 | 2.7 | 12.0/12.0 |
jfs (noatime) | 218.0 | 43.3 | 5.9 | 2.7 | 6.9/12.1 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 199.0 | 42.6 | 5.2 | 4.2 | 9.9/9.9 |
Hardware RAID
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | [1] | 164.0 | 5.6 | 6.2 | 7.0/8.4 |
ext2 (noatime) | 108.0 | 169.0 | 5.6 | 6.2 | 9.3/9.3 |
ext3 | 104.0 | 129.0 | 5.5 | 23.5 | 7.9/9.2 |
ext3 (noatime) | 94.6 | 131.0 | 5.5 | 23.6 | 8.1/9.4 |
ext3 (data=journal) | 107.0 | 79.2 | 5.5 | 9.7 | 8.0/5.4 |
ext3 (data=journal,noatime) | 108.0 | 79.0 | 5.5 | 9.4 | 5.3/8.0 |
jfs | 105.0 | 179.0 | 5.3 | 4.6 | 10.0/8.6 |
jfs (noatime) | 104.0 | 179.0 | 5.6 | 4.5 | 8.6/8.6 |
reiserfs | 108.0 | 76.5 | 5.6 | 58.8 | 8.3/9/7 |
reiserfs (noatime) | 109.0 | 75.3 | 5.6 | 59.0 | 9.7/8.5 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 113.0 | 170.0 | 5.8 | 6.4 | 10.9/7.9 |
ext2 (noatime) | 103.0 | 169.0 | 5.8 | 6.3 | 12.4/9.4 |
ext3 | 106.0 | 131.0 | 5.7 | 21.0 | 11.7/10.0 |
ext3 (noatime) | 112.0 | 135.0 | 5.7 | 21.4 | 11.9/10.4 |
ext3 (data=journal) | 112.0 | 74.8 | 5.7 | 8.0 | 11.7/10.3 |
ext3 (data=journal,noatime) | 112.0 | 74.2 | 5.8 | 8.1 | 11.7/5.8 |
ext3 (data=writeback) | 113.0 | 135.0 | 5.7 | 5.2 | 11.0/3.1 |
jfs | 110.0 | 178.0 | 5.5 | 4.4 | 9.5/11.1 |
jfs (noatime) | 110.0 | 172.0 | 5.8 | 4.5 | 9.8/9.5 |
reiserfs | 114.0 | 92.6 | 5.8 | 58.3 | 10.8/12.3 |
reiserfs (noatime) | 112.0 | 78.7 | 5.8 | 58.6 | 6.5/12.3 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 113.0 | 171.0 | 5.8 | 6.3 | 7.8/10.9 |
ext2 (noatime) | 108.0 | 169.0 | 5.8 | 6.3 | 7.8/10.9 |
ext3 | 109.0 | 132.0 | 5.7 | 21.0 | 10.5/9.0 |
ext3 (noatime) | 112.0 | 135.0 | 5.7 | 20.9 | 10.5/9.0 |
ext3 (data=journal) | 113.0 | 75.4 | 5.7 | 8.3 | 4.4/11.4 |
ext3 (data=journal,noatime) | 113.0 | 75.4 | 5.8 | 8.3 | 8.9/11.7 |
jfs | 110.0 | 172.0 | 5.8 | 4.5 | 8.0/11.1 |
jfs (noatime) | 111.0 | 178.0 | 5.8 | 4.6 | 6.4/9.6 |
reiserfs | 113.0 | 113.0 | 5.8 | 58.4 | 7.7/10.8 |
reiserfs (noatime) | 114.0 | 98.0 | 5.8 | 59.0 | 10.8/6.3 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 124.0 | 172.0 | 5.8 | 6.3 | 12.8/8.0 |
ext2 (noatime) | 125.0 | 174.0 | 5.8 | 6.3 | 8.0/11.2 |
ext3 | 118.0 | 131.0 | 5.7 | 21.0 | 10.7/5.1 |
ext3 (noatime) | 119.0 | 136.0 | 5.7 | 21.1 | 10.7/9.3 |
ext3 (data=journal) | 116.0 | 73.8 | 5.7 | 8.4 | 10.5/4.6 |
ext3 (data=journal,noatime) | 125.0 | 74.2 | 5.2 | 8.2 | 10.5/12.0 |
ext3 (data=writeback) | 121.0 | 134.0 | 5.7 | 5.2 | 6.6/7.9 |
ext3 (data=writeback,noatime) | 118.0 | 133.0 | 5.7 | 5.3 | 11.1/12.7 |
jfs | 121.0 | 178.0 | 5.8 | 4.6 | 8.4/9.8 |
jfs (noatime) | 120.0 | 178.0 | 5.8 | 4.5 | 9.9/9.8 |
reiserfs | 125.0 | 93.3 | 5.8 | 58.4 | 9.5/12.7 |
reiserfs (noatime) | 131.0 | 74.0 | 5.8 | 58.5 | 11.1/12.7 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 156.0 | 167.0 | 5.8 | 6.3 | 6.6/8.3 |
ext2 (noatime) | 141.0 | 171.0 | 5.8 | 6.4 | 11.6/9.9 |
ext3 | 154.0 | 133.0 | 5.7 | 21.2 | 6.3/11.0 |
ext3 (noatime) | 155.0 | 132.0 | 5.2 | 20.7 | 9.5/12.6 |
ext3 (data=journal) | 156.0 | 76.1 | 5.7 | 8.3 | 12.4/9.3 |
ext3 (data=journal,noatime) | 155.0 | 74.8 | 5.8 | 8.4 | 12.4/9.4 |
ext3 (data=writeback) | 163.0 | 132.0 | 5.7 | 5.2 | 10.9/9.3 |
ext3 (data=writeback,noatime) | 154.0 | 132.0 | 5.2 | 5.2 | 9.8/8.2 |
jfs | 169.0 | 179.0 | 5.5 | 4.6 | 10.1/10.1 |
jfs (noatime) | 170.0 | 178.0 | 5.8 | 4.5 | 11.8/11.8 |
reiserfs | 161.0 | 95.7 | 5.8 | 58.5 | 9.8/9.8 |
reiserfs (noatime) | 163.0 | 69.3 | 5.8 | 58.5 | 5.1/11.5 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 171.0 | 172.0 | 5.8 | 6.3 | 12.0/8.5 |
ext2 (noatime) | 173.0 | 171.0 | 5.8 | 6.3 | 10.2/13.6 |
ext3 | 172.0 | 136.0 | 5.7 | 21.3 | 11.6/9.8 |
ext3 (noatime) | 175.0 | 133.0 | 5.8 | 20.8 | 8.2/13.1 |
ext3 (data=journal) | 172.0 | 75.5 | 5.8 | 8.2 | 4.8/11.3 |
ext3 (data=journal,noatime) | 171.0 | 75.3 | 5.8 | 8.3 | 12.4/9.4 |
ext3 (data=writeback) | 178.0 | 135.0 | 5.7 | 5.2 | 11.4/9.7 |
ext3 (data=writeback,noatime) | 181.0 | 133.0 | 5.7 | 5.2 | 6.7/13.5 |
jfs | 188.0 | 172.0 | 5.8 | 4.4 | 14.0/10.5 |
jfs (noatime) | 187.0 | 178.0 | 5.5 | 4.3 | 14.0/12.3 |
reiserfs | 185.0 | 104.0 | 5.8 | 58.4 | 13.6/3.4 |
reiserfs (noatime) | 187.0 | 81.0 | 5.8 | 58.6 | 10.2/8.5 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 174.0 | 172.0 | 5.8 | 6.3 | 9.9/6.6 |
ext2 (noatime) | 171.0 | 165.0 | 5.8 | 6.3 | 8.6/10.4 |
ext3 | 172.0 | 135.0 | 5.7 | 20.7 | 13.3/8.3 |
ext3 (noatime) | 172.0 | 137.0 | 5.7 | 21.2 | 11.7/11.7 |
ext3 (data=journal) | 173.0 | 76.1 | 5.8 | 8.1 | 8.2/8.3 |
ext3 (data=journal,noatime) | 173.0 | 76.7 | 5.8 | 8.2 | 12.7/9.5 |
ext3 (data=writeback) | 173.0 | 133.0 | 5.7 | 5.2 | 10.3/10.3 |
ext3 (data=writeback,noatime) | 173.0 | 134.0 | 5.8 | 5.2 | 13.7/8.6 |
jfs | 174.0 | 173.0 | 5.8 | 4.4 | 12.5/5.4 |
jfs (noatime) | 173.0 | 178.0 | 5.8 | 4.5 | 12.5/12.5 |
reiserfs | 174.0 | 113.0 | 5.8 | 58.4 | 12.2/12.2 |
reiserfs (noatime) | 174.0 | 75.0 | 5.8 | 58.6 | 13.9/12.2 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 176.0 | 170.0 | 5.8 | 6.3 | 13.9/13.9 |
ext2 (noatime) | 187.0 | 173.0 | 5.4 | 6.3 | 5.4/14.0 |
ext3 | 179.0 | 135.0 | 5.7 | 21.1 | 8.4/13.5 |
ext3 (noatime) | 178.0 | 133.0 | 5.7 | 21.0 | 8.4/13.5 |
ext3 (data=journal) | 178.0 | 74.3 | 5.8 | 7.9 | 11.3/9.7 |
ext3 (data=journal,noatime) | 177.0 | 76.0 | 5.8 | 8.2 | 8.4/11.7 |
ext3 (data=writeback) | 175.0 | 134.0 | 5.7 | 5.2 | 8.7/12.1 |
ext3 (data=writeback,noatime) | 186.0 | 133.0 | 5.2 | 5.2 | 7.0/12.1 |
jfs | 185.0 | 172.0 | 5.8 | 4.5 | 12.7/12.8 |
jfs (noatime) | 185.0 | 172.0 | 5.8 | 4.5 | 9.1/14.5 |
reiserfs | 188.0 | 82.6 | 5.8 | 58.6 | 14.0/12.3 |
reiserfs (noatime) | 191.0 | 88.5 | 5.8 | 58.3 | 8.9/12.3 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 183.0 | 170.0 | 5.8 | 6.3 | 7.0/12.3 |
ext2 (noatime) | 183.0 | 171.0 | 5.8 | 6.3 | 8.8/8.8 |
ext3 | 177.0 | 136.0 | 5.7 | 21.2 | 10.2/11.9 |
ext3 (noatime) | 174.0 | 134.0 | 5.7 | 21.1 | 8.6/11.9 |
ext3 (data=journal) | 171.0 | 75.7 | 5.7 | 8.2 | 5.1/11.8 |
ext3 (data=journal,noatime) | 185.0 | 76.1 | 5.8 | 7.9 | 11.8/11.8 |
ext3 (data=writeback) | 185.0 | 131.0 | 5.7 | 5.2 | 8.7/12.2 |
ext3 (data=writeback,noatime) | 186.0 | 131.0 | 5.7 | 5.2 | 13.9/10.4 |
jfs | 193.0 | 172.0 | 5.8 | 4.5 | 9.2/12.8 |
jfs (noatime) | 194.0 | 172.0 | 5.8 | 4.5 | 11.0/9.2 |
reiserfs | 194.0 | 98.0 | 5.8 | 58.6 | 12.8/12.7 |
reiserfs (noatime) | 195.0 | 92.3 | 5.8 | 58.7 | 14.3/12.5 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 183.0 | 168.0 | 5.8 | 6.3 | 13.4/10.9 |
ext2 (noatime) | 187.0 | 162.0 | 5.8 | 6.3 | 12.9/9.2 |
ext3 | 173.0 | 132.0 | 5.7 | 21.1 | 12.0/8.7 |
ext3 (noatime) | 168.0 | 135.0 | 5.8 | 20.8 | 7.0/12.0 |
ext3 (data=journal) | 184.0 | 73.9 | 5.7 | 8.1 | 8.4/10.1 |
ext3 (data=journal,noatime) | 182.0 | 76.0 | 5.8 | 8.3 | 11.8/8.4 |
ext3 (data=writeback) | 180.0 | 133.0 | 5.7 | 5.2 | 11.3/7.3 |
ext3 (data=writeback,noatime) | 181.0 | 135.0 | 5.7 | 5.2 | 11.0/7.6 |
jfs | 204.0 | 172.0 | 5.8 | 4.5 | 8.0/15.6 |
jfs (noatime) | 204.0 | 173.0 | 5.8 | 4.4 | 15.6/9.7 |
reiserfs | 199.0 | 83.6 | 5.8 | 58.7 | 13.3/13.3 |
reiserfs (noatime) | 199.0 | 113.0 | 5.8 | 58.4 | 11.4/13.3 |
Four Disks
Software RAID
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 262.0 | 178.1 | 7.8 | 54.9 | 20.0/28.0 |
ext2 (noatime) | 262.0 | 183.0 | 6.9 | 56.9 | 26.5/19.8 |
ext3 | 168.0 | 140.0 | 7.6 | 28.2 | 22.8/27.4 |
ext3 (noatime) | 269.0 | 135.0 | 7.8 | 28.2 | 23.3/24.1 |
ext3 (data=journal) | 262.0 | 61.7 | 7.6 | 11.1 | 19.1/19.1 |
ext3 (data=journal,noatime) | 263.0 | 61.5 | 7.8 | 10.0 | 19.0/15.8 |
jfs | 329.0 | 162.0 | 7.8 | 7.2 | 25.3/25.3 |
jfs (noatime) | 328.0 | 163.0 | 7.8 | 7.3 | 33.7/21.0 |
reiserfs | 316.0 | 88.0 | 7.6 | 32.4 | 24.3/28.5 |
reiserfs (noatime) | 318.0 | 84.6 | 7.8 | 32.3 | 20.1/28.2 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 239.0 | 179.0 | 7.8 | 55.7 | 27.2/23.8 |
Hardware RAID
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 132.0 | 170.0 | 7.5 | 11.5 | 16.5/18.7 |
ext2 (noatime) | 131.0 | 172.0 | 7.6 | 11.9 | 16.5/18.7 |
ext3 | 127.0 | 115.0 | 7.4 | 40.5 | 6.7/15.7 |
ext3 (noatime) | 127.0 | 115.0 | 7.5 | 40.7 | 14.0/14.0 |
ext3 (data=journal) | 132.0 | 73.2 | 7.5 | 17.3 | 10.7/14.9 |
ext3 (data=journal,noatime) | 127.0 | 115.5 | 7.5 | 40.7 | 14.0/14.0 |
jfs | 135.0 | 145.0 | 7.6 | 32.4 | 24.3/28.5 |
jfs (noatime) | 137.0 | 148.0 | 7.6 | 8.1 | 12.8/12.6 |
reiserfs | 127.0 | 62.1 | 7.5 | 59.3 | 17.0/17.4 |
reiserfs (noatime) | 130.0 | 63.9 | 7.6 | 59.5 | 17.0/15.2 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 123.0 | 170.0 | 7.6 | 11.7 | 20.6/18.1 |
ext2 (noatime) | 122.0 | 172.0 | 7.6 | 11.7 | 15.5/15.5 |
ext3 | 117.0 | 122.0 | 7.5 | 38.5 | 17.0/12.9 |
ext3 (noatime) | 112.0 | 104.0 | 7.6 | 38.9 | 19.5/14.7 |
ext3 (data=journal) | 122.0 | 71.4 | 7.6 | 16.8 | 13.3/11.1 |
ext3 (data=journal,noatime) | 122.0 | 69.8 | 7.6 | 16.9 | 13.3/15.5 |
jfs | 122.0 | 146.0 | 7.7 | 8.0 | 18.2/13.0 |
jfs (noatime) | 123.0 | 146.0 | 7.7 | 8.0 | 18.3/15.7 |
reiserfs | 120.0 | 67.2 | 7.6 | 59.7 | 13.0/20.3 |
reiserfs (noatime) | 120.0 | 75.1 | 7.7 | 59.4 | 17.8/15.3 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 131.0 | 172.0 | 7.6 | 11.5 | 15.1/12.3 |
ext2 (noatime) | 131.0 | 172.0 | 7.6 | 11.6 | 9.9/19.9 |
ext3 | 127.0 | 115.0 | 7.4 | 40.1 | 16.3/9.3 |
ext3 (noatime) | 129.0 | 114.0 | 7.4 | 40.0 | 16.3/16.3 |
ext3 (data=journal) | 131.0 | 73.0 | 7.5 | 17.2 | 12.8/13.1 |
ext3 (data=journal,noatime) | 131.0 | 74.1 | 7.5 | 17.0 | 12.8/14.9 |
jfs | 133.0 | 148.0 | 7.6 | 8.2 | 20.2/12.6 |
jfs (noatime) | 134.0 | 149.0 | 7.6 | 7.8 | 20.3/15.2 |
reiserfs | 126.0 | 58.7 | 7.5 | 59.7 | 17.0/17.4 |
reiserfs (noatime) | 129.0 | 61.6 | 7.6 | 59.0 | 19.5/12.2 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 159.0 | 173.0 | 7.6 | 11.6 | 19.8/15.1 |
ext2-noatime | 158.0 | 170.0 | 7.6 | 11.7 | 17.0/19.8 |
ext3 | 155.0 | 120.0 | 7.5 | 38.7 | 12.1/12.1 |
ext3-journal | 157.0 | 71.9 | 7.6 | 16.9 | 11.4/16.0 |
ext3-journal-noatime | 146.0 | 70.2 | 7.6 | 16.7 | 11.4/15.9 |
ext3-noatime | 157.0 | 122.0 | 7.5 | 38.6 | 17.6/20.1 |
ext3-writeback | 157.0 | 117.0 | 7.6 | 7.7 | 17.3/10.2 |
ext3-writeback-noatime | 158.0 | 118.0 | 7.6 | 7.2 | 17.2/12.3 |
jfs | 161.0 | 150.0 | 7.7 | 8.2 | 14.4/17.3 |
jfs-noatime | 166.0 | 150.0 | 7.3 | 8.3 | 14.4/17.3 |
reiserfs | 158.0 | 65.8 | 7.6 | 59.6 | 17.9/17.4 |
reiserfs-noatime | 162.0 | 76.2 | 7.7 | 60.3 | 14.9/12.4 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 163.0 | 172.0 | 7.6 | 11.6 | 21.0/15.0 |
ext2-noatime | 164.0 | 169.0 | 7.6 | 11.7 | 15.5/15.0 |
ext3 | 163.0 | 121.0 | 7.6 | 39.1 | 15.5/15.5 |
ext3-journal | 161.0 | 72.1 | 7.6 | 17.1 | 17.5/8.8 |
ext3-journal-noatime | 162.0 | 71.2 | 7.6 | 16.6 | 15.4/11.3 |
ext3-noatime | 163.0 | 127.0 | 7.6 | 38.1 | 13.0/18.2 |
ext3-writeback | 161.0 | 118.0 | 7.6 | 7.7 | 20.6/15.5 |
ext3-writeback-noatime | 161.0 | 118.0 | 7.6 | 7.3 | 18.1/18.1 |
jfs | 171.0 | 146.0 | 7.7 | 8.0 | 22.2/25.4 |
jfs-noatime | 169.0 | 150.0 | 7.3 | 8.2 | 25.4/15.9 |
reiserfs | 167.0 | 63.8 | 7.6 | 59.3 | 18.2/18.2 |
reiserfs-noatime | 170.0 | 68.9 | 7.7 | 59.8 | 18.2/18.2 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 169.0 | 171.0 | 7.6 | 11.5 | 12.5/24.9 |
ext2-noatime | 168.0 | 171.0 | 7.6 | 11.6 | 15.9/22.5 |
ext3 | 169.0 | 117.0 | 7.5 | 38.9 | 15.7/15.7 |
ext3-journal | 168.0 | 69.4 | 6.9 | 17.1 | 11.1/15.6 |
ext3-journal-noatime | 168.0 | 71.6 | 7.6 | 17.4 | 13.5/11.1 |
ext3-noatime | 168.0 | 125.0 | 7.6 | 38.9 | 15.7/15.8 |
ext3-writeback | 165.0 | 118.0 | 6.9 | 7.6 | 16.0/18.6 |
ext3-writeback-noatime | 168.0 | 120.0 | 7.6 | 7.7 | 15.9/13.2 |
jfs | 174.0 | 147.0 | 7.7 | 8.3 | 23.5/23.5 |
jfs-noatime | 170.0 | 150.0 | 7.7 | 7.9 | 20.2/20.2 |
reiserfs | 172.0 | 62.5 | 7.6 | 60.0 | 16.9/13.5 |
reiserfs-noatime | 173.0 | 64.6 | 7.7 | 59.5 | 16.7/16.1 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 174.0 | 172.0 | 7.6 | 11.6 | 22.9/19.6 |
ext2-noatime | 173.0 | 171.0 | 7.6 | 11.6 | 18.4/18.4 |
ext3 | 174.0 | 122.0 | 7.6 | 38.7 | 16.7/17.1 |
ext3-journal | 174.0 | 71.3 | 7.6 | 16.7 | 11.3/18.1 |
ext3-journal-noatime | 172.0 | 70.9 | 7.6 | 16.8 | 15.4/17.6 |
ext3-noatime | 173.0 | 123.0 | 7.6 | 38.3 | 19.1/13.6 |
ext3-writeback | 174.0 | 117.0 | 7.6 | 7.4 | 13.8/19.3 |
ext3-writeback-noatime | 173.0 | 117.0 | 7.6 | 7.6 | 16.6/13.8 |
jfs | 178.0 | 146.0 | 7.7 | 8.1 | 27.5/17.2 |
jfs-noatime | 174.0 | 150.0 | 7.7 | 8.2 | 20.6/24.1 |
reiserfs | 178.0 | 82.9 | 7.6 | 59.6 | 19.7/19.7 |
reiserfs-noatime | 178.0 | 67.1 | 7.7 | 59.4 | 17.0/22.6 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
ext2 | 172.0 | 172.0 | 6.9 | 11.6 | 19.9/23.2 |
ext2-noatime | 170.0 | 172.0 | 7.6 | 11.6 | 23.2/20.0 |
ext3 | 173.0 | 118.0 | 7.6 | 39.1 | 11.3/16.8 |
ext3-journal | 171.0 | 71.5 | 7.6 | 16.8 | 13.6/11.4 |
ext3-journal-noatime | 171.0 | 70.7 | 7.6 | 16.8 | 13.6/11.3 |
ext3-noatime | 174.0 | 119.0 | 7.6 | 38.8 | 19.8/17.0 |
ext3-writeback | 171.0 | 119.0 | 7.6 | 7.3 | 16.6/19.5 |
ext3-writeback-noatime | 171.0 | 117.0 | 7.6 | 7.3 | 14.1/22.3 |
jfs | 188.0 | 149.0 | 7.3 | 8.3 | 17.5/24.4 |
jfs-noatime | 187.0 | 150.0 | 7.7 | 8.2 | 17.4/20.9 |
reiserfs | 185.0 | 62.6 | 7.6 | 60.1 | 23.0/11.5 |
reiserfs-noatime | 186.0 | 73.1 | 7.7 | 59.9 | 11.6/23.0 |
sequential read | sequential write | random read | random write | read-write | |
---|---|---|---|---|---|
reiserfs | 189.0 | 67.5 | 7.6 | 60.0 | 14.4/17.2 |
reiserfs-noatime | 188.0 | 73.3 | 7.7 | 59.6 | 20.1/17.2 |
Analysis
Hardware vs. Software Raid
Sequential Writes
One Disk | Two Disk Software RAID-0 (64KB stripe) | Two Disk Hardware RAID-0 (128KB stripe) | LVM (4MB stripe) | Three Disk Software RAID-5 (4MB stripe) | Three Disk Hardware RAID-5 (128KB stripe) | Three Disk Hardware RAID-5 (256KB stripe) | Three Disk Hardware RAID-5 (256KB stripe) w/ partition table | Four Disk Software RAID-10 (4MB stripe) | Four Disk Hardware RAID-10 (128KB stripe) | Four Disk Hardware RAID-10 (256KB stripe) | Four Disk Hardware RAID-10 (128KB stripe) w/ partition table | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
ext2 | 50-100 | 89.0 | 70-95 | 177.0 | 30 | 166.0 | 40 | 186.0 | 30 | 43.1 | 50 | 164.0 | 40 | 170.0 | 40 | 171.0 | 60-100 | 178.1 | 30 | 170.0 | 30 | 170.0 | 30 | 172.0 |
Readahead Buffer Size
The readahead buffer size can be set per device by using the 'blockdev' tool as part of the util-linux package. For example, the command is 'blockdev --setra <blocks>'. Note that a block is 512 bytes. 'man blockdev' for more info.
Hardware RAID
readahead buffer size | 128KB | 256KB | 512KB | 1024KB | 2048KB | 4096KB | 8192KB | 128MB |
---|---|---|---|---|---|---|---|---|
RAID-5 | 113 | 124 | 156 | 171 | 174 | 176 | 183 | 183 |
Presentations
- Linux Plumbers Conference 2008 - Filesystem I/O From a Database Perspective
- PostgreSQL West 2008 - Linux Filesystems, RAID, and more
- SCaLE Feb 2009 - What Assuptions Make: Filesystem I/O from a database perspective
Data used for these presentation are publicly available in a git repository: http://git.postgresql.org/gitweb?p=performance-tuning.git