TPC-H
The TPC-H Benchmark is a popular one for comparing database vendors. It's possible to run the TPC-H data set on PostgreSQL without having a formal testing kit, as demonstrated by the DBT-3 project's work to provide a full kit. Commercial database vendors are famously so focused on this benchmark they will "game" TPC-H runs--add optimizations specifically aimed at it--to make absolutely sure they do well.
PostgreSQL is still missing some features needed to seriously compete on this benchmark. The parallel query features added in PostgreSQL 9.6 and later helped pull down run times for several queries, see EDB's Parallel Query in PostgreSQL and TPC-H performance 8.3-13.
Columnar storage with compression is a well known way to accelerate the TPC-H queries. As covered below, Greenplum's PostgreSQL fork claimed some improved results against PG 8.4. Crunchy claims large improvements over PG 16 with their commercial columnar Iceberg based extensions.
Early reference notes about the benchmark:
- TPC-H data generation and loading
- TPC-H Performance Study using an old version of PostgreSQL
- Discussion of limitations started by the above
- TPC-H fun with Greenplum (single node edition) - leads into the below and discusses what's needed to do well on TPC-H
- GP SNE versus Postgres 8.4.2 - 22 Query Benchmark Results (similar to TPC-H queries, tweaked to run better)
- PostgreSQL and Benchmarks TPC-H, TPC-E, and EAStress 2004 review (2008)
