Meson

From PostgreSQL wiki
Jump to navigationJump to search

PostgreSQL devel documentation

See "Building and Installation with Meson" section of PostgreSQL devel docs.

Autoconf:meson command translations

NOTE: Make sure meson is version 0.57 or higher

Setup and build commands

description old command new command comment
set up build tree ./configure [options] meson setup [options] [builddir] sourcedir meson only supports building out of tree
set up build tree for visual studio perl src/tools/msvc/mkvcbuild.pl meson setup --backend vs [options] [builddir] sourcedir configures build tree for one build type (debug or release or ...)
show configure options ./configure --help meson configure shows options built into meson and PostgreSQL specific options
set configure options ./configure --prefix=DIR, --$somedir=DIR, --with-$option, --enable-$feature meson setup|configure -D$option=$value options can be set when setting up build tree (setup) and in existing build tree (configure)
enable cassert --enable-cassert -Dcassert=true
enable debug symbols ./configure --enable-debug meson configure|setup -Ddebug=true
specify compiler CC=compiler ./configure CC=compiler meson setup CC is only checked during meson setup, not with meson configure
set CFLAGS CFLAGS=options ./configure meson configure|setup -Dc_args=options CFLAGS is also checked, but only for meson setup
build make -s ninja ninja uses parallelism by default, launch from the root of the build tree.
build, showing compiler commands make ninja -v ninja uses parallelism by default, launch from the root of the build tree.
build the documentation make docs ninja docs ninja's default target builds everything except the docs.
install all the binaries and libraries make install ninja install use meson install --quiet for a less verbose experience
install files that changed only meson install --only-changed Routinely shaves a few hundred milliseconds off install time
clean build make clean ninja clean ninja uses parallelism by default, launch from the root of the build tree.
build documentation cd doc/ && make html && make man ninja docs Builds html documentation and man pages

Build directory

ninja tries to run from the root of the build directory. If you are not in the build directory, you can use the -C flag to have ninja "change directory" and run from there, e.g.:

ninja -C $builddir

Test related commands

description old command new command comment
list tests meson test --list Only shows tests from "tmp_install" test setup, since it is the default (--setup tmp_install is implied here)
list running/installcheck test variants meson test --setup running --list "running" test setup is used to run tests against an existing server
run all tests make check-world meson test -q --print-errorlogs runs all test, using parallelism by default
run all tests against existing server make installcheck-world meson test -q --print-errorlogs --setup running Currently makes brittle assumptions about test libraries being installed
run main regression tests make check meson test -q --print-errorlogs --suite setup --suite regress --suite setup required to get a tmp_install directory; see below
run specific contrib test suite make -C contrib/amcheck check meson test -q --print-errorlogs --suite setup --suite amcheck --suite setup required to get a tmp_install directory; see below
run a few specific Perl TAP tests meson test -q --print-errorlogs --suite setup && meson test -q --print-errorlogs recovery/013_crash_restart recovery/027_stream_regress --suite setup required to get a tmp_install directory; see below
run Perl TAP tests matching a wildcard pattern meson test -q --print-errorlogs --suite setup && meson test -q --print-errorlogs '*archive*' --suite setup required to get a tmp_install directory; see below
run main regression tests against existing server make installcheck meson test -q --print-errorlogs --setup running --suite regress-running
run specific contrib test suite against existing server make -C contrib/amcheck installcheck meson test -q --print-errorlogs --setup running --suite amcheck-running "running" amcheck suite variant doesn't include TAP tests

Test structure

When running a specific test suite against a temporary throw away installation, --suite setup should generally be specified. Otherwise the tests could end up running against a stale tmp_install directory, causing general confusion. This workaround is not required when running tests against an existing server (via the running test setup and variant test suites), since of course the installation directory being tested is whatever directory the external server installation uses.

Note that the top-level/default project name is postgresql, which is the only one we use in practice. The project name can be omitted when using a reasonably recent meson version (meson 0.46 or later), which we assume here.

You can list all of the tests from a given suite as follows:

/path/to/postgresql/build_meson $ meson test --list --suite amcheck
ninja: no work to do.
postgresql:amcheck / amcheck/regress
postgresql:amcheck / amcheck/001_verify_heapam
postgresql:amcheck / amcheck/002_cic
postgresql:amcheck / amcheck/003_cic_2pc

Note that there are distinct running/installcheck suites for most of the standard setup suites, though not all of the tests actually carry over to the running variant suites, as shown here:

/path/to/postgresql/build_meson $ meson test --list --suite amcheck-running
ninja: no work to do.
postgresql:amcheck-running / amcheck-running/regress

Running individual regression test scripts via an installcheck-tests style workflow

The Postgres autoconf build system supports running a subset of regression test scripts against an existing server using the installcheck-tests target, as shown here:

/path/to/postgresql/build_autoconf $ make installcheck-tests TESTS="test_setup create_index"
*** SNIP ***
============== dropping database "regression"         ==============
SET
DROP DATABASE
============== creating database "regression"         ==============
CREATE DATABASE
ALTER DATABASE
ALTER DATABASE
ALTER DATABASE
ALTER DATABASE
ALTER DATABASE
ALTER DATABASE
============== running regression test queries        ==============
test test_setup                   ... ok          300 ms
test create_index                 ... ok         1775 ms

=====================
 All 2 tests passed.
=====================

You can work around the current lack of an equivalent meson facility by invoking pg_regress directly:

/path/to/postgresql/build_meson $ src/test/regress/pg_regress --inputdir ../source/src/test/regress/ --dlpath=src/test/regress/ test_setup create_index
*** SNIP ***
=====================
 All 2 tests passed.
=====================

The same approach will also work for isolation tests:

/path/to/postgresql/build_meson $ src/test/isolation/pg_isolation_regress --inputdir ../source/src/test/isolation freeze-the-dead vacuum-no-cleanup-lock
*** SNIP ***
=====================
 All 2 tests passed.
=====================

Note that this assumes that the meson build directory is 'build_meson', and that the Postgres source code directory is 'source'. The 'source' directory is located in the same directory as 'build_meson' in this example (a directory layout often used with VPATH builds).

Installing Meson

FreeBSD

pkg install meson ninja

Arguments to meson setup/configure to find ports libraries:

meson setup -Dextra_lib_dirs=/opt/local/lib -Dextra_include_dirs=/opt/local/include $builddir $sourcedir

Linux

Debian / Ubuntu:

apt-get update && apt-get install -y meson ninja-build

Fedora:

dnf -y install meson ninja-build

RHEL 8:

dnf -y install dnf-plugins-core
dnf config-manager --set-enabled powertools
dnf -y install meson ninja-build

RHEL 9 (tested on Rocky Linux 9):

dnf -y install dnf-plugins-core
dnf config-manager --set-enabled crb
dnf -y install meson

macOS

With MacPorts:

sudo port install meson

Arguments to meson setup/configure to find MacPorts libraries:

meson setup -Dpkg_config_path=/opt/local/lib/pkgconfig -Dextra_lib_dirs=/opt/local/lib/ -Dextra_include_dirs=/opt/local/include $builddir $sourcedir

With Homebrew:

brew install meson

Arguments to meson setup/configure to find Homebrew libraries:

On arm64:

meson setup -Dpkg_config_path=/opt/homebrew/lib/pkgconfig -Dextra_include_dirs=/opt/homebrew/include -Dextra_lib_dirs=/opt/homebrew/lib $builddir $sourcedir

On x86-64:

meson setup -Dpkg_config_path=/usr/local/lib/pkgconfig -Dextra_include_dirs=/usr/local/include -Dextra_lib_dirs=/usr/local/lib $builddir $sourcedir

Windows

Assuming python is installed, the easiest way to get meson and ninja is:

pip install meson ninja

As documented on the meson website, a MSI installer is also available.

Using the most recent version of ActivePerl may be a bit challenging, as there is no direct access to a "perl" command except if enabling a project registered in the ActivePerl website, with a command like that:

state activate --default

An easy way to set up things is to install Chocolatey, and rely on StrawberryPerl. Here are the main packages to worry about:

choco install winflexbison
choco install sed
choco install gzip
choco install strawberryperl
choco install diffutils

The compiler detected will depend on the Command Prompt type used. For MSVC, use the command prompt installed for VS. A native Command prompt or Powershell may finish by linking to Chocolatey's gcc, which may be OK, still be careful with what's reported by meson setup.

Why and What

Autoconf is showing its age, fewer and fewer contributors know how to wrangle it. Recursive make has a lot of hard to resolve dependency issues and slow incremental rebuilds. Our home-grown MSVC buildsystem is hard to maintain for developers not using windows and runs tests serially. While these and other issues could individually be addressed with incremental improvements, together they seem best addressed by moving to a more modern buildsystem.

After evaluating different buildsystem choices, we chose to use meson, to a good degree based on the adoption by other open source projects.

We decided that it's more realistic to commit a relatively early version of the new buildsystem and mature it in tree.

The plan is to remove the msvc specific buildsystem in src/tools/msvc soon after reaching feature parity. However, we're not planning to remove the autoconf/make buildsystem in the near future. Likely we're going to keep at least the parts required for PGXS to keep working around until all supported versions build with meson.


Meson documentation


Development tree, other resources

Visualizing builds

When building with ninja, the generated .ninja_log can be uploaded to ui.perfetto.dev, which is very helpful to visualize builds.