Multithreading

From PostgreSQL wiki
Jump to navigationJump to search

Resources

pgsql-hackers thread: https://www.postgresql.org/message-id/31cc6df9-53fe-3cd9-af5b-ac0d801163f4%40iki.fi

PGConf.eu Presentation: https://www.postgresql.eu/events/pgconfeu2023/schedule/session/4845-multi-threaded-postgresql/

Work-in-progress git branch: https://github.com/hlinnaka/postgres/tree/threading

Ongoing work

These patches are contributing to the effort:


Done:

TODO

Global variables

Global variables in PostgreSQL fall into a few different categories:

  • per-session state
  • pointers to shared memory areas
  • other variables that are initialized at postmaster startup and never change after that
  • constants

Many global variables are used to hold GUCs, and they can fall into any of those categories depending on whether they're PGC_POSTMASTER or PGC_USERSET.

The plan:

  • Add annotations to all global variables, to mark which category they fall into. Variables holding per-session state are turned into thread-local variables. The distinction between the other categories is just for documentation purposes.
  • Provide a tool that can list all global variables that are missing the annotations (pgguclifetimes). Add that as a compile-time check ideally. This will also be useful for extension authors to find missing annotations in extensions.
    • alternative idea: could we use a simple perl script for this, like we do for the PGDLLIMPORT case?
  • Potential standalone value:
    • "modified once" annotated variables could be put in a separate binary section which would improve performance
    • Could help with the engineering overhead of figuring out where to initialize at postmaster startup

Extensions

Extensions will make the transition at their own pace

  • Add field to control file or the PG_MODULE magic to mark whether an extension supports multi-process mode only, or multi-threaded model only, or both
    • other considerations: good documentation will be key, also need to work with the extension community to build understanding and consensus
  • What about Perl, other PLs?
  • Robert volunteered to look into this at 2024.pgconf.dev

PIDs in user-facing interfaces

A few places expose PIDs to users:

  • pg_terminate_backend(<pid>)
  • pg_stat_activity
  • query cancellation

Plan:

  • Replace PID with a thread id.
  • Could use OS thread ID, but behavior isn't totally consistent across platforms
  • Consensus at 2024.pgconf.dev seemed to be that inventing our own 32-bit thread ID would be better
  • Possibly pgprocno + counter
  • Need to make sure we don't reuse thread IDs too quickly, else we might terminate the wrong backend.
  • Potential standalone value: decoupling the relationship between thread and session could enable other features
  • Robert volunteered to look into this at 2024.pgconf.dev

Unix Signals

Separate page with more details on this topic.

We use Unix signals between processes currently. It's hard to use them for inter-thread communication within the same process. When you send a signal, you send it to a process, and any thread in the process can handle it.

  • Refactor inter-process signals with something like Procsignal and latches
  • With a single-process, you cannot easily "kill <pid>" to send SIGTERM or SIGUSR1 to a single backend anymore. We can provide a "pg_ctl signal" command to replace that.
  • Description/Tasks/Components:
    • timers (which use sigalarm)
    • deadlock detection? (already done with waitlatch so nothing happening in signal handlers so nothing to do here?)
    • Sub-sub projects
      • multiplex waiting on a latch and don't rely on signals (latchification project -- replacing setlatch with interrupt driven concept)
      • Abstraction for the C11 thread functions that is portable (think about C11 features and C99 compiler)
  • Thomas and Heikki volunteered to look into this at 2024.pgconf.dev

Replace non-thread safe library functions with thread-safe variants

  • strerror() -> strerror_r()
  • getopt_long()
    • getopt_long implementation used for windows could be used for other platforms
    • rearchitect postmaster startup to change options passing
  • Make pg_strtok() re-entrant
  • not all string functions with _l have been implemented in glibc
  • Peter and Nathan volunteered to look into this at 2024.pgconf.dev

Misc

  • Add a GUC to enable multi-threaded mode
  • Replace fork() with posix_create(). Yay!
  • Make virtual filedescriptors (fd.c) work with threads
    • Heikki volunteered to look into this at 2024.pgconf.dev (with Andres identifying performance fallout)
    • increase the limit for max number of file descriptors
    • diagnose and fix the performance fallout (due to sharing VFD cache and maybe because some OSes don't cope well with large numbers of file descriptors)
  • Make bootparse.y and other bison/flex generated code re-entrant
    • main parser is already done, but others need work, which is thought to be simple
    • Peter volunteered to look into this at 2024.pgconf.dev
  • Refactoring guc_tables.c
    • currently uses memory address of global GUC variable (and that won't work anymore with threads)
  • Refactor connection acceptance
    • Currently the postmaster accepts connections and then spawns a process
    • In the threaded model we'll want 2 processes, a supervisor that only restarts the other process, and the main process with all the threads
    • But that means connection acceptance needs to be split out from supervisor tasks
  • verify_cb in be-secure-openssl.c uses a (static) global cert_errdetail variable to pass information. Refactor to use x509_store_ctx_get_ex_data

Open Questions

  • If an extension launches threads today, it might call SPI or other PostgreSQL functions from a different thread. That works today, if you're careful and only do that from one thread at at time. After replacing global variables with thread-local variables, that no longer works. Or did it work? We have things like longjmp() and stack depth checks that would already break that?
Retrieved from "https://wiki.postgresql.org/index.php?title=Multithreading&oldid=40153"