CustomExec

From PostgreSQL wiki
Jump to navigationJump to search

Brief Idea

The purpose of ExecNodeExtender allows extensions to replace a part of Plan node in PlannedStmt by the self defined node type, and run it as if built-in executor nodes. It enables a cutting-edge but not upstreamed feature to implement its own query execution instead of the standard one, without branching whole of the source code.

This idea tries to adds a new plan node type (ExtendExec) that is associated with several callback functions to be implemented in extension module. Existing planner_hook is a good strategic point to adjust the PlannedStmt being constructed by standard planner. Once an extension inject an ExtendExec node into the Plan tree underlying PlannedStmt, related callback shall be invoked on several points, therefore, extension can implement its own logic as a part of regular executor.

Also see the discussion in developer meeting 2013.

API Specification

TODO: add mode detailed description

Data Structure

typedef struct {
    const char *extender_name;
    ExecNodeExtender_BeginExec_function fn_begin_exec;
    ExecNodeExtender_GetNext_function   fn_getnext;
    ExecNodeExtender_ReScan_function    fn_rescan;
    ExecNodeExtender_EndExec_function   fn_end_exec;
    ExecNodeExtender_Explain_function   fn_explain;
} ENExecRoutine;
typedef struct {
    Plan    plan;
    Index   enexec_id;
    List   *private_list;
} ENExec;
typedef strcut {
    PlanState      ps;
    ENExecRoutine *routine;
    void          *private;
} ENExecState;

Registration

Index RegisterExecNodeExtender(const ENExecRoutine *routine);

It shall be usually called at shared_preload_libraries handler of extension, to register a set of callback routines associated with the self-defined exec node.

It saves a pair of name and callbacks of self-defined exec node on an internal array, and returns its index.

BeginExec

void
ExecNodeExtender_BeginExec(ENExecState *state, int eflags);

It shall be invoked at ExecInitNode(), to initialize ENExecState structure.

GetNext

TupleTableSlot *
ExecNodeExtender_GetNext(ENExecState *state);

It shall be invoked at ExecProcNode(), to return a tuple as result of execution of underlying exec nodes.

ReScan

void
ExecNodeExtender_ReScan(ENExecState *state);

It shall be invoked at ExecReScan(), to rewind this exec node.

EndExec

void
ExecNodeExtender_EndExec(ENExecState *state);

It shall be invoked at ExecEndNode(), to finalize this exec node.

Explain

void
ExecNodeExplain(ENExecState *state, ExplainState *es);

It shall be optionally invoked at ExplainNode(), to show explain text.

Possible use case

  • SeqScsn assisted by GPU
  • Sort/Aggregate assisted by GPU
  • Scan with RangeVar qualifier
  • Scan from cluster-wide relations
  • Aggregate functions assisted by CPU SIMD operators (contrib module for PoC)
  • ...and more?