PGXN v2/Service Disposition

A summary of the ambitiously-envisioned future PGXN services and architecture, followed by an examination of existing services and how they will gradually be refactored or replaced for the updated platform.

TL;DR: We intend to make the PGXN v2 metadata standard as backward compatible with the current meta spec as possible to minimize interface changes and to simplify swapping one service for another. This will be especially key for the Root Registry and its clients.

Services to build for PGXN v2

Root Registry REST API

Handles releases, indexing, UX APIs, Root Registry management, and maintenance tasks. Requires authentication and authorization.

Root Registry

Registry of rsync-able static files for all extensions, plus files listing all extensions, users, etc. Compatible with and extending the PGXN root registry. Managed by the Root Registry REST API. Read-only, no auth required.

User Management REST APIs

API for user registration, management, and credentials (including links to approved third-party identity providers); extension permissions; service registration; API tokens (OAuth2); and administration. Requires authentication and authorization.

Root Registry Search REST API

Provides additional APIs to serve the Web UX, including documentation generation and full text indexing. Read-only, no auth required.

Interactions

service to manage an event queue, post webhook requests for registered clients, and to accept requests from registered and authorized clients to update registry metadata for extensions.

Façade?

Façade service to encapsulate all these APIs under a single domain?

Web UX

Delivers all user functionality, including read-only public search, browsing of extensions, docs, users, tags, etc., and to read docs for PGXN v2 itself. Also includes support for user registration and authentication, profile editing, and extension permission management, as well as system administration tasks (user admin, mirror admin?, classification admin, etc.) for admin-authorized users.

Announcements

Interactions webhook listener that publishes release announcements.

Repo Stats

Interactions webhook listener that updates repository stats via the Interactions stats API. Also periodically updates stats (weekly?).

SDKs

Build and install extensions from source for multiple build pipelines, tools to create new extensions, manage metadata, test on multiple platforms and Postgres versions, and bundle and release. Designed for static binary CLIs; likely written in Rust or Go.

Managed binary build farm

Supporting VMs, containers, and bare metal. Runs code using the build SDK to handle the builds.

Binary build service

Tiggered by Interactions webhook calls for new releases, putting builds into queues for all supported platforms, and also placing new binary builds into the binary registry.

Binary Registry

A read-only static file service containing binary packages for all supported platforms. Managed by the Binary build service.

Additional REST APIs for the binary registry?

CLI

A command-line client that uses the SDKs for building and installing from source, installing from the binary registry, and managing development tasks and release tasks. Written in a static-compiling language that supports cross-compiling for easy shipping to multiple platforms, likely Rust or Go. Evolves from or borrows patterns and logic from the trunk CLI and pgrx.

CI/CD Plugin

A pattern to use the CLI to easily compile, install, and test extensions on a wide variety of platforms, architectures, OSes, and Postgres versions, as well as to bundle and release on PGXN. May or may not need several, but at least one: a GitHub action.

Existing Service Dispositions

pgxn-manager

Basic auth-gated web site and Postgres database for uploading extension releases, changing password, editing permissions, and administering the service, plus some static doc pages.

Disposition:
Use the Strangler fig pattern to gradually replace with the Root Registry REST API, User Management REST APIs, and Web UX services. The aim is to eventually replace it completely without the need for data migration by iterating from with the same database. Only the release API is used programmatically, and while it will change its auth protocol, it can maintain its existing interface.

pgxn-consumer

Service that publishes new release announcements on social media.

Disposition:
Continue to run as-is relying on NOTIFY messages from the Root Registry Postgres database, but eventually replace with the Announcements service, which depends on Interactions webhook calls.

master.pgxn.org

Root mirror of static JSON and zip files. Managed by pgxn-manager, including CLIs for some maintenance tasks.

Disposition:
Keep as is, but switch management from pgxn-manager to the new Root Registry REST API. Existing file locations, URL patterns, and download files will remain as-is, but more files and formats will be added over time. Rename to root.pgxn.org.

pgxn-api

Mirrors master.pgxn.org and indexes new releases for the UX to use, including doc generation, full text indexing of extensions, docs, distributions, users, and tags, plus some static documentation pages.

Disposition:
Replace with Root Registry Search REST API. Keep running at least until pgxn-site and pgxnclient cease to use it, monitoring logs for usage. Or, if there continues to be traffic, teach the new service to provide backward compatible APIs and swap it out.

pgxn-site

Thin Web HTML UX wrapper around pgxn-api.

Disposition:
Replace with the new Web UX.

pgxnclient

Python CLI for finding, downloading, installing, and enabling extensions.

Disposition:
Replace with the new CLI. May continue to be supported as long as pgxn-api runs, unless the new API provides compatible APIs the client can still use.

pgxn-utils

Ruby CLI for scaffolding an extension for development, managing metadata, bundling and releasing to PGXN. Does not currently build.

Disposition:
Replace with the new CLI.

pgxn-tools

Docker image for testing extensions on many versions of Postgres, plus simple bash scripts to bundle and release an extension on PGXN. Does not currently build.

Disposition:
Refactor to use the CLI, but eventually replace with the CI/CD Plugin for increased platform compatibility.

trunk Registry

Database and API for binary extension distribution.

Disposition:
Either evolve into the Binary Registry or at least provide code and patterns for it to adopt. May eventually be refactored to mirror the Binary Registry, or be replaced by it.

trunk CLI

Builds binaries for the trunk Registry and also finds, downloads, and installs binaries from the registry. Also has a command to publish extensions, but currently gated by pull requests.

Disposition:
Evolve into or supply patterns and logic for the new CLI, which should provide a compatible or very similar interface, to minimize the migration overhead for those who use it (e.g., Tembo Cloud).

pgt.dev

Website to discover, browse, and download binary extension packages.

Disposition:
Continue as-is until ready to either redirect people to the new PGXN Web UX --- which will include binary package information and links, docs, and classifications --- or until it can be updated to depend on the Root Registry REST API and the Root Registry Search REST API to provide its functionality.