Apt/Jenkins

From PostgreSQL wiki

< Apt
Jump to: navigation, search

Contents

Overview

Packages are built in a [Jenkins] installation at https://pgdgbuild.dus.dg-i.net/jenkins/ (login required).

We are using a slightly patched variant of Mika Prokop's [Jenkins-Debian-Glue] scripts. Our version is at https://github.com/dimitri/apt.postgresql.org/tree/master/jenkins.

For every package, there are two jobs:

  • foobar-source builds .dsc source packages from an VCS repository, which triggers
  • foobar-binaries builds .deb binary packages from the source packages

These jobs then trigger a common job:

  • dput uploads the source and binary packages to the repository host

Creating New Jobs

New jobs should use the templating mechanism provided by JoelJ's [Templating plugin].

  • Go to "Scaffolding"
  • Select the scaffold for the VCS the package is using, e.g. "git-package"
  • "Stand up" the scaffold, and fill in the variables
    • Append to Job Name: package name
    • giturl: Git URL for the package
    • package: package name
  • Go to the new source job (called like git-source-templatepackage), select configure, and rename the job to package-source
  • Go to the new binaries job (called like postgresql-binaries-templatepackage), select configure, and rename the job to package-binaries
  • Hit "build now" on the new source job
  • Watch the job's console output

Job Configuration

All "real" source/binaries jobs should be seeded from template jobs. The template jobs should then be configured as documented here.

Variables with one $ sign are jenkins variables (e.g. $JOB_NAME). Variables with two $$ signs are template variables (e.g. $$package).

Source Job Configuration

  • Jobs need to be "matrix" (multi-configuration) style jobs.
  • Project name: For the debian-jenkins-glue/pgdg scripts to work properly, source jobs must be named package-source. (There might be an extra optional suffix at the end.)
  • Discard Old Builds (optional): Max # of builds to keep: 3
  • This build is parameterized (optional):
    • String Parameter
    • Name: revision
    • Default Value: (blank)
    • Description: Number to use in the .pgdg+X package version suffix. Defaults to the .pgdg+X number found in the current changelog, or 1 otherwise.
  • Source Code Management, one of
    • Git
      • Repository URL: e.g. git://anonscm.debian.org/pkg-postgresql/pgpool2.git
        Template job: $$giturl
      • Advanced (the second "Advanced" button just above "Repository Browser"
        • "Local subdirectory for repo (optional)" (not optional): source
    • Subversion
      • ...
    • Bazaar (Needs Bazaar Plugin)
      • Repository URL: $$bzrurl
  • Configuration Matrix
    • User-defined Axis
      • Name: distribution
      • Values: sid wheezy squeeze
    • Run each configuration sequentially (recommended)
    • Execute touchstone builds first (highly recommended)
      • Filter: distribution=="sid"
  • Build Environment, one of
    • Implement Template
      • Template Name: git-source-template
      • Template Variables:
        giturl=git://anonscm.debian.org/pkg-postgresql/pgpool2.git
        package=pgpool2
        Note: this is the only place that should be edited for "Implement Template" jobs. All other settings will be overwritten on the next "Sync".
    • Make this a Template
  • Build
    • Execute Shell
      • Command: generate-pgdg-source
  • Post-build Actions
    • Archive the artifacts
      • result/*
    • Record fingerprints of files to track usage (optional)
      • Fingerprint all archived artifacts
    • IRC Notification (optional)
      • Advanced (Channels are usually configured globally, so empty here)
        • Notify for matrix projects: ONLY PARENT (recommended)
    • Trigger parameterized build of other projects
      • Projects to build: package-binaries
        Template jobs: $$package-binaries
      • Trigger build without parameters
        Note: a plain trigger would do the same, but does not work well with the Templating plugin.

Binaries Job Configuration

  • Jobs need to be "matrix" (multi-configuration) style jobs.
  • Project name: For the debian-jenkins-glue/pgdg scripts to work properly, source jobs must be named package-binaries. (There might be an extra optional suffix at the end.)
  • Discard Old Builds (optional): Max # of builds to keep: 3
  • Configuration Matrix (axis ordering does not matter, but should be consistent)
    • User-defined Axis
      • Name: architecture
      • Values: amd64 i386
    • User-defined Axis
      • Name: distribution
      • Values: sid wheezy squeeze
    • Run each configuration sequentially (recommended)
    • Execute touchstone builds first (highly recommended)
      • Filter: (distribution=="sid") && (architecture=="amd64")
  • Build Environment
    • Delete workspace before build starts (needs Jenkins Workspace Cleanup Plugin)
    • one of
    • Implement Template
      • Template Name: postgresql-binarlies-template
      • Template Variables:
        giturl=git://anonscm.debian.org/pkg-postgresql/pgpool2.git (the URL is not used in binaries jobs, but the variables should be the same for consistency)
        package=pgpool2
        Note: this is the only place that should be edited for "Implement Template" jobs. All other settings will be overwritten on the next "Sync".
    • Make this a Template
  • Build
    • Copy artifacts from another project
      • Project Name: pgpool2-source/distribution=$distribution
        Template job: $$package-source/distribution=$distribution
      • Which build: Upstream build that triggered this job
        • Use "Last successful build" as fallback
      • Artifacts to copy: result/*
      • Flatten directories
    • Execute Shell
      • Command: build-and-provide-package
  • Post-build Actions
    • Archive the artifacts
      • *.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes
    • Record fingerprints of files to track usage (optional)
      • Fingerprint all archived artifacts
    • IRC Notification (optional)
    • Trigger parameterized build of other projects
      • Projects to build: postgresql-testsuite or dput
      • Predefined Parameters: binaries=$JOB_NAME

Creating Users

  • Create a new user Manage Jenkins->Manage Users
  • Set rights in Manage Jenkins->Configure Global Security
  • Add the user to /home/jenkins/web/htgroup
  • Add the user to /home/jenkins/web/htpasswd (sudo -u jenkins htpasswd htpasswd $username)

Users can change their password in the web interface, but they need to send in an updated $apr1$ hash for the htpasswd file as well (htpasswd -n $USER).

Retrieved from "http://wiki.postgresql.org/wiki/Apt/Jenkins"
Personal tools