Apt/Jenkins

From PostgreSQL wiki

Revision as of 08:30, 13 October 2012

Overview

Packages are built in a [Jenkins] installation at https://pgdgbuild.dus.dg-i.net/jenkins/ (login required).

We are using Mika Prokop's [Jenkins-Debian-Glue] scripts.

For every package, there are two jobs:

• foobar-source builds .dsc source packages from an VCS repository
• foobar-binaries builds .deb binary packages from the source packages

These jobs then trigger common jobs:

• postgresql-testsuite runs the postgresql-common testsuite on the server packages built
• dput uploads the source and binary packages to the repository host

The postgresql-testsuite job triggers dput. Non-server packages trigger the dput job directly.

Creating New Jobs

New jobs should use the templating mechanism provided by JoelJ's [Templating plugin].

• Go to "Scaffolding"
• Select the scaffold for the VCS the package is using, e.g. "git-package"
• "Stand up" the scaffold, and fill in the variables
  • giturl: Git URL for the package
  • package: package name
• Go to the new source job (called like git-source-templatepackage), select configure, and rename the job to package-source
• Go to the new binaries job (called like postgresql-binaries-templatepackage), select configure, and rename the job to package-binaries
• Configure the package-binaries jobs to trigger the dput project (FIXME: this should be part of the template)
• Hit "build now" on the new source job
• Watch the job's console output

Job Configuration

All "real" source/binaries jobs should be seeded from template jobs. The template jobs should then be configured as documented here.

Variables with one $ sign are jenkins variables (e.g. $JOB_NAME). Variables with two $$ signs are template variables (e.g. $$package).

Source Job Configuration

• Jobs need to be "matrix" (multi-configuration) style jobs.
• Project name: For the debian-jenkins-glue/pgdg scripts to work properly, source jobs must be named package-source. (There might be an extra optional suffix at the end.)
• Discard Old Builds (optional): Max # of builds to keep: 3
• This build is parameterized (optional):
  • String Parameter
  • Name: revision
  • Default Value: (blank)
  • Description: Number to use in the .pgdg+X package version suffix. Defaults to the .pgdg+X number found in the current changelog, or 1 otherwise.
• Source Code Management, one of
  • Git
    • Repository URL: e.g. git://anonscm.debian.org/pkg-postgresql/pgpool2.git
      Template job: $$giturl
    • Advanced (the second "Advanced" button just above "Repository Browser"
      • "Local subdirectory for repo (optional)" (not optional): source
  • Subversion
    • ...
  • Bazaar (Needs Bazaar Plugin)
    • Repository URL: $$bzrurl
• Configuration Matrix
  • User-defined Axis
    • Name: distribution
    • Values: sid wheezy squeeze
  • Run each configuration sequentially (recommended)
  • Execute touchstone builds first (highly recommended)
    • Filter: distribution=="sid"
• Build Environment, one of
  • Implement Template
    • Template Name: git-source-template
    • Template Variables:
      giturl=git://anonscm.debian.org/pkg-postgresql/pgpool2.git
      package=pgpool2
      Note: this is the only place that should be edited for "Implement Template" jobs. All other settings will be overwritten on the next "Sync".
  • Make this a Template
• Build
  • Execute Shell
    • Command: generate-pgdg-source
• Post-build Actions
  • Archive the artifacts
    • result/*
  • Record fingerprints of files to track usage (optional)
    • Fingerprint all archived artifacts
  • IRC Notification (optional)
    • Advanced (Channels are usually configured globally, so empty here)
      • Notify for matrix projects: ONLY PARENT (recommended)
  • Trigger parameterized build of other projects
    • Projects to build: package-binaries
      Template jobs: $$package-binaries
    • Trigger build without parameters
      Note: a plain trigger would do the same, but does not work well with the Templating plugin.

Binaries Job Configuration

• Jobs need to be "matrix" (multi-configuration) style jobs.
• Project name: For the debian-jenkins-glue/pgdg scripts to work properly, source jobs must be named package-binaries. (There might be an extra optional suffix at the end.)
• Discard Old Builds (optional): Max # of builds to keep: 3
• Configuration Matrix (axis ordering does not matter, but should be consistent)
  • User-defined Axis
    • Name: architecture
    • Values: amd64 i386
  • User-defined Axis
    • Name: distribution
    • Values: sid wheezy squeeze
  • Run each configuration sequentially (recommended)
  • Execute touchstone builds first (highly recommended)
    • Filter: (distribution=="sid") && (architecture=="amd64")
• Build Environment
  • Delete workspace before build starts (needs Jenkins Workspace Cleanup Plugin)
  • one of
  • Implement Template
    • Template Name: postgresql-binarlies-template
    • Template Variables:
      giturl=git://anonscm.debian.org/pkg-postgresql/pgpool2.git (the URL is not used in binaries jobs, but the variables should be the same for consistency)
      package=pgpool2
      Note: this is the only place that should be edited for "Implement Template" jobs. All other settings will be overwritten on the next "Sync".
  • Make this a Template
• Build
  • Copy artifacts from another project
    • Project Name: pgpool2-source/distribution=$distribution
      Template job: $$package-source/distribution=$distribution
    • Which build: Upstream build that triggered this job
      • Use "Last successful build" as fallback
    • Artifacts to copy: result/*
    • Flatten directories
  • Execute Shell
    • Command: build-and-provide-package
• Post-build Actions
  • Archive the artifacts
    • *.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes
  • Record fingerprints of files to track usage (optional)
    • Fingerprint all archived artifacts
  • IRC Notification (optional)
  • Trigger parameterized build of other projects
    • Projects to build: postgresql-testsuite or dput
    • Predefined Parameters: binaries=$JOB_NAME