Apt/Jenkins

From PostgreSQL wiki

< Apt(Difference between revisions)
Jump to: navigation, search
(Update our git url)
(Add Groovy Postbuild dependency)
Line 141: Line 141:
 
* Git Plugin
 
* Git Plugin
 
* Copy Artifact Plugin
 
* Copy Artifact Plugin
 +
* Groovy Postbuild Plugin
 
* Parameterized Trigger Plugin
 
* Parameterized Trigger Plugin
 
* Green Balls Plugin
 
* Green Balls Plugin
 
* SCM Sync Plugin
 
* SCM Sync Plugin
 +
 +
The Jenkins config is autogenerated from a YAML description in our git repo using Jenkins Job Builder.

Revision as of 19:59, 7 April 2014

Contents

Overview

Packages are built in a [Jenkins] installation at https://pgdgbuild.dus.dg-i.net/jenkins/ (login required).

We are using a slightly patched variant of Mika Prokop's [Jenkins-Debian-Glue] scripts. Our version is at http://git.postgresql.org/gitweb/?p=pgapt.git;a=summary.

For every package, there are two jobs:

  • foobar-source builds .dsc source packages from an VCS repository, which triggers
  • foobar-binaries builds .deb binary packages from the source packages

These jobs then trigger a common job:

  • dput uploads the source and binary packages to the repository host

Creating New Jobs

New jobs should use the templating mechanism provided by JoelJ's [Templating plugin].

  • Go to "Scaffolding"
  • Select the scaffold for the VCS the package is using, e.g. "git-package"
  • "Stand up" the scaffold, and fill in the variables
    • Append to Job Name: package name
    • giturl: Git URL for the package
    • package: package name
  • Go to the new source job (called like git-source-templatepackage), select configure, and rename the job to package-source
  • Go to the new binaries job (called like postgresql-binaries-templatepackage), select configure, and rename the job to package-binaries
  • Hit "build now" on the new source job
  • Watch the job's console output

Job Configuration

All "real" source/binaries jobs should be seeded from template jobs. The template jobs should then be configured as documented here.

Variables with one $ sign are jenkins variables (e.g. $JOB_NAME). Variables with two $$ signs are template variables (e.g. $$package).

Source Job Configuration

  • Jobs need to be "matrix" (multi-configuration) style jobs.
  • Project name: For the debian-jenkins-glue/pgdg scripts to work properly, source jobs must be named package-source. (There might be an extra optional suffix at the end.)
  • Discard Old Builds (optional): Max # of builds to keep: 3
  • This build is parameterized (optional):
    • String Parameter
    • Name: revision
    • Default Value: (blank)
    • Description: Number to use in the .pgdg+X package version suffix. Defaults to the .pgdg+X number found in the current changelog, or 1 otherwise.
  • Source Code Management, one of
    • Git
    • Subversion
      • ...
    • Bazaar (Needs Bazaar Plugin)
      • Repository URL: $$bzrurl
  • Configuration Matrix
    • User-defined Axis
      • Name: distribution
      • Values: sid wheezy squeeze
    • Run each configuration sequentially (recommended)
    • Execute touchstone builds first (highly recommended)
      • Filter: distribution=="sid"
  • Build Environment, one of
    • Implement Template
      • Template Name: git-source-template
      • Template Variables:
        giturl=git://anonscm.debian.org/pkg-postgresql/pgpool2.git
        package=pgpool2
        Note: this is the only place that should be edited for "Implement Template" jobs. All other settings will be overwritten on the next "Sync".
    • Make this a Template
  • Build
    • Execute Shell
      • Command: generate-pgdg-source
  • Post-build Actions
    • Archive the artifacts
      • result/*
    • Record fingerprints of files to track usage (optional)
      • Fingerprint all archived artifacts
    • IRC Notification (optional)
      • Advanced (Channels are usually configured globally, so empty here)
        • Notify for matrix projects: ONLY PARENT (recommended)
    • Trigger parameterized build of other projects
      • Projects to build: package-binaries
        Template jobs: $$package-binaries
      • Trigger build without parameters
        Note: a plain trigger would do the same, but does not work well with the Templating plugin.

Binaries Job Configuration

  • Jobs need to be "matrix" (multi-configuration) style jobs.
  • Project name: For the debian-jenkins-glue/pgdg scripts to work properly, source jobs must be named package-binaries. (There might be an extra optional suffix at the end.)
  • Discard Old Builds (optional): Max # of builds to keep: 3
  • Configuration Matrix (axis ordering does not matter, but should be consistent)
    • User-defined Axis
      • Name: architecture
      • Values: amd64 i386
    • User-defined Axis
      • Name: distribution
      • Values: sid wheezy squeeze
    • Run each configuration sequentially (recommended)
    • Execute touchstone builds first (highly recommended)
      • Filter: (distribution=="sid") && (architecture=="amd64")
  • Build Environment
    • Delete workspace before build starts (needs Jenkins Workspace Cleanup Plugin)
    • one of
    • Implement Template
      • Template Name: postgresql-binarlies-template
      • Template Variables:
        giturl=git://anonscm.debian.org/pkg-postgresql/pgpool2.git (the URL is not used in binaries jobs, but the variables should be the same for consistency)
        package=pgpool2
        Note: this is the only place that should be edited for "Implement Template" jobs. All other settings will be overwritten on the next "Sync".
    • Make this a Template
  • Build
    • Copy artifacts from another project
      • Project Name: pgpool2-source/distribution=$distribution
        Template job: $$package-source/distribution=$distribution
      • Which build: Upstream build that triggered this job
        • Use "Last successful build" as fallback
      • Artifacts to copy: result/*
      • Flatten directories
    • Execute Shell
      • Command: build-and-provide-package
  • Post-build Actions
    • Archive the artifacts
      • *.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes
    • Record fingerprints of files to track usage (optional)
      • Fingerprint all archived artifacts
    • IRC Notification (optional)
    • Trigger parameterized build of other projects
      • Projects to build: postgresql-testsuite or dput
      • Predefined Parameters: binaries=$JOB_NAME

Creating Users

  • Create a new user Manage Jenkins->Manage Users
  • Set rights in Manage Jenkins->Configure Global Security
  • Add the user to /home/jenkins/web/htgroup
  • Add the user to /home/jenkins/web/htpasswd (sudo -u jenkins htpasswd htpasswd $username)

Users can change their password in the web interface, but they need to send in an updated $apr1$ hash for the htpasswd file as well (htpasswd -n $USER).

Jenkins Plugins

We are using these Jenkins Plugins:

  • Timestamper Plugin
  • IRC Plugin
  • Bazaar Plugin
  • Git Plugin
  • Copy Artifact Plugin
  • Groovy Postbuild Plugin
  • Parameterized Trigger Plugin
  • Green Balls Plugin
  • SCM Sync Plugin

The Jenkins config is autogenerated from a YAML description in our git repo using Jenkins Job Builder.

Personal tools