Jump to content

Continuous integration/Tutorials/Adding basic checks

From mediawiki.org

This tutorial shows up how to add a very basic check in Jenkins, we will base this tutorial on operations/dns.git which hold Wikimedia DNS configuration. The aim is to validate the syntax of the DNS zone files.

What we want to achieve:

Whenever a patch is send against operations/dns.git we want to expand the gdnsd templates and have it validate the syntax. When no error is detected, Jenkins should vote Verified +2 to let people merge the change after review, if something is wrong, we would vote Verified -1 to let people know the syntax is invalid.

Prerequisites

[edit]

Editing Jenkins configuration

[edit]

The miscellaneous operations/ repositories are configured inside operations-misc.yaml. We will need to create a new Jenkins project that gives the Gerrit repository name and list the jobs we want to attach to it. 

- project:
    # Parameters that will be passed to job templates
    name: 'operations-dns'

Now lets create our job template. It needs: - a name so we can reference it in the project definition - a few default parameters which let the job be triggered by Zuul - we then add the shell commands we want to execute: 

- job-template:

    # The name of our template. Will be referenced to in the project:
    name: 'operations-dns-lint'

    # Default settings to let Zuul trigger the job against the submitted patchset:
    defaults: use-zuul
    triggers:
     - zuul

    # The shell commands. $WORKSPACE is an environnement variable that contains
    # the full path to where Jenkins cloned the repository with the submitted patch
    builders: 
     - shell: |
         # Prepare the zone files
         mkdir -p "$WORKSPACE"/chroot/zones
         authdns-gen-zones "$WORKSPACE"/templates "$WORKSPACE"/chroot/zones
        
         # lint them
         gdnsd -d "$WORKSPACE"

Now we want to edit the project definition to reference this template. 

- project:
    name: 'operations-dns'
    jobs:  # Jobs we want to generate using the parameters above
     - 'operations-dns-lint'  # job referenced there

Now have Jenkins Job Builder to generate the configuration using jenkins-jobs test -o - config/ operations-dns-lint. It will parse the YAML definition, realize the operations-dns-lint job and write out on standard output the XML Jenkins job definition. If something went wrong, you want to double check your YAML definition, usually the errors are related to a wrong indentation (use two spaces for indentation).

The command is: 

$ jenkins-jobs --conf jenkins_jobs.ini test -o - config/ operations-dns-lint
// snip XML output

The most important things to check are: - the git repository name - the builders section (the actual commands that would be run).

The git configuration is 

  <scm class="hudson.plugins.git.GitSCM">
    <configVersion>2</configVersion>
    <userRemoteConfigs>
      <hudson.plugins.git.UserRemoteConfig>
        <name>origin</name>
        <refspec>$ZUUL_REF</refspec>
        <url>/srv/ssd/zuul/git/$ZUUL_PROJECT</url>
      </hudson.plugins.git.UserRemoteConfig>
    </userRemoteConfigs>
    <branches>
      <hudson.plugins.git.BranchSpec>
        <name>$ZUUL_COMMIT</name>
      </hudson.plugins.git.BranchSpec>
    </branches>
    <!-- snip various git settings -->
  </scm>

The repository URL is /srv/ssd/zuul/git/$ZUUL_PROJECT which trailling part is the gerrit project. The Gerrit project name $ZUUL_PROJECT, the refspec $ZUUL_REF and branch specifier $ZUUL_COMMIT are provided by Zuul and let the Jenkins git plugin to fetch the patchset.

Then verify the shell commands you want to execute under the builders section. Note that some characters needs to be escaped (such as double quotes) since it is an XML file: 

  <builders>
    <hudson.tasks.Shell>
      <command># Prepare the zone files
mkdir -p &quot;$WORKSPACE&quot;/chroot/zones
authdns-gen-zones &quot;$WORKSPACE&quot;/templates &quot;$WORKSPACE&quot;/chroot/zones

# lint them
gdnsd -d &quot;$WORKSPACE&quot;</command>
    </hudson.tasks.Shell>
  </builders>

Once happy, you can deploy the configuration on Jenkins: 

 $ jenkins-jobs --conf jenkins_jobs.ini update config/ operations-dns-lint

If all went well, the job should be available at https://integration.wikimedia.org/ci/job/operations-dns-lint/ :

Screenshot of https://integration.wikimedia.org/ci/job/operations-dns-lint/ after the job has been generated by Jenkins job builder.

When clicking the configure link, you can review the final configuration. Browsing down at the bottom of that page you will see the actual shell commands that would be run:

Screenshot of https://integration.wikimedia.org/ci/job/operations-dns-lint/configure focusing on the 'Execute shell' Jenkins build command.

Editing Zuul configuration

[edit]

The configuration is held in integration/zuul-config.git repository which has a single file: layout.yaml.

Under the projects section add gerrit repository name and trigger the operations-dns-lint in the check-voter pipeline. This pipeline will make Zuul vote Verified +2 on success and Verified -1 on failure: 

projects:
  - name: operations/dns
    check-voter:
      - operations-dns-lint

You will need the change to be deployed on Zuul server and have Zuul reloaded. It will then clone the repository and wait for changes received from Gerrit. From there, it would trigger the Jenkins job with your patchset and report back in Gerrit the status of the zone linting.

References

[edit]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Continuous_integration/Tutorials/Adding_basic_checks&oldid=3833908"