Continuous Integration with Jenkins

Jenkins is an open source tool which helps in Continuous integration. Continuous integration is a process in which all development work is integrated at a predefined time or at a regular defined interval or through a trigger and the resulting work is automatically tested and built. The basic functionality of Jenkins is to poll a version control tool, if any changes are there, build the code using a build tool like  Apache Ant or Maven. Jenkins monitors the build process reports success or errors to the administrator and /or the code comitters.

The functionality of Jenkins can be extended by using plug-ins for specific tasks, for example report generation.

Various tools that help in implementing the CI with Jenkins are:

Code Repositories :  SVN, Mercurial, Git

Test Frameworks : JUnit,Cucumber, CppUnit

Artifact Repositories : Nexus, Artifactory

Using Jenkins:

Download the jenkins.war file from Here. And copy to Webapps folder of tomcat. Start Tomcat, and Jenkins installation will be available under http://localhost:8080/jenkins

We need to have accessible code repository in a version control tool like SVN/GIT/ClearCase. A plugin is used to connect to the code repository. A working set of build scripts in Ant/Maven. Artifact repository like Nexus(if using Maven) and application deployment destination (WebSphere). Build jobs are set up in jenkins to cater to a integration branch(ideal scenario). In the build job, the poll time is defined (say 10 mins). 

Jenkins also supports master-slave setup. In which, there will be a master and multiple slaves to distribute the build job load. Slave set up has to be done on the systems requires. Cross OS set ups are also supported. For example: Master being on Linux, slaves can be set up on windows systems. Jobs can be linked to built by a specific jenkins slave. A “master” is an installation of Jenkins. When you weren’t using the master/slave support, a master was all you had. Even in the master/slave mode, the role of a master remains the same. It will serve all HTTP requests, and it can still build projects on its own.Slaves are computers that are set up to build projects for a master. Jenkins runs a separate program called “slave agent” on slaves. In other words, there is no need to install the full Jenkins (package or compiled binaries) on a slave node. 

When slaves are registered to a master, a master starts distributing loads to slaves. The exact delegation behavior depends on configuration of each project. Some projects may choose to “stick” to a particular machine for a build, while others may choose to roam freely between slaves. For people accessing Jenkins website, things works mostly transparently. You can still browse javadoc, see test results, download build results from a master, without ever noticing that builds were done by slaves.  In other words, the master becomes a sort of “portal” to the entire build farm.

Jenkins has a built-in SSH client implementation that it can use to talk to remote sshd and start a slave agent. This is the most convenient and preferred method for Unix slaves, which normally has sshd out-of-the-box. Click Manage Jenkins, then Manage Nodes, then click “New Node.” In this set up, you’ll supply the connection information (the slave host name, user name, and ssh credential). Note that the slave will need the master’s public ssh key copied to ~/.ssh/authorized_keys. Jenkins will do the rest of the work by itself, including copying the binary needed for a slave agent, and starting/stopping slaves. If your project has external dependencies (like a special ~/.m2/settings.xml, or a special version of java), you’ll need to set that up yourself, though.

This is the most convenient set up on Unix. However, if your are on Windows and you don’t have ssh commands with cygwin for example, you can use a tool like PuTTY and PuTTYgen to generate your private and public pair of keys.For Windows slaves, Jenkins can use the remote management facility built into Windows 2000 or later. In this set up, you’ll supply the username and the password of the user who has the administrative access to the system, and Jenkins will use that remotely create a Windows service and remotely start/stop them.This is the most convenient set up on Windows, but does not allow you to run programs that require display interaction.

Another way of doing this is to start a slave agent through Java Web Start (JNLP). In this approach, you’ll interactively logon to the slave node, open a browser, and open the slave page. You’ll be then presented with the JNLP launch icon. Upon clicking it, Java Web Start will kick in, and it launches a slave agent on the computer where the browser was running.This mode is convenient when the master cannot initiate a connection to slaves, such as when it runs outside a firewall while the rest of the slaves are in the firewall.

If you need display interaction (e.g. for GUI tests) on Windows and you have a dedicated (virtual) test machine, this is a suitable option. Create a jenkins user account, enable auto-login, and put a shortcut to the JNLP file in the Startup items (after having trusted the slave agent’s certificate). This allows one to run tests as a restricted user as well.

This launch mode uses a mechanism very similar to Java Web Start, except that it runs without using GUI, making it convenient for an execution as a daemon on Unix. To do this, configure this slave to be a JNLP slave, take slave.jar as discussed above, and then from the slave, run a command like this:

$ java -jar slave.jar -jnlpUrl http://yourserver:port/computer/slave-name/slave-agent.jnlp

Some slaves are faster, while others are slow. Some slaves are closer (network wise) to a master, others are far away. So doing a good build distribution is a challenge. Currently, Jenkins employs the following strategy:

1. If a project is configured to stick to one computer, that’s always good.
2. Jenkins tries to build a project on the same computer that it was previously built.
3. Jenkins tries to move long builds to slaves, because the amount of network interaction between a master and a slave tends to be logarithmic to the duration of a build (IOW, even if project A takes twice as long to build as project B, it won’t require double network transfer.) So this strategy reduces the network overhead.

Node Monitoring:

Jenkins has a notion of a “node monitor” which can check the status of a slave for various conditions, displaying the results and optionally marking the slave offline accordingly. Jenkins bundles several, checking disk space in the workspace; disk space in the temporary partition; swap space; clock skew (compared to the master); and response time.The builds can be triggered in many ways. As we have seen, due to SCM polling, url triggers, Fixed interval trigger etc.

Access to jenkins can also be made secure by using jenkins own database – by setting up user permissions by creating logins and using matrix security or by integrating with enterprise LDAP. Alternatively security can be delegated to the servlet container or use unix user/group database.It is recommended to secure Jenkins. Manage Jenkins and then Configure Global Security. Select the Enable securityflag. The easiest way is to use Jenkins own user database. Create at least the user “Anonymous” with read access. Also create entries for the users you want to add in the next step.

Quiet Period:

Commits often come in a burst. This seems to happen mainly for two reasons — people sometimes forget to commit some files, and in the tranquility of waiting for your SCM to finish a commit, people sometimes realize the problems in the commit and they quickly make follow-up changes. The conventional wisdom is that the CI server should wait for the burst to finish before attempting a build. This is said to reduce the chance of having broken build, and it is also sometimes useful in reducing the average turn-around time for builds that take longer.

As such, Hudson is capable of waiting for a commit burst to be over before it triggers a new build, and this feature is called “quiet period.” There are two parts in Hudson that interacts with the quiet period. One is the SCM polling behavior and the other is the queue.

The queue portion of the quiet period is straight-forward. When a build is scheduled into the queue with quiet period, the build will sit in the queue until the quiet period expires. If during this period, additional attempts are made to put the same build in the queue, the quiet period resets to its initial value. For example, if the quiet period is 5 minutes, and the build is put into the queue 9:00am and 9:03am, the actual build will only happen after 9:08am.

Finger Print Tracking:

When you have interdependent projects on Jenkins, it often becomes hard to keep track of which version of this is used by which version of that. Jenkins supports “file fingerprinting” to simplify this.

For example, suppose you have the TOP project that depends on the MIDDLE project, which in turn depends on the BOTTOM project. You are working on the BOTTOM project. The TOP team reported that bottom.jar that they are using causes an NPE, which you thought you fixed in BOTTOM #32. Jenkins can tell you which MIDDLE builds and TOP builds are using (or not using) your bottom.jar #32. The fingerprint of a file is simply a MD5 checksum. Jenkins maintains a database of md5sum, and for each md5sum, Jenkins records which builds of which projects used. This database is updated every time a build runs and files are fingerprinted.

To avoid the excessive disk usage, Jenkins does not store the actual file. Instead, it just stores md5sum and their usages. These files can be seen in $JENKINS_HOME/fingerprints.

Splitting Jobs:

A build is normally a fairly sequential process, and in a big project, a full execution can easily take hours. While one could bring such a job on Jenkins, a long turn-around time to the result tends to reduce the value of continuous integration. This page discusses a technique to cope with this problem.

The idea to is to split a big build into multiple stages. Each stage is executed sequentially for a particular build run, but this works like a CPU pipeline and increase the throughput of CI, and also reduces the turn-around time by reducing the time a build sits in the build queue.

In this situation, your earlier stage needs to pass files to later stages. A general way to do this is as follows:

1. An earlier stage archives all the files into a zip/tgz file at the end of the build.
2. Tell Jenkins to archive this zip/tgz file as a post-build action, take a fingerprint of it, then trigger the next stage.
3. The first thing the next stage does in its build is to obtain this bundle through the permalink for the last successful artifact, then unzip it. Do keep this archive file around because we’ll take a fingerprint of it here, too.
4. The build proceeds by using the files obtained from the earlier stage.
5. Tell Jenkins to fingerprint the zip/tgz file. This allows you to correlate executions of these stages to track the flow.

If you have more than 2 stages, you can repeat this process. In some cases, this “zip/tgz” file would have to contain the entire workspace.

Jenkins Backup:

Jenkins stores all the settings, logs and build artifacts in its home directory, for example, in /var/lib/jenkins under the default install location of Ubuntu.To create a backup of your Jenkins setup, just copy this directory.

The jobs directory contains the individual jobs configured in the Jenkins install. You can move a job from one Jenkins installation to another by copying the corresponding job directory. You can also copy a job directory to clone a job or rename the directory.

Click reload config button in the Jenkins web user interface to force Jenkins to reload configuration from the disk.

As we have understood by now, Jenkins is the most widely used CI tool and is an industry standard. This is supported by the widely available plugins which ease many tasks. Setting up and configuring jenkins is a breeze. Being in development for quite sometime, it is very stable and has a great online community support with some companies providing paid support as well.

One interesting feature is the multi-configuration project. A multi-configuration project is useful for instances where your builds will make many similar build steps, and you would otherwise be duplicating steps. The configuration matrix allows you to specify what steps to duplicate, and create a multiple-axis graph of the type of builds to create.