Mar 29, 2016
Over the last year, GitLab Runner has become a significant part of the GitLab
family. We are happy to announce that GitLab Runner 1.1 is released today; a
release that brings major improvements over its predecessor. There is one
feature though that we are excited about and is the cornerstone of this release.
Without further ado, we present you GitLab Runner 1.1 and its brand-new, shiny
About GitLab Runner
GitLab has built-in continuous integration to allow you to run a
number of tasks as you prepare to deploy your software. Typical tasks
might be to build a software package or to run tests as specified in a
YAML file. These tasks need to run by something, and in GitLab this something
is called a Runner; an application that processes builds.
GitLab Runner 1.1 is the biggest release yet. Autoscaling provides the ability
to utilize resources in a more elastic and dynamic way. Along with autoscaling
come some other significant features as well. Among them is support for a
distributed cache server, and user requested features like passing artifacts
between stages and the ability to specify the archive names are now available.
Let’s explore these features one by one.
The Challenge of Scaling
Other continuous integration platforms have a similar functionality.
For example, Runners are called “Agents” in Atlassian’s Bamboo (which integrates
with Bitbucket.) Some services, like Bamboo, charge individually for using these
virtual machines and if you need a number of them it can get quite expensive,
quite quickly. If you have only one available Agent or Runner, you could be
slowing down your work.
We don’t charge anything for connecting Runners in GitLab, it’s all built-in.
However, the challenge up until now has been the scaling of these Runners. With
GitLab, Runners can be specified per project, but this means you have to create
a Runner per project, and that doesn’t scale well.
An alternative up until now was to create a number of shared Runners which
can be used across your entire GitLab instance.
However, what happens when you need more Runners than there are available?
You end up having to queue your tasks, and that will eventually slow things down.
Hence the need for autoscaling.
Autoscaling increases developer happiness
We decided to build autoscaling with the help of Docker Machine.
Docker Machine allows you to provision and manage multiple remote Docker hosts
and supports a vast number of virtualization and cloud providers,
and this is what autoscaling currently works only with.
Because the Runners will autoscale, your infrastructure contains only as
many build instances as necessary at anytime. If you configure the Runner to
only use autoscale, the system on which the Runner is installed acts as a
bastion for all the machines it creates.
Autoscaling allows you to increase developer happiness. Everyone hates to wait
for their builds to be picked up, just because all Runners are currently in use.
The autoscaling feature promotes heavy parallelization of your tests, something
that is made easy by defining multiple jobs in your
While cutting down the waiting time to a minimum makes your developers happy,
this is not the only benefit of autoscaling. In the long run, autoscaling
reduces your infrastructure costs:
- autoscaling follows your team’s work hours,
- you pay for what you used, even when scaling to hundreds of machines,
- you can use larger machines for the same cost, thus having your builds run
- the machines are self-managed, everything is handled by docker-machine, making
your Administrators happy too,
- your responsibility is to only manage GitLab and one GitLab Runner instance.
Below, you can see a real-life example of the Runner’s autoscale feature, tested
on GitLab.com for the GitLab Community Edition project:
Each machine on the chart is an independent cloud instance, running build jobs
inside Docker containers. Our builds are run on DigitalOcean 4GB machines, with
CoreOS and the latest Docker Engine installed.
DigitalOcean proved to be one of the best choices for us, mostly because of
the fast spin-up time (around 50 seconds) and their very fast SSDs, which are
ideal for testing purposes. Currently, our GitLab Runner processes up to 160
builds at a time.
If you are eager to test this yourself, read more on configuring the new
Distributed cache server
In GitLab Runner 0.7.0 we introduced support for caching. This release brings
improvements to this feature too, which is especially useful with autoscaling.
GitLab Runner 1.1 allows you to use an external server to store all your caches.
The server needs to expose an S3-compatible API, and while you can use for
example Amazon S3, there are also a number of other servers that you can run
on-premises just for the purpose of caching.
Read more about the distributed cache server and learn how to set
up and configure your own.
We listen closely to our community to extend GitLab Runner with user requests.
One of the often-requested features was related to passing artifacts between
Up until now, you had to use an external method if you wanted to pass the files
from one job to another. With GitLab Runner 1.1 this happens automatically.
Going one step further, GitLab 8.6 allows you to fine-tune what should be
passed. This is now handled by defining dependencies:
1 2 3 4 5 6 7 8
The second most-requested feature was the ability to change the name of the
uploaded artifacts archive. With GitLab Runner 1.1, this is now possible with
this simple syntax:
1 2 3
Read more about naming artifacts.
With GitLab Runner 1.1 we’ve also released improved documentation, explaining
all executors and commands. The documentation also describes what features are
supported by different configurations.
Read the revised documentation.
Using Runner on OSX
So far we described the biggest features, but these are not all the changes
introduced with GitLab Runner 1.1. We know that even the smallest change can
make a difference in your workflow, so here is the complete list:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
You can see why we think version 1.1 is fantastic.
Go get it, test it and share your feedback with us!
You can meet with the CI team in our upcoming webcast.
Live webcast: GitLab CI
Sign up for our webcast on April 14th, which includes an overview and tutorial
about using GitLab CI. Meet people from the GitLab CI team and get your questions
- Date: Thursday, April 14, 2016
- Time: 5pm (17:00) UTC; 12pm EST; 9am PST
- Register here
Can’t make it? Register anyway, and we’ll send you a link to watch it later!
Get our GitLab newsletter twice monthly.
Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/7KYg7WYvsac/