Varnish Website’s IT Infrastructure

One of the major reasons for the website upgrade the Varnish Project
has been going through this month, was in an effort to eat more of
our own dogfood.

The principle of eating your own dogfood is important for software
quality, that is how you experience what your users are dealing with
and I am not the least ashamed to admit that several obvious improvements
have already appeared on my TODO list as a result of this transition.

But it is also important to externalize what you learn doing so, and
therefore I thought I would document here how the projects new “internal
IT” works.


Who cares?

Yes, we use some kind of hardware, but to be honest I don’t know what
it is.

Our primary site runs on a RootBSD ‘Omega’
virtual server somewhere near CDG/Paris.

And as backup/integration/testing server we can use any server,
virtual or physical, as long as it has a internet connection and
contemporary performance, because the entire install is scripted
and under version control (more below)

Operating System

So, dogfood: Obviously FreeBSD.

Apart from the obvious reason that I wrote a lot of FreeBSD and
can get world-class support by bugging my buddies about it, there
are two equally serious reasons for the Varnish Project to run on
FreeBSD: Dogfood and jails.

Varnish Cache is not “software for Linux”, it is software for any
competent UNIX-like operating system, and FreeBSD is our primary
“keep us honest about this” platform.


You have probably heard about Docker and Containers, but FreeBSD
have had jails
since I wrote them in 1998
and they’re a wonderful way to keep your server installation

We currently have three jails:

Script & Version Control All The Things

We have a git repos with shell scripts which create these jails
from scratch and also a script to configure the host machine

That means that the procedure to install a clone of the server
is, unabridged:

# Install FreeBSD 10.3 (if not already done by hosting)
# Configure networking (if not already done by hosting)
# Set the clock
service ntpdate forcestart
# Get git
env ASSUME_ALWAYS_YES=yes pkg install git
# Clone the private git repo
git clone ssh://
# Edit the machines IP numbers in /etc/pf.conf
# Configure the host
sh |& tee
# Build the jails
foreach i (Tools Hitch Varnish)
        (cd $i ; sh build* |& tee

From bare hardware to ready system in 15-30 minutes.

It goes without saying that this git repos contains stuff
like ssh host keys, so it should not go on github.


Right now there is nothing we need to backup.

When I move the mailserver/mailman/mailing lists over, those will
need to be backed up, but here the trick is to only backup the
minimal set of files, and in a “exchange” format, so that future
migrations and upgrades can slurp them in right away.

The Homepage

The new homepage is built with Sphinx
and lives in its own
github project (Pull requests
are very welcome!)

We have taken snapshots of some of the old webproperties, Trac, the
Forum etc as static HTML copies.

Why on Earth…

It is a little bit tedious to get a setup like this going, whenever
you tweak some config file, you need to remember to pull the change
back out and put it in your Admin repos.

But that extra effort pays of so many times later.

You never have to wonder “who made that change and why” or even try
to remember what changes were needed in the first place.

For us as a project, it means, that all our sysadmin people
can build a clone of our infrastructure, if they have a copy of
our “Admin” git repos and access to github.

And when FreeBSD 11
comes out, or a new version of sphinx or something else, mucking
about with things until they work can be done at leisure without
guess work.

For instance I just added the forum snapshot, by working out all
the kinks on one of my test-machines.

Once it was as I wanted it, I pushed the changes the live machine and then:

varnishadm vcl.use backup
# The 'backup' VCL does a "pass" of all trafic to my server
cd Admin
git pull
cd Tools
sh |& tee
varnishadm vcl.load foobar varnish-live.vcl
varnishadm vcl.use foobar

For a few minutes our website was a bit slower (because of the
extra Paris-Denmark hop), but there was never any interruption.

And by doing it this way, I know it will work next time also.

2016-04-25 /phk

All that buzz about “reproducible builds” ? Yeah, not a new idea.

Original URL:

Original article

Comments are closed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: