Anna Shipman : JFDI

Creating a Puppet module

09 September 2013

I learnt three very interesting things recently, courtesy of the brilliant Dan Carley.

  1. git filter-branch
  2. how to create a puppet module using our puppet module skeleton
  3. shopt -s dotglob

While creating the Puppet manifests for the GOV.UK mirror, we realised we needed a module that was part of the GOV.UK Puppet, so our task was to pull it out into a standalone module and have both GOV.UK and our new GOV.UK mirror use it.

The first step in creating a new module is to pull it out into its own git repository.

First, clone the repository the module is currently in. (Extra learning here: you don't need to clone it from GitHub, you can just git clone the existing repository.) You want --no-hardlinks so it's entirely separate:

git clone --no-hardlinks puppet puppet-auditd

At this stage our puppet-auditd repo still points at our puppet repo, so we want to remove the remote:

git remote rm origin

Now we want to get rid of everything that's not our module and make our module the root, and this is where we use git filter-branch (update 2014-04-12: GitHub have removed this without a redirect – tsk! – but you can find a historical copy here):

git filter-branch --subdirectory-filter modules/audit-d HEAD

The next thing we want to do is create the files in the framework around the subdirectory to make it a Puppet module, for example the Modulefile. For this we can use our Puppet module skeleton. However, we don't want to directly follow the instructions there as we already have a lot of the classes we need. Instead of creating it from scratch as per those instructions, we generate our new module stucture:

puppet module generate gdsoperations-auditd

and then then we copy over the generated files.

However, a normal recursive copy won't work:

cp -r A/* B

This will only copy non-dotfiles, and we want the dotfiles as well, like .gitignore.

However, attempting to copy the dotfiles as well:

cp -r A/.* B

leads to hilarious results. .. is a directory too, so the above command copies all those contents recursively as well. Try it. It's most entertaining. But it's not really what we want.

What we can do is toggle what is included in A/* for this shell. The documentation is here, but this is what we need:

shopt -s dotglob

In order to check what is going to be copied now you can:

echo A/*

and if you like it:

cp -r A/* B

The next step is to go through the files to work out which modifications to keep and which ones are not relevant.

Another tip from Dan: vimdiff is really useful for this task. I won't go into how and what we decided; you can see that here.