Skip to content

Puppet

Puppet works in a client-server architecture. This means that in a Puppet setup there is a one "master" server - node, from which all the agents go and fetch their configuration.

Installing Puppet#

More info here: https://www.puppet.com/docs/puppet/8/install_puppet#install_puppet

Because of the way Puppet works, the easiest way to configure a Puppet environment is to have them in the same network environment. This is why we are gonna need to configure a Puppet server inside the CSC environment. For this we need to make another dedicated server in CSC

Puppet requires quite a lot of processing power, so for CSC environment it is recommended to use standard.medium flavour instance running Ubuntu-20.04 or newer.

Once you have created the installation, setup it as you would the other servers, refer to this guide: xxxx

Installation steps:#

Puppet uses DNS hostnames to issue certificates. In CSC environment there is already a DNS server that issues names to the servers. E.g if you have a server named "Development01", other servers in the environment can ping the server with the address "Development01.openstacklocal". However it is a good practise to add the hostnames to the /etc/hosts file

On the Puppet master server:

#Insert the IP of the Puppet client server
127.0.0.1 localhost puppet
x.x.x.x Devserver01.local.domain puppetclient

And on the Puppet client:

# Insert the IP of the Puppet master server
x.x.x.x puppetmaster.local.domain puppetmaster puppet

After this you need to add the Puppet apt repository. Select the correct flavour from here:

https://apt.puppet.com/

In this case because we're running Ubuntu-20.04. we select the puppet8-release-focal.deb

You can check the OS version on Linux using command cat /etc/os-release (20.04 = Focal, 24.04.1 Noble, etc)

Download the repository and add the package to the package manager:

wget https://apt.puppet.com/puppet8-release-focal.deb
sudo dpkg -i puppet8-release-focal.deb

Update apt package list:

sudo apt update && upgrade # If you're doing a fresh install on a new server

After this install the Puppet:

apt install puppetserver

Start Puppet server service and enable it to start on boot:

sudo systemctl start puppetserver

Refresh bash CLI using bash -l to get access to the commands

Check the version usin the command puppetserver -v

You might also need to add the bin path to the directory. One way is to use the command sudo source /etc/profile.d/puppet-agent.sh or sudo export PATH=/opt/puppetlabs/bin:$PATH. Alternatively you can also add the bin path manually with sudo visudoand append :/opt/puppetlabs/bin to the end of the secure_path list.

Installing Puppet agent#

Let's install Puppet agent to the client server.

First you need to add the Puppet apt repository. Select the correct flavour from here:

https://apt.puppet.com/

In this case because we're running Ubuntu-20.04. we select the puppet8-release-focal.deb

You can check the OS version on Linux using command cat /etc/os-release (20.04 = Focal, 24.04.1 Noble)

Download the repository and add the package to the package manager:

wget https://apt.puppet.com/puppet8-release-focal.deb
sudo dpkg -i puppet8-release-<flavour>.deb

Install Puppet agent:

sudo apt install puppet-agent

Start the Puppet agent:

sudo /opt/puppetlabs/bin/puppet resource service puppet ensure=running enable=true

You might also need to add the bin path to the directory. One way is to use the command sudo source /etc/profile.d/puppet-agent.sh or sudo export PATH=/opt/puppetlabs/bin:$PATH. Alternatively you can also add the bin path manually with sudo visudoand append :/opt/puppetlabs/bin to the end of the secure_path list.

Refresh bash with bash -l

After this you need to add the puppet master server to the puppet config. Use the following command:

# The same DNS name that refers to the puppet master server in the /etc/hosts file.
puppet config set server puppetmaster.local.domain --section main

The puppet config command modifies the /etc/puppetlabs/puppet/puppet.conf file You can also add the commands manully by modifying the /etc/puppetlabs/puppet/puppet.conf. Just insert the following:

[main]
server = puppetmaster.local.domain

Establishing TLS connection between master and agent#

When the master server and the agent client is running, you can establish the SSL connection between the servers.

Go to the Puppet agent server, and insert the following command:

puppet ssl bootstrap

After this you might get an error that says "Couldn't fetch certificate from CA server; you might still need to sign this agent's certificate". This is expected as no auto signing of certificates has been enabled.

Go to the Puppet master server and you can see the requests by using the following command:

sudo puppetserver ca list

It should look something like this:

Requested Certificates:
devserver01.openstacklocal  (SHA256)  1A:7C:E..

You can accept this request with the command:

sudo puppetserver ca sign --certname devserver01.openstacklocal     # Or whatever name you can see in the ca list.

After this you can wait for the agent to request the certificate again, the default retry time is 120 seconds.

Sometimes the certficates may fail. You can attempt to establish the certficiates by stopping the service on the agent with systemctl stop puppet.service and then deleting files inside /etc/puppetlabs/ssl and .puppetlabs/etc/ssl

On the main server you can use the command sudo puppetserver ca clean --cername cert.name.here

You can see the certificates listed with sudo puppetserver ca list --all

Creating manifests#

You can create manifest files using puppet which dictate the environment. You can set a state in which you want the environment to be in. The default path to the manifests route is /etc/puppetlabs/code/environments/production/manifests/

Let's create a manifest file that installs docker into the client.

First you need to install a docker module to the puppet server: https://forge.puppet.com/modules/puppetlabs/docker/readme

sudo puppet module install puppetlabs-docker --version 10.0.1

Create a file "docker.pp" in the /etc/puppetlabs/code/environments/production/manifests/ folder.

class { 'docker':
    version => latest,
}

You can alternatively also just use

include 'docker'

This install the newest version of docker. You can also pin the version with additional commands.

Because the default time for agents to refresh the configuration is 30 minutes, we can iniate this manually on the Puppet client with the command sudo puppet agent -t.

This should iniate the process of applying the configuration to the server.

When the install is done you can check the docker installation with the command docker --version

docker --version
Docker version 27.3.1, build ce12230

Building the docker stack with Puppet#

The Puppet docker module also has ways to setup a docker stack using docker compose.

First, create a folder "files" to the path /etc/puppetlabs/code/environments/production/modules/docker/files. Puppet automatically expects to find a file in the "file" folder when referenced in the manifest file.

After this modify the docker.pp file in the production manifest folder:

user { 'ubuntu':
    groups  => ['docker','sudo','adm'],
}

class { 'docker':
    version => latest,
}

file { '/home/ubuntu/docker-compose.yml':   # File path on the remote server
    ensure => 'file',
    source => 'puppet:///modules/docker/docker-compose.yml',  # Local file, puppet:///modules/docker path uses the "files" folder inside the docker module.
}

file { '/home/ubuntu/Caddyfile':
    ensure => 'file',
    source => 'puppet:///modules/docker/Caddyfile',
}

docker_volume { 'caddy_data':
    ensure => present,
}

docker_compose { 'docker_up':
    compose_files => ['/home/ubuntu/docker-compose.yml'],
ensure        => present,
}

After this, test out the configuration by goin to the Puppet client server and type sudo puppet agent -t, it should setup the Docker compose stack and enable the service. just to compose the environment down, just change present to absent

Using GitLab Control Repo and r10k to manage Puppet#

Instead of just updating the puppet server manually, you can also modify it to use r10k to update configuration straight from a Gitlab repository.

First you need to follow the instructions listed in the official puppetlabs repository: https://github.com/puppetlabs/control-repo. Here's a snippet:

Gitlab:
Create a project called control-repo, and set the Namespace to be the puppet group.
Clone this control repository to your laptop/workstation:
git clone <repository url>
cd control-repo
Remove this repository as the origin remote:
git remote remove origin
Add your internal repository as the origin remote:
git remote add origin <url of your gitlab repository>
Push the production branch of the repository from your machine up to your git server
git push origin production

According to the instructions listed on https://www.puppet.com/docs/pe/2023.8/control_repo#managing_environments_with_a_control_repository (This is the Puppet Enterprise version, but the instructions can be used to setup r10k which doesn't need a licence.)

Installing r10k to the puppetserver:

Official r10k repository: https://github.com/puppetlabs/r10k

According to the r10k instructions on the gitlab repo, r10k should be installed using gem shipped with puppetserver.

/opt/puppet/bin/gem install r10k

Then we should create a ssh folder for puppetserver

mkdir /etc/puppetlabs/puppetserver/ssh
ssh-keygen -m PEM -t rsa -b 2048 -P '' -f /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa

Then let's create a direcotry for r10k: mkdir /etc/puppetlabs/r10k

And add a file to the folder called r10k.yaml

sources:
  control-repo:
    remote: 'git@gitlab.labranet.jamk.fi:xxx/r10k-test.git'
    basedir: '/etc/puppetlabs/code/environments'

After this you should add the deploy key you generated previously (The SSH .pub key) to gitlab. Go to your repository, then settings > Deploy keys > and add the ssh key from cat /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa.pub

After this you need to create a config folder inside the puppetserver/.ssh folder. nano /opt/puppetlabs/server/data/puppetserver/.ssh/config

host gitlab.labranet.jamk.fi
HostName gitlab.labranet.jamk.fi
IdentityFile /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa
User git

After this you should be able to pull the repository contents to the server with the command sudo -n -H -u puppet /opt/puppetlabs/puppet/bin/r10k deploy environment --verbose --puppetfile. The command is run with the puppet user, so you might need to tweak the folder access rights with chmod puppet:puppet -R

After this you can automate the deployment process with a gitlab runner register and install the runner to the puppet server. After this you can just create a bash file to the home folder of the gitlab-runner with sudo nano /home/gitlab-runner/puppet.sh:

sudo -n -H -u puppet bash -c "/opt/puppetlabs/puppet/bin/r10k deploy environment $1 --verbose --puppetfile"

And then give it executable rights:

chmod +x /home/gitlab-runner/puppet.sh

You might also need to tweak the sudo rights with gitlab-runner with sudo usermod -a -G sudo gitlab-runner and sudo visudo. Add the following to the end of the file:

gitlab-runner ALL=(ALL) NOPASSWD: ALL # or specify the /opt/bin path of r10k command

after this just insert this to the gitlab pipeline:

stages:
  - deploy

deploy-job:
stage: deploy
environment: production
script:
    - /home/gitlab-runner/puppet.sh

If you already have a puppet agent running you need to also specify which environment it listens to. The default is production, but it can be changed by modifying the puppet.conf file in /etc/puppetlabs/puppet/puppet.conf (if you don't have the file, you can create it)

[main]
environment = main # Or anything you have your environment set as

After this, you can just try to push a new code change to the environment.

Skipping Git submodules setup
Executing "step_script" stage of the job script
00:03
$ /home/gitlab-runner/puppet.sh
INFO     -> Deploying environment /etc/puppetlabs/code/environments/main
WARN     -> Overwriting local modifications to /etc/puppetlabs/code/environments/main
INFO     -> Environment main is now at 41a4212606824bd8629552aff9eed74f5b94c3a6
INFO     -> Using Puppetfile '/etc/puppetlabs/code/environments/main/Puppetfile'
INFO     -> Deploying module to /etc/puppetlabs/code/environments/main/modules/inifile
INFO     -> Deploying module to /etc/puppetlabs/code/environments/main/modules/stdlib
INFO     -> Deploying module to /etc/puppetlabs/code/environments/main/modules/concat
INFO     -> Deploying module to /etc/puppetlabs/code/environments/main/modules/apt
INFO     -> Deploying module to /etc/puppetlabs/code/environments/main/modules/reboot
INFO     -> Deploying module to /etc/puppetlabs/code/environments/main/modules/docker
INFO     -> Removing unmanaged path /etc/puppetlabs/code/environments/production
Cleaning up project directory and file based variables
00:00
Job succeeded

In the control repo you might need to also check dependencies and modules inside the Puppetfile. For example the docker module requires these dependencies:

mod 'puppetlabs-inifile', '6.1.1'
mod 'puppetlabs-stdlib', '9.6.0'
mod 'puppetlabs-concat', '9.0.2'
mod 'puppetlabs-apt', '9.4.0'
mod 'puppetlabs-reboot', '5.0.0'
mod 'puppetlabs-docker', '10.0.1'