Skip to main content

The Matt-Cloud Way

Introduction

This is effectively a copy of the Jenkins and Ansible article under The Nerdy Stuff. This provides an overview of how I have things set up in Matt-Cloud before getting into the specifics.

Jenkins Process

On my own Jenkins instance I use Github to sync my Pipelines with jenkinsfiles on my server, along with LDAP and OIDC for authentication which is way more complicated than a default Jenkins setup. You might have access; you can check your group membership at the Matt-Cloud Group page. Along with that, I use an open-source project to host all my pipelines and playbooks in a website, which itself is hosted behind my SSO. This is not something anyone but me has access to due to how insecure the access is. It is full write access to all files without any knowledge of the connected user. It works by effectively hosting files that live on a Linux system in the browser. I have that running in a docker container, with the paths of the ansible playbooks and jenkins pipelines in it so they can all be viewed and edited in the browser. I also have my code-server container customized so I can sync my github from the built-in terminal in the browser. Github lets me easily sync the Jenkinsfiles between the filesystem and Jenkins itself. Jenkins itself is configured in the browser, so the Docker settings for Jenkins are only a dozen lines or so of mostly volumes. Here are some screenshots of my stuff

Ansible/Jenkins Code Server

image.png

Jenkins Home Page

image.png

Sample Pipeline Config

image.png

Some OIDC Config and Security Settings

image.png

image.png

Let me start with my own Jenkins. I have recently shared my jenkinsfiles on my Gitea instance. To show how this works I will use the update-endpoint pipeline I have as an example. Most of my pipelines have the same stages of Generate Inventory FileAnsible Playbook, and Remove Inventory File. Once you have Jenkins up and running, and have stored your needed keys, you should be ready to run Ansible. The biggest thing is to make sure the SSH key is accessible. I will include the code for a dynamic inventory file generator here. This makes it obvious how Ansible authenticates. Since the inventory file is intended to be an inventory, it can be used to run the same playbook on multiple hosts.

Inventory File Generation
inventory.sh
#!/bin/bash

# Dynamic inventory generation script ansible

# Function to display usage
usage() {
    echo "Usage: $0 -i IP_LIST -u JENKINS_USER -g JENKINS_GROUP [-a SERVER_SUBNET_GROUP] [-s] [-v] [-e]"
    echo "Options:"
    echo "  -i IP_LIST                  Comma-separated list of IPs. Will not fail if blank, but why 0_o"
    echo "  -u JENKINS_USER             Jenkins user"
    echo "  -g JENKINS_GROUP            Jenkins primary group"
    echo "  -a SERVER_SUBNET_GROUP      Jenkins group for SSH access, need to pass something when called"
    echo "  -q                          Be quieter"
    echo "  -s                          Set variable to true if more than one IP is passed"
    echo "  -v                          Display Ansible Version"
    exit 1
}

# Initialize variables with default values
skip=false
more_than_one=false
display_version=false
allsubnet_group=missing
be_quiet=false

# Parse command line options
while getopts ":i:u:g:a:svq" opt; do
  case ${opt} in
    i ) # process option i
        IP_LIST=$OPTARG
        ;;
    u ) # process option u
        JENKINS_USER=$OPTARG
        ;;
    g ) # process option g
        JENKINS_GROUP=$OPTARG
        ;;
    s ) # process option s
        skip=true
        ;;
    v ) # process option v
        display_version=true
        ;;
    q ) # process option q
        be_quiet=true
        ;;
    a ) # process option a
        allsubnet_group=$OPTARG
        ;;
    \? ) usage
        ;;
  esac
done
shift $((OPTIND -1))
# Check if all required options are provided
if [ -z "$JENKINS_USER" ] || [ -z "$JENKINS_GROUP" ]; then
    usage
fi

if $display_version; then
    if ! $be_quiet; then
        echo "Showing ansible version"
        ansible --version   
    fi
fi

# Generate an 8-character hash from the IP list
hash=$(echo -n "$IP_LIST" | md5sum | cut -c 1-8)

if ! $be_quiet; then
    echo "IP List:"
    echo $IP_LIST
    echo $hash
fi


# Define the inventory file path with the hash
inventory_file="/var/jenkins_home/ansible/.inv/inventory-$hash.yml"

if $skip; then
    IFS=',' read -ra IPS <<< "$IP_LIST"
    if [ ${#IPS[@]} -gt 1 ]; then
        more_than_one=true
    fi
fi

if $skip; then
    if ! $be_quiet; then
        echo "Single host option set"
    fi
    if $more_than_one; then
        if ! $be_quiet; then
            echo "IP list provided, inventory will be emptied"
        fi
        IP_LIST=""
    fi
fi

# Initialize the YAML inventory content
inventory_content="---
all:
  hosts:
"

# Loop through each IP in the comma-separated list
IFS=',' read -ra IPS <<< "$IP_LIST"
for IP in "${IPS[@]}"; do
    ip_check=$(curl -s http://172.25.100.15:15010/ip_check?ip=${IP} | jq .in_subnets)
    # if this is a restricted subnet, then check the group
    if $ip_check; then
        if ! $be_quiet; then
            echo "Subnet restricted, checking group membership"
        fi
        if [ "$allsubnet_group" == "$SERVER_SUBNET_GROUP" ]; then
            if ! $be_quiet; then
                echo "IP Check Passed, adding endpoint ${IP} to inventory"
            fi
            inventory_content+="    ${IP}:
      ansible_host: ${IP}
        
"
        else
            if ! $be_quiet; then
                echo "Warning: User ${JENKINS_USER} not member of ${SERVER_SUBNET_GROUP}!"
                echo "Auth Check Failed for endpoint ${IP}, not adding to inventory"
            fi
        fi
    # if the subnet is not restricted, just add the endpoint to the inventory
    else
        if ! $be_quiet; then
            echo "Unrestricted subnet, adding endpoint ${IP} to inventory"
        fi
        inventory_content+="    ${IP}:
      ansible_host: ${IP}
"
    fi
done

inventory_content+="  vars:
    ansible_connection: ssh
    ansible_ssh_private_key_file: /var/jenkins_home/jenkins_key
    ansible_python_interpreter: /usr/bin/python3
    jenkins_user: '${JENKINS_USER}'
    jenkins_group: '${JENKINS_GROUP}'
    subnet_group_check: '${allsubnet_group}'
    SERVER_SUBNET_GROUP: '${SERVER_SUBNET_GROUP}'
"

# Write the inventory content to the file
echo "$inventory_content" > $inventory_file

# echo inventory
if ! $be_quiet; then
    echo "Inventory file created at $inventory_file with the following content:"
    cat $inventory_file
fi

This script outputs something like this depending on the flags that are set. This screenshot is also from an older version of the inventory script. 

image.png

After generating the inventory file, Ansible can just be ran by calling the playbook for the inventory. When you pick apart the Jenkinsfiles, the critical portion is this bit. 

ansible-playbook -i \$inventory_file \
/var/jenkins_home/ansible/playbooks/update-endpoint.yaml --ssh-common-args='-o StrictHostKeyChecking=no'

You can see here how Ansible is called on the dynamically generated inventory on a particular playbook. The update playbook is a simple playbook but it still runs a list of roles. This is a good transition to speaking about how Ansible terminology. Ansible is the program, and the standard is that it runs Playbooks, which are lists of Roles, which themselves are lists of Tasks that run. There is a lot more technical stuff about Ansible including some magic word called Idempotency. The straight definition of Idempotency is the property of an operation that can be applied multiple times without changing the result beyond the initial execution. Effectively, this means that an Ansible playbook should have no effect when ran multiple times. Anyway, I just wanted to get that out of the way. I like to use Ansible for tinkering with Linux so that I don't have a result that I don't know how to reproduce. When I start my project in Ansible, I can keep track of every bit I need for it to work from start to finish, and if I screw up catastrophically I can just wipe it back to my base image.