Jenkins Slave Self Register

Question

I am creating a Jenkins master/slave cluster and I am having trouble finding a way to have new slaves auto register themselves with the master.

My current set up is I run some Terraform scripts that will create the master and 5 slaves. Then I have to log in to the master node and Manage Jenkins -> Manage Nodes -> New Node and manually create the number of nodes I want.

Then I RDP into my slaves and run the command java -jar agent.jar -jnlpUrl http://yourserver:port/computer/agent-name/slave-agent.jnlp. This works perfectly fine, but I would like a way to auto scale up/down the number of agents without having to manually log into the slaves every time I create a new one.

Is there a plugin or some documentation I'm missing about how to dynamically self register nodes?

NOTE: This only applies to windows nodes. I am using the Kubernetes plugin to auto scale up/down linux nodes, but Kubernetes does not have stable windows nodes support so I can't use that. I have to support classic .NET applications (not .NET Core) so I have to build on windows nodes.

Answer 1

Here's a bash script I use on Linux, it could be adapted fairly easily for Windows.

#!/bin/bash

set -xe

MASTER_URL=$1
MASTER_USERNAME=$2
MASTER_PASSWORD=$3
NODE_NAME=$4
NUM_EXECUTORS=$5

# Download CLI jar from the master
curl ${MASTER_URL}/jnlpJars/jenkins-cli.jar -o ~/jenkins-cli.jar

# Create node according to parameters passed in
cat <<EOF | java -jar ~/jenkins-cli.jar -auth "${MASTER_USERNAME}:${MASTER_PASSWORD}" -s "${MASTER_URL}" create-node "${NODE_NAME}" |true
<slave>
  <name>${NODE_NAME}</name>
  <description></description>
  <remoteFS>/home/jenkins/agent</remoteFS>
  <numExecutors>${NUM_EXECUTORS}</numExecutors>
  <mode>NORMAL</mode>
  <retentionStrategy class="hudson.slaves.RetentionStrategy\$Always"/>
  <launcher class="hudson.slaves.JNLPLauncher">
    <workDirSettings>
      <disabled>false</disabled>
      <internalDir>remoting</internalDir>
      <failIfWorkDirIsMissing>false</failIfWorkDirIsMissing>
    </workDirSettings>
  </launcher>
  <label></label>
  <nodeProperties/>
  <userId>${USER}</userId>
</slave>
EOF
# Creating the node will fail if it already exists, so |true to suppress the
# error. This probably should check if the node exists first but it should be
# possible to see any startup errors if the node doesn't attach as expected.


# Run jnlp launcher
java -jar /usr/share/jenkins/slave.jar -jnlpUrl ${MASTER_URL}/computer/${NODE_NAME}/slave-agent.jnlp -jnlpCredentials "${MASTER_USERNAME}:${MASTER_PASSWORD}"

This is somewhat similar to the agent launchers included in the docker slave images, but before it runs jnlp it uses the jenkins cli to create the node on jenkins. Some of the parameters would need adapting to windows obviously.

EDIT: and to get that xml the easiest way is to create a node how you want in the web ui then use jenkins-cli to retrieve it.

Answer 2

I've been testing this on AWS.

https://plugins.jenkins.io/swarm

Although you cannot use broadcast on AWS you can add to the java command and specify the hostname or URL of the LB of the jenkins master.

I have not checked to see how this works on windows yet but will be doing that soon and will let you know how it goes.

Answer 3

I am using the Swarm Plugins https://plugins.jenkins.io/swarm to connect my Windows Clients to Jenkins. At start-up I let my VMs running this Powershell Skript:

function startJenkinsSlave()
{
    [CmdletBinding(SupportsShouldProcess=$true)]
    param (
        [parameter(Position = 1, Mandatory = $true)]
        [string]$jkMasterUrl,
        [parameter(Position = 2, Mandatory = $true)]
        [string]$jkSlaveName,       
        [parameter(Position = 3, Mandatory = $true)]
        [string]$jkSlaveUser,
        [parameter(Position = 4, Mandatory = $true)]
        [string]$jkSlaveSecret
    )

    Write-Host "--- start jenkins swarm slave ---"
    Write-Host "download new Version of swarm-client.jar"   
    $jkSwarmJarUrl="$jkMasterUrl/swarm/swarm-client.jar"
    $jkJarFilePath="C:\Program Files\Jenkins\swarm-client-$($jkSlaveName).jar"
    $javaExePath="C:\ProgramData\Oracle\Java\javapath\java.exe"
    Try {
        [io.file]::OpenWrite($jkJarFilePath).close()
        Get-ItemProperty -Path $jkJarFilePath -ErrorAction SilentlyContinue
        $client = new-object System.Net.WebClient
        $client.DownloadFile($jkSwarmJarUrl, $jkJarFilePath)
        Write-Host "neueste Version vom swarm-client.jar wurde heruntergeladen"
        Get-ItemProperty -Path $jkJarFilePath
    }
    Catch {
        Write-Warning "Unable to write to output file $jkJarFilePath"
    }


    Write-Host "Jenkins slave will start:"
    & $javaExePath '-Dfile.encoding=UTF8' -jar $jkJarFilePath -deleteExistingClients -master $jkMasterUrl -username $jkSlaveUser -password $jkSlaveSecret -labels "W10-swarm $jkSlaveName"
}


$jkSlaveUser='JenkinsUserForSwarm'
# Use access-token and not password!
$jkSlaveSecret='1d1a700e0a0981ef74f23efa9a6c90d39d'
$jkMasterUrl='http://jenkins.onmyhost.local:8080'
$vmName=(Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Virtual Machine\Guest\Parameters' -name 'VirtualMachineName').VirtualMachineName

Write-Host "Starting Jenkins Swarm"
while($true)
{
  try
  {
    startJenkinsSlave -jkMasterUrl $jkMasterUrl -jkSlaveName $vmName -jkSlaveUser $jkSlaveUser -jkSlaveSecret $jkSlaveSecret
  }
  finally
  {
    # ctrl-c will the skript.
    Write-Host "Jenkins Slave service ended. Restart in 120 seconds" 
    Start-Sleep -Seconds 120
  }
}

This script works stabile and do the job for Windows slaves usefull when you need UI-iteractions. You may also use env variable for the different parameters (jenkins master, user, token, ...). For Unix slaves there may be better Solutions with k8s ans ssh.

Regards, Éric.