Clean the slaves

In my current project we’re generating jobs for each branch of each project with the help of the Jenkins Job DSL. Branches are created for each feature and bugfix – hotfixes are branched from release branches but these are treated a little differently. Anyway, as you can imagine we tend to have a lot of branches even though we delete them once they’re merged back into the master. Since we also have a lot of different jobs for building/testing/deploying, the amount of jobs explodes quite quickly ;)
Also, to have acceptable job execution times and fast feedback for the developers, we’re using multiple slaves.

Problem

When deleting a job, e.g. because the branch is removed, its workspace on the slave is not deleted. This is a little annoying since it’s wasting a lot of space on the slaves’ disks and in the past required some manual effort to clean them up. Luckily, Jenkins provides some nice APIs :)

Solution

We created a job which is executing a “system groovy script” (important!) and must be run on the Jenkins master. It simply runs through all slaves, checks if their workspace contains a folder which is orphaned and deletes it. And yes, it is as simple as it sounds. Just a few lines of code which are run after the job generation and all orphaned workspaces are removed :mrgreen:

import hudson.model.*
import hudson.node_monitors.*
import jenkins.model.*

for (node in Jenkins.instance.nodes) {
    computer = node.toComputer()
    if (computer.getChannel() == null) {
        continue
    }

    def rootPath = node.getWorkspaceRoot()
    def space = DiskSpaceMonitor.DESCRIPTOR.get(computer)

    println "Checking node ${node.name}..."
    println "Got path ${rootPath} and remaining space ${space}"

    rootPath.list().sort().each { dir ->
        if (!Jenkins.instance.getItem(dir.name)) {
            println "${dir} orphaned. Deleting..."
            dir.deleteRecursive()
        } else {
            println "${dir} still in use. Skipping."
        }
    }
}

2 comments

  • What we do in order to overcome this hassle is firing up docker containers to test/build any application. This solves the issue by design.
    We use gitlab CI instead of Jenkins, but it shouldn’t differ too much. If you didn’t explore that option, you may consider it!

  • Indeed we’re using docker for some jobs but it isn’t a solution for all of them, e.g. running a Windows7/10 or MacOSX to test on the appropriate browsers. So some slaves still need to be cleaned.

Leave a Reply