How to identify hung slaves for a given jenkins master

Question

I have a Jenkins master and a number of slaves statically (!) connected to it. That is to say, that each node doesn't have executors. I'm trying to identify hung slaves by monitoring how much time the slave handles a certain job in oreder to reboot the slave. I read carefully Groovy's documentation for Jenkins here: http://javadoc.jenkins-ci.org/ but could not find any way to identify a busy slave. The only way I found that it can be done is relevant only if exectors are used (in a similar way to this: https://wiki.jenkins-ci.org/display/JENKINS/Find+builds+currently+running+that+has+been+executing+for+more+than+N+seconds).

Any idea how can I do it?

edited: This is my solution using system Groovy script:

import hudson.model.*
import hudson.node_monitors.*
import hudson.slaves.*
import java.util.concurrent.*
import jenkins.model.Jenkins
import hudson.util.RemotingDiagnostics    
jenkins = Hudson.instance    
import javax.mail.internet.*;
import javax.mail.*
import javax.activation.*    
import hudson.*;
import hudson.tasks.*;
import hudson.model.AbstractBuild
import hudson.Launcher
import hudson.model.BuildListener
import hudson.FilePath
import groovy.io.FileType
import jenkins.util.VirtualFile;



int MAX_ALLOWED_DURATION_IN_SECONDS = 60 * 30 // 30 minutes


for (computer in jenkins.model.Jenkins.instance.computers)
{
    for(e in computer.executors)
    {
        if(e.isBusy())
        {
            int durationInSeconds = (System.currentTimeMillis() - e.executable.getStartTimeInMillis())/1000.0
            if(durationInSeconds > MAX_ALLOWED_DURATION_IN_SECONDS )
                {
                    print "restarting slave:"
                    println computer.getName()
                    println "\n\n\n"

                    def channel = computer.getChannel()
                    println RemotingDiagnostics.executeGroovy(""""cmd /c shutdown -r".execute().text""",channel)
                    break
                }
        }
    }
}
println "Done"

return 0

Answer 1

I haven't tested it myself, but try to apply Computer.countBusy() or Computer.isIdle() from Computer class.