I have an interesting ImageJ scripting problem that I wanted to share. An imaging scientist gives to me a data set with 258 time points with 13 'Z-stack' images. All together there are 3,354 tif images. He has a macro that he made using the imageJ macro recording functionality, and the macro works on his windows machine except it takes forever. I have access to a very large academic computing cluster where I could conceivably ask for as many nodes as time-points. The input files are the 3,354 tif images labelled like 'img_000000000_ZeissEpiGreen_000.tif', the nine digit number increases 1-258 and the three digit number is the Z-stack order 1-13, the other input file is a point-spread function image made with tiny beads at sub-resolution. Here is the Macro "iterative_parallel_deconvolution.ijm". I changed the paths to correspond to the necessary paths in the cluster.
//******* SET THESE VARIABLES FIRST! ********
path = "//tmp//images//";
seqFilename = "img_000000000_ZeissEpiGreen_000.tif";
PSFpath = "//tmp//runfiles//20xLWDZeissEpiPSFsinglebeadnoDICprismCROPPED64x64.tif";
numTimepoints = 258;
numZslices = 13;
xyScaling = 0.289 //microns/pixel
zScaling = 10 //microns/z-slice
timeInterval = 300; //seconds
//********************************************
getDateAndTime(year1, month1, dayOfWeek1, dayOfMonth1, hour1, minute1, second1, msec); //to print start and end times
print("Started " + month1 + "/" + dayOfMonth1 + "/" + year1 + " " + hour1 + ":" + minute1 + ":" + second1);
//number of images in sequence
fileList = getFileList(path);
numImages = fileList.length;
//filename and path for saving each timepoint z-stack
pathMinusLastSlash = substring(path, 1, lengthOf(path) - 1);
baseFilenameIndex = lastIndexOf(pathMinusLastSlash, "\\");
baseFilename = substring(pathMinusLastSlash, baseFilenameIndex + 1, lengthOf(pathMinusLastSlash));
saveDir = substring(path, 0, baseFilenameIndex + 2);
//loop to save each timepoint z-stack and deconvolve it
for(t = 0; t < numTimepoints; t++){
time = IJ.pad(t, 9);
run("Image Sequence...", "open=[" + path + seqFilename + "] number=" + numImages + " starting=1 increment=1 scale=100 file=[" + time + "] sort");
run("Properties...", "channels=1 slices=" + numZslices + " frames=1 unit=um pixel_width=" + xyScaling + " pixel_height=" + xyScaling + " voxel_depth=" + zScaling + " frame=[0 sec] origin=0,0");
filename = baseFilename + "-t" + time + ".tif";
saveAs("tiff", saveDir + filename);
close();
// WPL deconvolution -----------------
pathToBlurredImage = saveDir + filename;
pathToPsf = PSFpath;
pathToDeblurredImage = saveDir + "decon-WPL_" + filename;
boundary = "REFLEXIVE"; //available options: REFLEXIVE, PERIODIC, ZERO
resizing = "AUTO"; // available options: AUTO, MINIMAL, NEXT_POWER_OF_TWO
output = "SAME_AS_SOURCE"; // available options: SAME_AS_SOURCE, BYTE, SHORT, FLOAT
precision = "SINGLE"; //available options: SINGLE, DOUBLE
threshold = "-1"; //if -1, then disabled
maxIters = "5";
nOfThreads = "32";
showIter = "false";
gamma = "0";
filterXY = "1.0";
filterZ = "1.0";
normalize = "false";
logMean = "false";
antiRing = "true";
changeThreshPercent = "0.01";
db = "false";
detectDivergence = "true";
call("edu.emory.mathcs.restoretools.iterative.ParallelIterativeDeconvolution3D.deconvolveWPL", pathToBlurredImage, pathToPsf, pathToDeblurredImage, boundary, resizing, output, precision, threshold, maxIters, nOfThreads, showIter, gamma, filterXY, filterZ, normalize, logMean, antiRing, changeThreshPercent, db, detectDivergence);
}
//save deconvolved timepoints in one TIFF
run("Image Sequence...", "open=["+ saveDir + "decon-WPL_" + baseFilename + "-t000000000.tif] number=999 starting=1 increment=1 scale=100 file=decon-WPL_" + baseFilename + "-t sort");
run("Stack to Hyperstack...", "order=xyczt(default) channels=1 slices=" + numZslices + " frames=" + numTimepoints + " display=Grayscale");
run("Properties...", "channels=1 slices=" + numZslices + " frames=" + numTimepoints + " unit=um pixel_width=" + xyScaling + " pixel_height=" + xyScaling + " voxel_depth=" + zScaling + " frame=[" + timeInterval + " sec] origin=0,0");
saveAs("tiff", saveDir + "decon-WPL_" + baseFilename + ".tif");
close();
getDateAndTime(year2, month2, dayOfWeek2, dayOfMonth2, hour2, minute2, second2, msec);
print("Ended " + month2 + "/" + dayOfMonth2 + "/" + year2 + " " + hour2 + ":" + minute2 + ":" + second2);
The website for the ImageJ plugin Parallel Iterative Deconvolution is here: https://sites.google.com/site/piotrwendykier/software/deconvolution/paralleliterativedeconvolution
Here is the PBS script which I used to submit the job to the cluster, with this command: 'qsub -l walltime=24:00:00,nodes=1:ppn=32 -q largemem ./PID3.pbs'. I could have requested up to 40 ppn, but program states that they must be a power of 2.
#PBS -S /bin/bash
#PBS -V
#PBS -N PID_Test
#PBS -k n
#PBS -r n
#PBS -m abe
Xvfb :566 &
export DISPLAY=:566.0 &&
cd /tmp &&
mkdir -p /tmp/runfiles /tmp/images &&
cp /home/rcf-proj/met1/pid1/runfiles/* /tmp/runfiles/ &&
cp /home/rcf-proj/met1/pid1/images/*.tif /tmp/images/ &&
java -Xms512G -Xmx512G -Dplugins.dir=/home/rcf-proj/met1/software/fiji/Fiji.app/plugins/ -cp /home/rcf-proj/met1/software/imagej/ij.jar -jar /home/rcf-proj/met1/software/imagej/ij.jar -macro /tmp/runfiles/iterative_parallel_deconvolution.ijm -batch &&
tar czf /tmp/PIDTest.tar.gz /tmp/images &&
cp /tmp/PIDTest.tar.gz /home/rcf-proj/met1/output/ &&
rm -rf /tmp/images &&
rm -rf /tmp/runfiles &&
exit
We have to use Xvfb to keep ImageJ from sending out images to a non-fake display the display number is arbitrary. This program ran for six hours but made no output images, is it because I needed to have a open image?
I would like to redesign this macro so that I can split up each time-point and send it to its own node for processing. If you have any ideas on how you would go about this, we would be really grateful for your feedback. The only caveats are that we have to use the parallel iterative deconvolution software plugin with ImageJ
Thanks!
