groovy script not working in nifi executescript processor

Question

I'm trying to execute something via executescript processor; a groovy code inside. In the code I'm trying to create a scala script which is to be executed on spark in a further processor.

// Get flow file
def flowFile = session.get()
if (!flowFile) return
// Create output directory
def userInputDir = flowFile.getAttribute("user.input.path")
def finalFolder = new File(userInputDir + "/" + innerDir)

try
{
if (!finalFolder.exists()) finalFolder.mkdirs()
// Write script
file = "spark.sqlContext.setConf(\"hive.exec.dynamic.partition\", \"true\")\n"
file = file + "spark.sqlContext.setConf(\"hive.exec.dynamic.partition.mode\", \"nonstrict\")\n"
file = file + "import org.apache.spark.sql._"
file = file + "\n"
file = file + "import java.io._"
file = file + "\n"

} . . . The rest other steps are adding some other spark specific commands to the script variable. Script is huge so skipping the full code paste. Finally, closing with a catch

// Output file path
flowFile = session.putAttribute(flowFile, "generatedScript", scalaFile.getCanonicalPath())
session.transfer(flowFile, REL_SUCCESS)
}
catch(Exception e)
{
 log.info("File: {}\n", finalFolder.file)
 session.transfer(flowFile, REL_FAILURE)
}

The processor is not even beginning to start the groovy script execution and it fails with the error:

groovy.lang.MissingPropertyException: No such property: script for calss: javal.io.File

By the statement 'not even beginning to start' means that the previous queue is not empty and the processor throws the error. I'm guessing it's a syntax issue but I don't find any syntactical problems related in the script. I also tried running the script in local machine's groovy shell and have the same error there as well, but no syntax issue.

Googling the error got me the suggestion to include the imports in the script but even after including the relevant imports the error is the same.

Any clues?

You try to access 'script' property on a 'File' object. As soon as you are not providing full script and error line - not possible to help (ihmo). Try to use Execute groovy script processor remove try-catch - and you will see detailed stacktrace in log file with line number in your script. — daggett

mattyb mattyb · Accepted Answer · 2018-05-22T13:24:02

You're referring to a variable "innerDir" which is not defined anywhere. Are you expecting the user to add a user-defined property to ExecuteScript called innerDir? If so, the innerDir variable in the script is a PropertyValue object, so you'd need to call getValue() on it to get the actual value of the property:

innerDir.value

Also you refer to scalaFile.getCanonicalPath() but scalaFile is not defined above, and getCanonicalPath() won't give you the contents of the script, is that what you meant?

I reworked the partial script above to assume that innerDir is a user-defined property and you write the contents of the file variable to a File pointed to by scalaFile; also I made it more Groovy by using heredoc instead of appending to the file variable:

// Get flow file
def flowFile = session.get()
if (!flowFile) return
// Create output directory
def userInputDir = flowFile.getAttribute("user.input.path")
def finalFolder = new File(userInputDir + "/" + innerDir?.value ?: '')

try
{
  if (!finalFolder.exists()) finalFolder.mkdirs()
  // Write script
  file = 
"""
  spark.sqlContext.setConf("hive.exec.dynamic.partition", "true")
  spark.sqlContext.setConf("hive.exec.dynamic.partition.mode", "nonstrict")
  import org.apache.spark.sql._
  import java.io._
"""
  scalaFile = new File(finalFolder, 'script.scala')
  scalaFile.withWriter {w -> w.write(file)}
  // Output file path
  flowFile = session.putAttribute(flowFile, "generatedScript", scalaFile.canonicalPath)
  session.transfer(flowFile, REL_SUCCESS)
}
catch(Exception e) {
  log.info("File: {}\n", finalFolder.file)
  session.transfer(flowFile, REL_FAILURE)
}

groovy script not working in nifi executescript processor

1 Answers