How to pass arguments to composed task from scdf shell?

Question

I'm trying to launch a composed task via scdf shell. The arguments are not passed to the child tasks properly

Composed-task definition :

composed-task --graph='app1 && app2'

Command to launch from shell:

task launch composed-task-name --arguments "--name=test"

The arguments are passed and executed within composed-task jar instead of child tasks and getting failed. while using the web UI, there is an option called composed-task-arguments which recognize the argument as child tasks' argument.

Composed-task-definition in web UI:

composed-task --graph='app1 && app2' --composed-task-arguments ='--name=test'

--composed-task-arguments is not recognized in data shell. Only --arguments is valid. How do I pass arguments only to child tasks from data shell ?

Answer 1

The section Launching a Composed Task walks explicitly through the scenarios for when properties and arguments needed to be passed to child-tasks from the Shell.

To pass arbitrary arguments to child-tasks in particular, you'd have to use the --arugments property still, but the values would be passed through the composed-task-arguments property. Here's an example.

create:

dataflow:>task create composed-task --definition "t1: timestamp && t2: timestamp"

launch:

dataflow:>task launch composed-task --arguments "--composed-task-arguments=foo=one bar=two"

With this task-launch, both t1 and t2 tasks should have received "foo=one" and "bar=two" properties at runtime.