Streaming in mule

1
votes

We can operate dataweave over a streaming object [refer]. I have a scenario in which I need to stream 2 separate files and merge the content to write it to a file. What is the most efficient way of doing this...I am avoiding using of async scope as this creates more dependency on processor

1
mulemule-studiomule-componentdataweave
By merge, do you mean just combine two contents or you would be doing any transformations too? - Manik Magar
yeah I want to transform and combine them. - scorpion
Did you try any methodology in mule so far ? - Naveen Raj

1 Answers

0
votes

Streaming is usually forward-only so it works well with single streamed input. Even if you read another streamed input to flow variable, it may be difficult use transformations on them together. Have you tried reading one stream as payload and other into flow variable and then work in dataweave? BTW, will you be doing record (line) level transformation?

Does record of file 1 relates to records of file 2 by some common data OR they are independent set of data with same metadata definitions?

If both are not related, may be you can first combine both files into one (File outbound with append=true) and then stream the combined file and work on transformation.

So it depends on transformation :)