How can I merge two files in Pentaho Data Integration (Kettle)

Question

I'm working with Pentaho Data Integration (Kettle) and I have a question.

I have two input files file1.txt and file2.txt with the same header:

• file1.txt

  NAME;AGE
  alberto;22
  angela;22
  madelaine;23
  

• file2.txt

  NAME;AGE
  carlos;56
  fernando;30
  ana;16
  

and I want to merge both files into one, files_together.txt

NAME;AGE
alberto;22
angela;22
madelaine;23
carlos;56
fernando;30
ana;16

I've tried all (I think) and I don't know how to do it. I've been searching in Google, Youtube... with no positive match.

Thank you very much.

Answer 1

Answer; just put the output of each file you want to merge as input of the final one.

Answer 2

I personally found the "Append Stream" to be more useful as it kept the streams together. By pointing two inputs into one output, they are running in parallel so the results will be interlaced, depending on various factors. Using Append Stream will give you results from file1 then results from file2 in the output.

Answer 3

You must "Select Values" step. The name of the fields must be the same.

Answer 4

I was trying something similar with .csv files. Tried dong what you suggested but it didn't work for me. Many other blogs said "It would be better to use Excel scripting then employing Pentaho Data Integration (Kettle) for this." Which is not true.

You can use "Append Stream" step which is under flow category of Transformation. Which takes two input merge it provide you with expected merged file. You can also this step to merge more number of file with each other.