Pentaho Spoon Data Integration, Merge two csv

0
votes

I want to merge two csv files. The first looks like that: url;name;api. url fields contain 50 times the same url, name is empty and api fields contain 50 times the same api key. The second csv looks like that: url;name;api. url is empty, name contains 50 different names, api is empty.

I want to create one csv file: url;name;api with the information from both csv, so url and api contains the value from the first and name from the second csv.

so far i just create two csv input and hooked them with one csv output. The resulting csv has 100 records, 50 with null as name and 50 with null in url and api.

. enter image description here

enter image description here

enter image description here

1
pentahokettlespoon
So the first file is 50 copy of the same row. Is it the real use case, or is it a simplification of the real use case? How many rows do you want in the result: 50 rows or 50x50=2500 rows ? - AlainD
i want to have 50 rows which look like url+name+api. My idea is to get 50 artist names from last.fm and than get the songs from this 50 artist by a second rest call. If there is a better way to create several calls with changing parameter i would be happy if u could tell me. All i found is a data grid, but there i have to insert all the url manually. - Jonas

1 Answers

2
votes

If you want to increase the number of columns, you need a Lookup or a Join.

You have a main stream and for each row of the stream, you fetch one row (Lookup) or many rows (Join) in the secondary stream.

In your case, I would first remove all the unnecessary field (the name in the first csv file, and the url and api in the second), and unnecessary rows (keep only one row in the first csv file). Then drop both flows in a Join Rows steps.

And, if I do understand your use case, you did probably over complicated the case, because to direct the flow in the same step (here the Text output 2) needs the columns to be the same, so that they can add the rows of the input one after the other.