Adding two extra columns to input data - Pentaho Kettle

0
votes

I am working on a transformation step for Pentaho Kettle. It selects several input columns and based on that adds two new columns during transformation. I am unable to understand (based on code from other plugins), how I can add the two new columns so that 1) steps downstream are aware of these columns and 2) i can push the transformed data into these columns.

transformation

Thanks in advance.

2
pentahokettlegeokettle
Could you edit your question to include a screenshort of your transformation? – Marcus Rickert
abhi.am/wp-content/uploads/2014/10/tstepkettle11.png – Abhishek Dey Das
My step is a custom plugin that I am working on where I need to collect the rows from upstream, process it, add two columns and send it downstream. – Abhishek Dey Das

2 Answers

0
votes

IF i have understood your question correctly, i think you are trying to create an output file with dynamic column. So you can do this by checking on the "fast dumping" option in Text File Output Step. While doing so , donot define any column names in the "Fields" tab Check my image below:

enter image description here

Hope it helps :)

0
votes

You might need to override meta.getStepFields() to add new ValueMetaInterface objects to the RowMetaInterface passed in. This is the standard way to add columns at runtime; however, the row's metadata (i.e. list of ValueMetaInterface objects) must be the same from row to row or else the next step in your transformation will complain.

Often when doing data-driven custom plugins, you consume as many rows as you need (using getRow()) in order to figure out what the outgoing row format/metadata will be, then you can construct a RowMetaInterface (usually using meta.getStepFields()) that will be passed into the putRow() call. If you intend to pass through the incoming fields, do something like:

RowMetaInterface outputRowMeta = getInputRowMeta().clone();

If you're creating new rows use this:

RowMetaInterface outputRowMeta = new RowMeta();

Either way when you call meta.getStepFields(outputRowMeta, ...) it should populate outputRowMeta with the appropriate fields, by adding/changing/removing ValueMetaInterface objects from outputRowMeta.

I've got a blog post using Groovy to add/replace fields in the incoming rows here:

http://funpdi.blogspot.com/2014/10/flatten-json-to-key-value-pairs-in-pdi.html

Not sure if that is similar to your use case or not. If you have more questions, feel free to find me on IRC at ##pentaho (my nick is usually mburgess_pdi)