User defined JSON input to Expected JSON output using PDI Metadata Injection

0
votes

I have created a transformation which will read the Flat Json data from a '.js' file and modify the JSON fields(keys) based on the metadata input field names and meta data target field names specified by the user.

Also, I am using the ETL Metadata Injection step to inject meta data inputs to the steps: JSON input, Select/Rename Values and Json Output.

Please find transformations which I have created for Metadata Injection for JSON Input:

Transformation 1: To get the directory which contains JSON files and calls the JSON Transformation Executor.

Transformation 2: Get the Metadata Inputs, Metadata Target Field, JSON filename and send them to ETL Metadata Injection step

Metadata Injection

Transformation 3: Process the JSON Input from metadata injection step and convert the input JSON into Expected JSON data

Input JSON data to Expected JSON output

When I run the above transformations, I got the below error:

2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Unexpected error 2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : org.pentaho.di.core.exception.KettleException: 2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - The specified field '{0}' could not be found in the input. 2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - 2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - at org.pentaho.di.trans.steps.jsonoutput.JsonOutput.processRow(JsonOutput.java:204) 2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 2017/09/13 15:23:49 - JSON to Expected JSON Output.0 - at java.lang.Thread.run(Thread.java:745)

1
jsonpentahoetlpentaho-data-integration

1 Answers

0
votes

The guilty step is JSON to Expected JSON Output, and the fault is that The specified field '{0}' could not be found in the input.

It seams that this step (which is a JSON output) does not find the first field in the input stream. Which means that the first field you specified on the field tab of the JSON output, is not in the list of the selected field of the previous step. Hint: right click on a step to get the list of input and output fields.

When that works, then you can think about Injecting the metadata, by first changing the Excel input in JSON input.

As I mentioned in your previous post: I am not sure the Metadata Injection is the right approach to rename fields in a JSON. Get your JSON from a Text file in one String and give it to the Javascript step, then use the javascript rich set of high level function to changes the names in the javascript object. And put that modified one String in the output file.

You say "the meta data target field names are specified by the user", how is it done: do you have a dictionary on a file? A two column csv file?