Pentaho PDI (Kettle/spoon) Meta Data Injection for text files

0
votes

I am interested in importing a Fixed Width file using Pentaho PDI. I have used its main GUI tool that sets the widths graphically in Spoon. BUT if the number of fields is very large, like a few hundred fields, it would be prone to error and take a lot of time. In other ETL tools, I am able to import a meta-file that describes the column properties, such as name, size etc.

I see that pentaho has this thing calls Meta-Data Injection, but there is not much tutorials at all, just a couple, and either the use cases are really complex and make use of Javascript for scripting or they describe it in very abstract ways.

So hope someone who is familiar can explain my particular use case of Fixed Width files.

1
pentahopentaho-spoonpentaho-data-integration
unable to understand what you are asking? do you want meta data injection sample?] – Helping Hand..
Hi, I am not asking for a sample, I am asking what are the steps to do Meta Data Injection in Pentaho. Its not clear at all based on the little tutorials that are out there. This is how I imagine it might work, I create a text-file that has the properties of the data file, like name of column the width size of columns, etc. Is there a way to import this file in the Meta-data step have it read the properties then in next step import the data-file based on the details of the properties file. – Palu

1 Answers

1
votes

Yes, You can use metadata injection step for applying dynamic properties like filename, fields, length datatype etc..

For that,

  1. You need to create one transformation with file input step.
  2. Create another transformation with the metadata injection step and where you can add a transformation that created in step1.
  3. In inject Metadata tab of metadata injection step you can add length from the input step.