Convert Avro To Parquet in Apache Nifi

0
votes

I wanted to convert my json data to parquet form.

I have used GenerateFlowFile and put dummy json value [{"firstname":"prathik","age":21},{"firstname":"arun","age":22}].

I have already converted json to Avro using ConvertRecord processor, i have then used ConvertAvroToParquet processor.

I got the following output (When viewed as original):

PAR1&&,6(prathikarunprathikarun,(<Hnifi.person% firstname%%age,&% firstnamenn&<6(prathikarun&v%agejj&v<(Ø,parquet.avro.schema€{"type":"record","name":"person","namespace":"nifi","fields":[{"name":"firstname","type":"string"},{"name":"age","type":"int"}]}writer.model.nameavroJparquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a),ÃPAR1

Where is a red dot.

I tried viewing the output in "formatted" form but i get this message : Unable to generate view of data: Not a data file

Thanks in advance

1
jsonapache-nifiavroparquet
You won't be able to view Parquet in the content viewer in NiFi. You can send the flow file to PutFile and then use parquet-tools – Bryan Bende
Could you elaborate parquet-tools. Thanks. – prathik vijaykumar
github.com/apache/parquet-mr/tree/master/parquet-tools stackoverflow.com/questions/53306327/installing-parquet-tools – Bryan Bende

1 Answers

0
votes

Parquet is a binary file format and not intended to be viewable within the NiFi content viewer. In newer versions of NiFi you have the ParquetWriter than can be used with ConvertRecord to do the conversion and get the converted Avro data ready to be uploaded.