NiFi: The incoming flow file can not be read as an Avro file

0
votes

I just started with NiFi 1.4.

I am trying to send pipe delimited message via kafka into Hive. So I am using ConsumeKafkaRecord_0_10 and PutHivStreaming processor. Consume Kafka reader sends data on success to PutHiveStreaming.

Consume Kafka reader writing data in avro format but PutHiveStreaming gives error as

The incoming flow file can not be read as an Avro filee: java.io.IOException: Not a data file."

1
hiveapache-kafkaavroapache-nifi

1 Answers

3
votes

PutHiveStreaming can only read Avro datafiles, so you have to make sure the writer used by ConsumeKafkaRecord is an AvroRecordSetWriter with Schema Write Strategy set to Embedded Schema.

If the schema isn't embedded then when it gets to the Hive processor, it won't be a valid Avro datafile.