Avro in-memory with parquet as storage

Question

I am new to parquet and trying to understand the various intricacies that come with using parquet. One of the points that frequently pop is that parquet as a storage format should be used with Avro as an in-memory representation. Unfortunately, I dont understand how Avro can be useful with Parquet. Can someone please explain this combination to me?

Thanks!!

Answer 1

Avro is a binary format to represent an object structure. It has the advantage of being compressed.

I don't know enough about Parquet's inner details to say why Avro, but it may be optimized to store that specific format. Nevertheless using Avro as the object format for storage is usually a big space saver.