How does Pig handle unstructured data while Hive can't?

3
votes

According to this and other references Pig is better than Hive to process unstructured data. So, the data first cleansed with Pig and then processed with Hive.

But, in the data factory, data may not be in a nice, standardized state yet. This makes Pig a good fit for this use case as well, since it supports data with partial or unknown schemas, and semi-structured or unstructured data.

Would like to know more how Pig can handle unstructured data while Hive can't.

2
comparisonhiveapache-pig
You may want to read this: bluecanarydata.com/hive-for-un-structured-data - Claudio

2 Answers

2
votes

Pig is built to processes schema less data sets..whereas in hive we enforce a schema which is stored in derby or can be configured to store in mysql..Now it is not clear what you are looking for!

0
votes

The key difference between Pig and Hive is that Pig is a dataflow language while Hive is a declarative language. With that being said, Pig can handle unstructured data with no schema defined whereas Hive requires a schema.Also, in some cases Pig can also be used to connect data with a schema giving it an upper hand over Hive. In contrast, Hive converts Hadoop into a dataware house and acts like a SQL dialect. Lastly, you might want to know about Jaql which is another dataflow language. Unlike Pig, its native data structure format is JSON. Similarly, Jaql does not require a schema. Hope this helps.