pig jsonloader array parse issue

0
votes

I have a json encoded data below need to be parsed using pig.

{"arr":[1,2,3,4]}

According a document at http://help.mortardata.com/technologies/pig/json#toc_4JsonLoaderSchemaGuidelines , my loading schema is:

a = load 'testJsonPig.log' using JsonLoader('arr:{t:(i:int)}');

and dump it:

dump a;

but I got an unepected result:

()

the result is empty, and I don't know why it happened, It seemed that I met a same question as this link Pig default JsonLoader schema issue described.

My pig version is

Apache Pig version 0.12.1 (r1585011)

and wish someone could help me, thx in advance.

4
jsonhadoopapache-pig

4 Answers

1
votes

PIG-2949 was an issue that described this problem. It was resolved and should work as of Pig 0.15.0 out-of-the-box. So yes - as the OP discovered - elephantbird is the only easy way to handle such arrays now, but soon Pig will be able to handle them natively.

(full disclosure - I closed this issue myself)

1
votes

arr:bag{a:tuple(a:int)} should work.

0
votes

finally I use elephantbird jsonloader to solve my problem.

https://github.com/kevinweil/elephant-bird/

0
votes

the bag and tuple keywords in the above solution could be skipped.

a = load 'testJsonPig.log' using JsonLoader('arr:{(int)}');
DUMP a;

output: ({(1),(2),(3),(4)})

creates a bag of tuples, each containing 1 element.