How to load a file in Pig that has a variable number of fields

0
votes

I have an input file which contains records as follows:

Movie1     Actor 1, Actor 2, Actor 3, ......, Actor n
Movie2     Actor 1, Actor 2,.......Actor n

I want to load this data into a bag in Pig

 movies = LOAD 'movies.imdb' AS (......);

I am not sure how to fill up my "AS" field because the records in my input file may have variable number of fields.

1
hadoopapache-pig

1 Answers

2
votes

You could just do this:

movies = LOAD 'movies.imdb' USING PigStorage(',');

If the records don't have fixed fields, using AS won't be possible