Fetching Pig Script Output in Java

Question

Can I load the output of a pig script storing the output in a Pig storage in Java?

The last line of the pig script I am referring to is something like -

STORE D INTO '$output' USING PigStorage();

I want to import the relation in java for further processing and rendering in UI.

If yes, how can I do it?

Nishu Tayal Nishu Tayal · Accepted Answer · 2016-02-04T07:16:25

Yes, you can do it. Use Embedded Pig concept, it provides PigServer which helps to run the pigscript in java program.

import java.io.IOException;
import org.apache.pig.PigServer;

public class WordCount {
   public static void main(String[] args) {
      PigServer pigServer = new PigServer();
      try {
         pigServer.registerJar("/mylocation/tokenize.jar");
         runMyQuery(pigServer, "myinput.txt";
        } 
      catch (IOException e) {
         e.printStackTrace();
        }
   }

   public static void runMyQuery(PigServer pigServer, String inputFile) throws IOException {        
       pigServer.registerQuery("A = load '" + inputFile + "' using TextLoader();");
       pigServer.registerQuery("B = foreach A generate flatten(tokenize($0));");
       pigServer.registerQuery("C = group B by $1;");
       pigServer.registerQuery("D = foreach C generate flatten(group), COUNT(B.$0);");

       pigServer.store("D", "myoutput");
   }
}

Here are the more examples : http://www.programcreek.com/java-api-examples/index.php?api=org.apache.pig.PigServer

Fetching Pig Script Output in Java

1 Answers