2
votes

I am trying to parse JSON file and store the data in a SAS dataset Here is the sample of JSON file .

Right now I'm only trying to parse S array as an initial step. It has 2 objects so the output SAS dataset(Final goal is to create an hash table) would have 2 observations as shown below.

1   14162   E       NG     1455209    98
2   14163   EN      NGG    1455210    784  

Here is the code that I tried....but not getting any rows. As DS2 is new in SAS 9.4M3, i couldn't find much documentation aswell. I guess it's going wrong at "do while (j.ISRIGHTBRACE(tokenType));" but not sure.

proc ds2;
data Snaps (overwrite=yes);
dcl package json j();
dcl nvarchar(30)  Key value;
dcl varchar(1000000) character set utf8 response;
dcl int rc;

dcl package http webQuery();
dcl int rc tokenType parseFlags;
dcl nvarchar(128) token;
dcl integer i rc;
drop response rc;
retain i 0;

method init();
webQuery.createGetMethod('http://XXXXXXXX.com:9090/query?Date=20160211');
webQuery.executeMethod();
webQuery.getResponseBodyAsString(response, rc);
rc = j.createParser( response );
do while (rc = 0);
     i=i+1;
     j.getNextToken( rc, token, tokenType, parseFlags);
     put token=;
     if lowcase(strip(token)) = 'shots' then do;
     put 'inside shots =-----------------------------------------';
           j.getNextToken( rc, token, tokenType, parseFlags);
           put '--------------------ARRAY---------------------' token=;
           if j.ISLEFTBRACKET(tokenType) then do; *Entered in to an Array;
                j.getNextToken( rc, token, tokenType, parseFlags);
                put '------------------OBJECT-----------------------' token=;
                if j.ISLEFTBRACE(tokenType) then do;*Entered in to an Object;
                     do while (j.ISRIGHTBRACE(tokenType));
                           j.getNextToken( rc, token, tokenType, parseFlags);
                           put '------------------KEY-----------------------' token=;
                           if j.ISSTRING(tokenType) then do;
                                                Key=token;
                                                     j.getNextToken( rc, token, tokenType, parseFlags);
                                                     put '---------------------VALUE--------------------' token=;
                                                Value=token;
                                                Output;

                           end;
                     end;
                end;
           end;
     end; 
    if i > 100 then stop;
end;

end; * method init;
method term();
    rc = j.destroyParser();
end;

enddata;
run;
quit;

Thanks.........

1
Since you're doing it using DS2, I'm assuming you've seen this blog post: blogs.sas.com/content/sasdummy/2015/09/28/parse-json-from-sas . I also have to parse a JSON file, and it is very difficult. SAS just does not have an easy way to parse these...I managed to make a very very basic parse using the data step, but it's not reliable. I applaud you for using DS2! It's very powerful, but so new that very few people know it well.Stu Sztukowski
@StuSztukowski : Yeah, that was the first blog that I stumbled up, which uses very basic json file. Ours is very complicated Json structure.The Json file that I pasted in my question is only a part of our actual Json.SAS_learner
Could you provide an example of the full structure of your JSON file? The sample you've provided is simple enough to deal with via a data step in my opinion, and without this extra information it will be difficult for people to help you any further.user667489

1 Answers

2
votes

I am trying to parse JSON file and store the data in a SAS dataset

Like any good analyst these days know to get your data read in your preferred processing language is key, but don't always think one dimension. My Answer comes by using Python to convert the JSON file into an easier format for SAS to then turn into dataset, such as a CSV file.

Taking my cue from this reddit post https://www.reddit.com/r/Python/comments/2l3fx9/converting_json_to_csv/

Depends on your platform, but something like this

at a command line type 'python'. If it is installed go to 3. otherwise

download and install python 2.7 from https://www.python.org/downloads/

You may need to install pip, run "python get-pip.py" (http://docs.python-guide.org/en/latest/starting/install/win/)

at a terminal (as per their instructions) type: $ pip install -e

Now at a terminal type (as per their instructions): python json_to_csv_converter.py yelp_academic_dataset.json

the file yelp_academic_dataset.json needs to be in your current location.

And obverse here you would change yelp_academic_dataset.json with your JSON file name.