How to merge multiple csv files into single dataset using SAS

1
votes

I have ~2300 CSV files and colunm 1 variable name is different for each CSV file. I want to merge all files by panelistID (colunm 2) and run frequencies on column 1 to get frequency of each CSV file. Please can someone help?

Below example of file layout: 

File1

mat1_pen1, panelistID

0,  10075001

20, 10086001

44, 10086002

10, 10096001

File2

mat2_pen2, panelistID

74, 10118002

40, 10118003

77, 10128001

77, 10128003


file 3

mat3_pen4, panelistID

77, 10128003

51, 10137001

0,  10148001

0,  10148002

0,  10157001
2
sas
Can you post any code you've already tried or explain what you've already attempted? That will help other people answer your question. - dethtron5000
Can I vote for "get the person who thought it was a good idea to give you 2300 CSV files fired"? - Joe

2 Answers

4
votes

Simply use a wildcard on the infile statement to read in all the files, and the filename= option to store the current file in a temporary variable _f, storing it into f.

Then manipulate f and var accordingly.

data big ;
  length _f f $256. ;
  infile "*.csv" truncover filename=_f dlm=',' ;
  f = _f ;
  input var
        panellistID
        ;
run ;
1
votes
filename mycsv "*.csv";

data mydataset(drop=tmp);
   infile mycsv dsd dlm=',' eov=eov;
   retain mat_pen_id;

   if _n_ = 1  or eov then do;        *when using wildcard-concatenated input files, ;
      input mat_pen_id $20. tmp $20.;    *eov is true for first line of second file.; 
      eov = 0;
    else do;                          * _n_ =1 is true for first line of first file only;
        input mat_pen panelistID;
   end;
run;

proc sort data= mydataset;
   by panelistID;
run;

proc transpose
  data=mydataset out=wide_data;
      by panelistID;
      id mat_pen_id;
      var mat_pen;
      run;

proc print data=wide_data;
run;

This will give you a dataset called wide_data like:

obs   panelistID mat1_pen1 mat2_pen2 mat3_pen3 etc

1      10075001     0        22            33