concatenate many tables in sas

Question

This is to follow the post Efficiently concatenate many sas datasets: Efficiently concatenate many sas datasets

Well, I'm having a similar task, i.e to concatenate more than 10000 sas tables of the same structure. Each table consists of 2 rows and 13 columns and are named in sequential (data, data1, data 2, ...). All my tables are saved in an external drive: "D:\MySASfolder\". The concatenation using SET in datastep seems to be running out of memory and the proc append may be the solution. However, I've some difficulties to understand the code made by BellevueBob, in particular I'm not sure how the macro can revoke my 10000 data tables. Can anyone please help me to go through the code lines please. Many thanks, Mai

Bob's answer uses a simple macro %do loop to work through all the tables. Do a bit of background reading on SAS macros and then post another question if there's a part of it that you still don't understand. — user667489
Possible duplicate of Efficiently concatenate many sas datasets — user667489
Did you try using member list in the SET statement? data want; set data data1-data9999; run; Does it work with than many datasets referenced in a single SET statement? — Tom
Beyond the correct solutions below also consider the INDSNAME option that allows you to identify the input data set name. — Reeza

Joe Joe · Accepted Answer · 2015-10-19T15:13:33

The other question actually isn't exactly the same as yours; it has a problem in that the data sets individually were being created on demand, and then appended.

If the datasets already exist, you have a far easier problem.

For example, let's make data1-data3 (which you've already done in a prior step) and then append them, using the single dash operator (which creates a new name list that contains all names in between the first and last name, in numeric order).

data data1;
  x=1;
run;

data data2;
  x=2;
run;

data data3;
  x=3;
run;

data want;
  set data1-data3;
run;

Of course, if you have data you will have to also add that.

data want;
  set data data1-data3;
run;

You can put a libname on this as well.

data want;
  set mylib.data1-mylib.data3;
run;

concatenate many tables in sas

1 Answers