IML correlation from different matrices

1
votes

given a matrix X(n * p), I want to split X into Y1(n * p-k) and Y2(n * k), where Y1 is composed by the first k columns of X and Y2 the others.

Now, in R I can get the "crossed" correlation between the columns of Y1 and Y2 calling cor(Y1,Y2, use="pairwise.complete.obs"), how can I get the same result in SAS IML where the corr function admits only 1 dataset?

I tried to find an appropriate solution or algorithm to implement it but with bad results.

Can anyone help with this? Also pointing me some literature about this kind or correlation would be great! I don't want you to code it for me, simply some help or hint on existing functions or algorithms to translate.

Thank you.

EDIT: don't search on the web for crossed correlation, I wrote it simply for trying to explain myself.

2
sassas-iml
(I seriously hope that Rick from "the do loop" blog will come here) - stat
I added the R tag for hopes that an R guru will be able to point you to literature. - DomPazz
I don't think r is appropriate. Tags aren't supposed to be used unless it's actually relevant to the question, and (unless it's edited to address this) the question doesn't ask what the r function does. Maybe it should, but... - Joe

2 Answers

1
votes

Looking up "crossed correlation" leads you to a series of literature on signal processing and a function much like the autocorrelation function. In fact, in R it is documented with acf https://stat.ethz.ch/R-manual/R-devel/library/stats/html/acf.html.

But that is not what your code is doing. In R:

n = 100
p = 6
k = 2

set.seed(1)

r = rnorm(n*p)
x= matrix(r,n,p)

y1 = x[,1:k]
y2 = x[,(k+1):p]

cor.ys = cor(y1,y2,use="pairwise.complete.obs")

cor.x = cor(x)

(cor.ys - cor.x[1:k,(k+1):p])

You see the result from cor(y1,y2) is just a piece of the correlation matrix from x.

You should be able to put this in IML easily.

1
votes

I can think of a few ways to do this. The simplest is to compute the full matrix of Pearson correlations (using the pairwise option) and then subset the result. (What DomPazz said.) If you have hundreds of variables and you only want a few of the correlations, it will be inefficient, but it is very simple to program:

proc iml;
n = 100;  p = 6;  k = 2;
call randseed(1);
x = randfun(n//p, "Normal");
varNames = "x1":"x6";

corr = corr(x, "pearson", "pairwise");   /* full matrix */
idx1 = 1:k;                              /* specify VAR */
idx2 = (k+1):p;                          /* specify WITH */
withCorr = corr[idx2, idx1];             /* extract submatrix */
print withcorr[r=(varNames[idx2]) c=(varNames[idx1])];

Outside of SAS/IML you can use PROC CORR and the WITH statement to do the same computation, thereby validating your SAS/IML program:

proc corr data=test noprob nosimple;
var x1-x2;
with x3-x6;
run;