Compare only pairwise comparisons BETWEEN two matrices using rcorr() in R

Question

hopefully this post isn't a duplicate, but I've spent quite some time searching and haven't been able to find the answer.

I have two matrices that I am trying to make pairwise correlations between, that look like this.

Matrix 1
       gene1     gene2     gene3
ID1    12        32        43
ID2    94        34        95
ID3    90        54        23
ID4    43        76        65

Matrix2
       TE1       TE2       TE3    
ID1    94        90        82
ID2    23        46        94
ID3    23        49        39
ID4    39        34        46

I'm able to get a table of R^2 values using the base function cor(), which only makes comparisons BETWEEN the matrices, in a pairwise way. The result looks something like this:

     gene1        gene2        gene3
TE1  0.98         0.48         0.45
TE2  0.77         0.46         0.76
TE3  0.45         0.56         0.76

Which is great! But the problem is I need p values too, so I can cut the matrix down to those that have only p values less than a certain cutoff (because my real matrices are [30,800] and [30,1000] and I need a way to cut down the data to something intelligible).

The rcorr() package is great at this, as it produces a matrix r of correlations, a matrix P of p values, and a vector of the number of observations. BUT, I have not been able to figure out a way to compare only BETWEEN matrices- it also compares WITHIN matrices. I get a result looking like this:

       gene1   gene2   gene3   TE1   TE2   TE3
gene1  1.0     0.5     0.5     0.5   0.3   0.9
gene2          1.0     0.4     0.7   0.7   0.5
gene3                  1.0     0.8   0.8   0.5
TE1                            1.0   0.8   0.2
TE2                                  1.0   0.7
TE3                                        1.0

This is made up data, but it illustrates the point. This produces twice as much data as I actually need, and slows down the calculations as well as produces correlation graphs that are nonsense, visually.

So my question is this: Is there a way to compare only between, not within matrices using the Hmisc package function rcorr()?

I've also tried the cor.test on lists of the matrices:

cor.test(c(matrix1),c(matrix2), method="pearson")

but I get an error that 'x' and 'y' must have the same length.

Next I have to figure out how to actually subset the matrices based on the list of those with high correlation and low p value, but I see there's some helpful answers here that I need to scrutinize.

Ernest A Ernest A · Accepted Answer · 2016-07-28T12:20:54

There probably is an easier way to do it, but one option is to do a cor.test for every column pair

tmp <- with(expand.grid(seq(ncol(matrix1)), seq(ncol(matrix2))),
            mapply(function(i, j) cor.test(matrix1[, i], matrix2[, j]),
                   Var1, Var2))

and then extract the elements from the test objects

matrix(unlist(tmp['estimate', ]), nrow=ncol(matrix1),
       dimnames=list(colnames(matrix1), colnames(matrix2)))
#             TE1        TE2          TE3
#gene1 -0.8757869 -0.4755768 -0.008312574
#gene2 -0.3567850 -0.7585136 -0.834883959
#gene3 -0.2723512 -0.3764091  0.546779587

matrix(unlist(tmp['p.value', ]), nrow=ncol(matrix1),
       dimnames=list(colnames(matrix1), colnames(matrix2)))
#             TE1       TE2        TE3
#gene1 0.05156122 0.4181472 0.98941622
#gene2 0.55555798 0.1371765 0.07851595
#gene3 0.65756758 0.5323119 0.34025894

You can check that it's correct by comparing the output of cor(matrix1, matrix2) with the matrix of estimates, the matrices should be equal.

Compare only pairwise comparisons BETWEEN two matrices using rcorr() in R

1 Answers