Converting from dgCMatrix/dgRMatrix to scipy sparse matrix

Question

I am working on the netflix data set and attempting to use the nmslibR package to do some KNN type work on the sparse matrix that results from the netflix data set. This package only accepts scipy sparse matrices as inputs, so I need to convert my R sparse matrix to that format. When I attempt to do so, I am getting the following error. dfm2 is a 1.1 gb dgCMatrix, I have also attempted it on a dgRMatrix with the exact same error.

dfm3<-TO_scipy_sparse(dfm2)
Error in TO_scipy_sparse(dfm2) : attempt to apply non-function

I don't know how to provide a good sample dataset for my problem, the sparse matrix I'm working with is 1.1 gb, so if someone has a suggestion on how I can make it easier to help me please let me know. I would also be open to hearing other packages that will do KNN/KNN type functions in r for sparse matrices.

Edit:

I use the following code to generate a sample sparse matrix in the dgCMatrix format and attempt to convert it to a sci py sparse matrix and get the following error.

library(Matrix)
library(nmslibR)
sparse<-Matrix(sample(c(1,2,3,4,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0),10000,
               replace=T),
               ncol=50,
               byrow=T)
dfm3 <- TO_scipy_sparse(sparse)
Error in TO_scipy_sparse(sparse) : attempt to apply non-function

To answer a question about whether sparse is a dgCMatrix:

str(sparse)
Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:2414] 0 6 9 10 13 20 22 23 25 49 ...
  ..@ p       : int [1:51] 0 45 92 146 185 227 277 330 383 435 ...
  ..@ Dim     : int [1:2] 200 50
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x       : num [1:2414] 4 1 1 2 5 3 2 5 3 5 ...
  ..@ factors : list()

Here an indirect way using an intermediate file: stackoverflow.com/questions/31596850/… — Ralf Stubner
So that looks like it is focused on importing into Python (unless I'm missing something). I'm trying to get the data to work with a package in r that needs the matrix in a python sparse matrix format. — user2355903
Ok. Then it is probably best if you try to build a MWE using just a small sparse matrix that you can build by hand. I am sure the Matrix documentation contains examples for that. Or use one of the example matrices in that package, c.f. data(package = "Matrix"). — Ralf Stubner
Are you sure that sparse is a dgCMatrix? In the docs example, rdocumentation.org/packages/nmslibR/versions/1.0.2/topics/…, there's a spare=True parameter. — hpaulj

lampros lampros · Accepted Answer · 2018-07-02T18:09:37

The 'attempt to apply non-function' error is a known issue when something is wrong with the python configuration in the operating system. There are similar issues for other Python packages that I ported from Python to R. You can have a look here. You should also know that the nmslibR package uses the reticulate package for the interface between Python and R, so there must be similar issues too. If the error persists then you can open an issue in the nmslibR repository providing some sample data.

Converting from dgCMatrix/dgRMatrix to scipy sparse matrix

1 Answers