Sklearn: How to speed up Decision function

Question

I am implementing multilabel text classification by training 4709 separate binary logistic regression classifiers in Sklearn, using HashingVectorizer [(n_features=2**24,binary=True,ngram_range=(1,2)].

Accuracy is pretty good, but prediction latency is huge. Average sparsity ratio of learned matrices is 0.967, and shape of matrices are (1, 16777216). Using build in predict_proba function prediction time for one entry is 147.9 secs (on server with one Intel Xeon E5 2630v4). Most of the time (80%) is spent by scipy sparse csc_tocsr function.

When I pre-process matrices with:

cf[i] = sparse.csr_matrix(clf.coef_.T)

and infer probability (I do not need normalization, just order of probabilities) directly by

prob[i] = x*cf[i]

it takes only 0.043 sec to infer 407 (10%) classifiers but memory consumption is 25GB, so I would need about 250GB of RAM to keep all classifiers in memory.

Is there any way to speed up decision function while keeping matrices sparse, or some other way of pre-processing that would take less memory.

Answer 1

To answer my question, I've found a simple solution. All all coefficient matrices can be combined (stacked) into one, that is transposed and used for inference. So the loading/preparation code is:

coefs = sparse.vstack([c.coef_ for c in clfs])
coefs = coefs.T

where clfs are logistic regression classifiers (but it works for all linear classifiers). Inference code is simple

x = vectorizer.fit_transform([q])
r = x*coefs

Inference time is less then 1/10 seconds in my case of 4709 classifiers.