sklearn LogisticRegression without regularization

Question

Logistic regression class in sklearn comes with L1 and L2 regularization. How can I turn off regularization to get the "raw" logistic fit such as in glmfit in Matlab? I think I can set C = large number but I don't think it is wise.

see for more details the documentation http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression

Yes, you can use l2 penalty and set the C parameter large. How beneficial is it not to penalize? If you do this with a completely separable dataset, then the weights will diverge. — eickenberg
its beneficial if you want the maximum likelihood solution without regularization, for example if you have a lot of data and you don't worry about over fitting. — Hanan Shteingart
Is there a well-tested Python package that does regular logistic regression? I feel like this is what @HananShteingart is looking for. — Dylan Daniels
I believe the statsmodels API does logistic regression without regularization - see stats.stackexchange.com/questions/203740/… — elz

Yu Chen Yu Chen · Accepted Answer · 2016-11-24T09:46:03

Yes, choose as large a number as possible. In regularization, the cost function includes a regularization expression, and keep in mind that the C parameter in sklearn regularization is the inverse of the regularization strength.

C in this case is 1/lambda, subject to the condition that C > 0.

Therefore, when C approaches infinity, then lambda approaches 0. When this happens, then the cost function becomes your standard error function, since the regularization expression becomes, for all intents and purposes, 0.

sklearn LogisticRegression without regularization

3 Answers