attribute error in sklearn svm.SVC

Question

having issues with attribute errors when implementing a linear SVM with scikit-learn. I'm using a linear classifier with cross-validation through the RFECV method, and I can't access any of the attributes of the SVC. Not sure if it has to do with the feature selection or base model.

    model = svm.SVC(kernel='linear')
    selector=RFECV(model)
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=pct_test)
    selector=selector.fit(X_train, Y_train)
    my_prediction = selector.predict(X_test)
    f1.append(metrics.f1_score(Y_test, my_prediction))
    kappa.append(metrics.cohen_kappa_score(Y_test, my_prediction))
    precision.append(metrics.precision_score(Y_test, my_prediction))
    recall.append(metrics.recall_score(Y_test, my_prediction))
    print model.intercept_
    print model.support_vectors_
    print model.coef_

Metrics work fine, attributes all fail. The error message is:

AttributeError: 'SVC' object has no attribute 'intercept_'

Documentation: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC

Aside: I'm very new to OOP. If there's an underlying concept I'm missing, please elaborate or send over a link.

Vivek Kumar Vivek Kumar · Accepted Answer · 2017-03-22T05:39:23

You are fitting (training the data) on the RFECV object selector, but trying to access attributes of SVC object model. But it is not trained. Hence there is no attribute intercept_ in it.

To access the intercept of SVC, you should use:

selector.estimator_.intercept_

But understand that the above estimator is fitted only on the reduced dataset (After eliminating features as specified)

Explanation:

You see, RFECV internally uses RFE to get important features in each iteration. And RFE clones the supplied estimator for the purpose. So when you initialize RFECV with model, it is trained on the clone of the model. Checking the source code:

Line 407 (inside the fit method of RFECV):

rfe = RFE(estimator=self.estimator, 
          n_features_to_select=n_features_to_select, 
          step=self.step, verbose=self.verbose)

Line 428 (for estimating the scores):

scores = parallel(func(rfe, self.estimator, X, y, train, test, scorer)
                       for train, test in cv.split(X, y))

And then Line 165 (Inside fit method of RFE):

estimator = clone(self.estimator)

attribute error in sklearn svm.SVC

1 Answers