Feature Selection Scikit Learn

Question

After run feature selection in scikit-learn I would like to expose relevant variables, show me the variables that was select from method, how is it possible? The command X.shape just show the number of variables, I wanna see the name of variables after feature selection.

from sklearn.datasets import load_iris
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import chi2

iris = load_iris()

X, y = iris.data, iris.target

X.shape

X_new = SelectKBest(chi2, k=2).fit_transform(X, y)

X_new.shape

Answer 1

from sklearn.datasets import load_iris

from sklearn.feature_selection import SelectKBest

from sklearn.feature_selection import chi2

iris = load_iris()

X, y = iris.data, iris.target

X.shape

skb = SelectKBest(chi2, k=2)
skb.fit(X, y)
X_new = skb.transform(X)

X_new.shape

print skb.get_support(indices=True)

This will give you indices of the features selected.

Answer 2

You can get the names but you need to use pandas and convert the numpy to dataframes.

Example:

from sklearn.datasets import load_iris
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import chi2
import pandas as pd

iris = load_iris()
X = pd.DataFrame(iris.data, columns=iris.feature_names)
y = pd.DataFrame(iris.target)

selector = SelectKBest(chi2, k=2)
selector.fit(X, y)

X_new = selector.transform(X)
X_new.shape

#text format
X.columns[selector.get_support(indices=True)]
#vector format
vector_names = list(X.columns[selector.get_support(indices=True)])

print(vector_names)

#2nd way 
X.columns[selector.get_support(indices=True)].tolist()

Result:

Index([u'petal length (cm)', u'petal width (cm)'], dtype='object')
['petal length (cm)', 'petal width (cm)']
['petal length (cm)', 'petal width (cm)']

Answer 3

After feature selection, if you want to select only those features that were selected as significant("True") for building a newer model you could do the following:

feats = X.T.tolist()

optimised_feats = []
for i,j in zip(X_new.support_,feats):
    if i == True:
        optimised_feats.append(j)
optimised_feats=np.array(optimised_feats).T