Apply the PCA reduction after choosing n components

Question

I have a data set and I want to reduce dimensionality with PCA method.
I import the data set, split it to train and validation sets and apply PCA to see what is the cumulative variance sum.

df = read_csv("train.csv")
X = df.drop(['label'], axis = 1)
y = df['label']
X_train, X_validation, y_train, y_validation = train_test_split(X, y, test_size = 0.2, random_state = 42)
model = PCA()
model.fit_transform(X_train_scaled)
variance = model.explained_variance_
cum_var = np.cumsum(variance)/np.sum(variance)

plt.figure(figsize = (15,15))
plt.bar(range(1,26), cum_var*100, alpha = 0.5, align = 'center', label = 'cummulative variance')
plt.legend()
plt.ylabel('Variance')
plt.xlabel('Principal components')

for x,y in zip(range(1,26),cum_var):
    label = "{:.2f}".format(y)
    plt.annotate(label, # this is the text
             (x,y), # this is the point to label
             textcoords = "offset points", # how to position the text
             xytext = (0,10), # distance from text to points (x,y)
             ha = 'center')
plt.show()

I choose n_components = 16.
How can I apply the n_components = 16 on X_train and X_validation?

yatu yatu · Accepted Answer · 2020-07-09T06:57:17

Once you know the appropriate number of components, you could just run PCA again, setting n_components=16, and just keep the new reduced features as your X_train:

model = PCA(n_components=16)
model.fit(X_train)
X_train = model.transform(X_train)
X_validation = model.transform(X_validation)

Apply the PCA reduction after choosing n components

1 Answers