Sklearn-Pandas DataFrameMapper: mapper.fit_transform gives ValueError: bad input shape (8, 2)

Question

I was able to replicate the example given in the Github repo. However, when I tried it on my own data, I got the ValueError.

Below is a dummy data that, which gives the same error as my real data.

import pandas as pd
import numpy as np
from sklearn_pandas import DataFrameMapper
from sklearn.preprocessing import LabelEncoder, StandardScaler, MinMaxScaler

data = pd.DataFrame({'pet':['cat', 'dog', 'dog', 'fish', 'cat', 'dog','cat','fish'], 'children': [4., 6, 3, 3, 2, 3, 5, 4], 'salary':   [90, 24, 44, 27, 32, 59, 36, 27], 'feat4': ['linear', 'circle', 'linear', 'linear', 'linear', 'circle', 'circle', 'linear']})

mapper = DataFrameMapper([
    (['pet', 'feat4'], LabelEncoder()),
    (['children', 'salary'], [StandardScaler(),
                              MinMaxScaler()])
]) 

np.round(mapper.fit_transform(data.copy()),2)

Below is the error

ValueError Traceback (most recent call last) in () ----> 1 np.round(mapper.fit_transform(data.copy()),2)

C:\Users\E245713\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\base.py in fit_transform(self, X, y, **fit_params) 453 if y is None: 454 # fit method of arity 1 (unsupervised transformation) --> 455 return self.fit(X, **fit_params).transform(X) 456 else: 457 # fit method of arity 2 (supervised transformation)

C:\Users\E245713\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn_pandas\dataframe_mapper.py in fit(self, X, y) 95 for columns, transformers in self.features: 96 if transformers is not None: ---> 97 transformers.fit(self._get_col_subset(X, columns)) 98 return self 99

C:\Users\E245713\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\preprocessing\label.py in fit(self, y) 106 self : returns an instance of self. 107 """ --> 108 y = column_or_1d(y, warn=True) 109 _check_numpy_unicode_bug(y) 110 self.classes_ = np.unique(y)

C:\Users\E245713\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\utils\validation.py in column_or_1d(y, warn) 549 return np.ravel(y) 550 --> 551 raise ValueError("bad input shape {0}".format(shape)) 552 553

ValueError: bad input shape (8, 2)

Can anyone help?

thanks

jeff carey jeff carey · Accepted Answer · 2016-07-27T21:08:48

You should only submit multiple arrays to a transform if it indeed takes multiple inputs (e.g. sklearn.decomposition.PCA(1) in the documentation). In your case the error ultimately comes from this line:

(['pet', 'feat4'], LabelEncoder()),

Even this does not work:

(['pet', 'feat4'], [LabelEncoder(), LabelEncoder()]),

You instead have to do something like this:

mapper_good = DataFrameMapper([
(['pet'], LabelEncoder()),
(['feat4'], LabelEncoder()),
(['children'],  StandardScaler()),
(['salary'],    MinMaxScaler())
])

np.round(mapper_good.fit_transform(data.copy()),2)

Sklearn-Pandas DataFrameMapper: mapper.fit_transform gives ValueError: bad input shape (8, 2)

1 Answers