Python Typeerror: All intermediate steps should be transformers and implement fit and transform

Question

I am currently working through the book "Hands-On Machine Learning with Scikit-Learn and TensorFlow" by Aurélien Géron. When I run the following code (which I copied) I get an error message. The error message seems quite clear, but I still don't understand it to be honest. Clearly, I lack understanding, but even after reviewing considerably, I could not locate the issue. Could anyone kindly help out?

from sklearn.base import BaseEstimator, TransformerMixin 
rooms_ix, bedrooms_ix, population_ix, households_ix = 3, 4, 5, 6 

class CombinedAttributesAdder(BaseEstimator, TransformerMixin): 
    def __init__( self, add_bedrooms_per_room = True): # no *args or ** kargs 
        self.add_bedrooms_per_room = add_bedrooms_per_room 
        def fit(self, X, y = None): 
            return self # nothing else to do 
        def transform(self, X):
            rooms_per_household = X[:, rooms_ix] / X[:, households_ix] 
            population_per_household = X[:, population_ix] / X[:, households_ix] 
            if self.add_bedrooms_per_room: 
                bedrooms_per_room = X[:, bedrooms_ix] / X[:, rooms_ix] 
                return np.c_[X, rooms_per_household, population_per_household, bedrooms_per_room] 
            else: 
                return np.c_[X, rooms_per_household, population_per_household] 
            attr_adder = CombinedAttributesAdder(add_bedrooms_per_room = False) 
            housing_extra_attribs = attr_adder.transform(housing.values)

from sklearn.pipeline import Pipeline 
from sklearn.preprocessing import StandardScaler 
num_pipeline = Pipeline([('imputer', SimpleImputer(strategy ="median")), ('attribs_adder', CombinedAttributesAdder()), ('std_scaler', StandardScaler()),])
housing_num_tr = num_pipeline.fit_transform(housing_num)

The error message:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) in 20 from sklearn.pipeline import Pipeline 21 from sklearn.preprocessing import StandardScaler ---> 22 num_pipeline = Pipeline([('imputer', SimpleImputer(strategy ="median")), ('attribs_adder', CombinedAttributesAdder()), ('std_scaler', StandardScaler()),]) 23 housing_num_tr = num_pipeline.fit_transform(housing_num)

~\Miniconda3\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs) 70 FutureWarning) 71 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)}) ---> 72 return f(**kwargs) 73 return inner_f 74

~\Miniconda3\lib\site-packages\sklearn\pipeline.py in init(self, steps, memory, verbose) 112 self.memory = memory 113 self.verbose = verbose --> 114 self._validate_steps() 115 116 def get_params(self, deep=True):

~\Miniconda3\lib\site-packages\sklearn\pipeline.py in _validate_steps(self) 157 if (not (hasattr(t, "fit") or hasattr(t, "fit_transform")) or not 158 hasattr(t, "transform")): --> 159 raise TypeError("All intermediate steps should be " 160 "transformers and implement fit and transform " 161 "or be the string 'passthrough' "

TypeError: All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'CombinedAttributesAdder()' (type <class 'main.CombinedAttributesAdder'>) doesn't

Many thanks in advance!

amiola amiola · Accepted Answer · 2020-10-15T21:50:09

For me the problem could be in the wrong indentation you have (unless it is just a mistyping). Methods fit() and transform() are not correctly indented (and so are attr_adder and housing_extra_attribs assignments). This way the instance of the class CombinedAttributesAdder that you are using in the pipeline is not a transformer, which is causing the error.

Python Typeerror: All intermediate steps should be transformers and implement fit and transform

1 Answers