Fit dataframe into linear regression sklearn

Question

I am making a project for a class, and i am trying to predict nfl socre games using linear regression and predict functions from sklearn, my problem comes when i want to fit the training data into de fit function, here is my code:

onehotdata_x1 = pd.get_dummies(goal_model_data,columns=['team','opponent'])

# Crea el object de regression linear
regr = linear_model.LinearRegression()

# Train the model using the training sets
regr.fit(onehotdata_x1[['home','team','opponent']], onehotdata_x1['goals'])

This is the structure of dataframe(goal_model_data):

team opponent  goals  home
 NE       KC     27     1
BUF      NYJ     21     1
CHI      ATL     17     1
CIN      BAL      0     1
CLE      PIT     18     1
DET      ARI     35     1
HOU      JAX      7     1
TEN      OAK     16     1

and this is the error that i get when i run the program:

Traceback (most recent call last):
  File "predictnflgames.py", line 76, in <module>
    regr.fit(onehotdata_x1[['home','team','opponent']], onehotdata_x1['goals'])
  File "C:\Python27\lib\site-packages\pandas\core\frame.py", line 2133, in __getitem__
    return self._getitem_array(key)
  File "C:\Python27\lib\site-packages\pandas\core\frame.py", line 2177, in _getitem_array
    indexer = self.loc._convert_to_indexer(key, axis=1)
  File "C:\Python27\lib\site-packages\pandas\core\indexing.py", line 1269, in _convert_to_indexer
    .format(mask=objarr[mask]))
KeyError: "['team' 'opponent'] not in index"

You are trying to access columns that do not exist after using pd. get_dummies. see my answer for more details — seralouk

seralouk seralouk · Accepted Answer · 2017-11-24T22:37:36

The problem is that after pd.get_dummies there are no team and opponent columns.

I use this data in txt format for my example: https://ufile.io/e2vtv (same as yours).

Try this and see:

import pandas as pd
from sklearn.linear_model import LinearRegression

goal_model_data = pd.read_table('goal_model_data.txt', delim_whitespace=True)

onehotdata_x1 = pd.get_dummies(goal_model_data,columns=['team','opponent'])

regr = LinearRegression()

#see the columns in onehotdata_x1
onehotdata_x1.columns

#see the data (only 2 rows of the data for the example)
onehotdata_x1.head(2)

Results:

Index([u'goals', u'home', u'team_BUF', u'team_CHI', u'team_CIN', u'team_CLE',
       u'team_DET', u'team_HOU', u'team_NE', u'team_TEN', u'opponent_ARI',
       u'opponent_ATL', u'opponent_BAL', u'opponent_JAX', u'opponent_KC',
       u'opponent_NYJ', u'opponent_OAK', u'opponent_PIT'],
       dtype='object')

goals  home  team_BUF  team_CHI  team_CIN  team_CLE  team_DET  team_HOU  \
0     27     1         0         0         0         0         0         0
1     21     1         1         0         0         0         0         0

team_NE  team_TEN  opponent_ARI  opponent_ATL  opponent_BAL  opponent_JAX  \
0        1         0             0             0             0             0
1        0         0             0             0             0             0

opponent_KC  opponent_NYJ  opponent_OAK  opponent_PIT
0            1             0             0             0
1            0             1             0             0

EDIT 1

Based on the original code, you might want to do something like the following:

import pandas as pd
from sklearn.linear_model import LinearRegression

data = pd.read_table('data.txt', delim_whitespace=True)

onehotdata = pd.get_dummies(data,columns=['team','opponent'])

regr = LinearRegression()

#in x get all columns except goals column
x = onehotdata.loc[:, onehotdata.columns != 'goals']

#use goals column as target variable
y= onehotdata['goals']

regr.fit(x,y)
regr.predict(x)

Hope this helps.

Fit dataframe into linear regression sklearn

2 Answers