Python; SciKit Learn; SVM; Shape Mismatch

Question

I have time series of 40,000 assets. i've split the data into training and target data. The training data has 119 days of returns and the target data has 59 days. I split it up this way on purpose.

Train: (119 rows of returns, 40000 different series) Target: (59 rows of returns, same 40000 series)

I ran the following code to FIT the model:

SVR_model = svm.SVR(kernel='rbf',C=100,gamma=.001).fit(t_train_scale.transpose(), t_test.transpose())
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-185-2a0fd827e2a4> in <module>()
      1 
      2 
----> 3 SVR_model = svm.SVR(kernel='rbf',C=100,gamma=.001).fit(t_train_scale.transpose(), t_test.transpose())

C:\Users\nnayyar\Anaconda\lib\site-packages\sklearn\svm\base.pyc in fit(self, X, y, sample_weight)
    174 
    175         seed = rnd.randint(np.iinfo('i').max)
--> 176         fit(X, y, sample_weight, solver_type, kernel, random_seed=seed)
    177         # see comment on the other call to np.iinfo in this file
    178 

C:\Users\nnayyar\Anaconda\lib\site-packages\sklearn\svm\base.pyc in _dense_fit(self, X, y, sample_weight, solver_type, kernel, random_seed)
    229                 cache_size=self.cache_size, coef0=self.coef0,
    230                 gamma=self._gamma, epsilon=self.epsilon,
--> 231                 max_iter=self.max_iter, random_seed=random_seed)
    232 
    233         self._warn_from_fit_status()

C:\Users\nnayyar\Anaconda\lib\site-packages\sklearn\svm\libsvm.pyd in sklearn.svm.libsvm.fit (sklearn\svm\libsvm.c:1864)()

ValueError: Buffer has wrong number of dimensions (expected 1, got 2)

From research I see the most common answer using SVM is that the shapes have to 'match up' but how do I fit the SVM with data of various size?

Edit: Still need some help with this, how do I forecast thousands of predictions, not just the next 1?

What do you mean by training has 119 days return and target has 59 days of return? — WoodChopper
I have 119 days of return information in the training set and 59 days of returns in the target set. So Stock 1 of 40,000; Day 1 return X, Day 2 return Y....Make sense? — user2977664

Radu Gheorghiu Radu Gheorghiu · Accepted Answer · 2015-10-30T22:56:35

The problem is with your second parameter in the fit call.

As per documentation, the second parameter needs to be an array of n samples, where n is the number of instances, equal to the number of rows of the matrix that you pass as the first parameter (X).

fit(X, y, sample_weight=None)

X : {array-like, sparse matrix}, shape (n_samples, n_features)

y : array-like, shape (n_samples,)

So, if your first parameter has a size of 40.000 x 119 (since you transposed it), the second parameter needs to be an array of size 40.000 x 1.

But, judging by the error it is possible that your second parameter has more than 1 column (i.e. 2 columns).

Python; SciKit Learn; SVM; Shape Mismatch

1 Answers