Received “ValueError: If using all scalar values, you must pass an index” in Python

Question

I have run the following code on python in order to retrieve various crypto currency closing prices from their inception. I have run it successfully using the following tickers:

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH',
 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

I now have changed it as follows (full code included) and get a ValueError.

[LN1]

 def CryptoDataCSV(symbol, frequency):

        #Params: String symbol, int frequency = 300,900,1800,7200,14400,86400

        #Returns: df from first available date

        url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

        df = pd.read_json(url)

        df.set_index('date',inplace=True)

        df.to_csv(symbol + '.csv')

        print('Processed: ' + symbol)

[LN2]

import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

[LN3]

    tickers = 'ETH_BTC','STR_BTC','XMR_BTC','XRP_BTC','LTC_BTC','DASH_BTC',
'ETC_BTC','POT_BTC','OMG_BTC','FCT_BTC','ZEC_BTC','BTS_BTC','VTC_BTC',
'XEM_BTC','MAID_BTC','DGB_BTC','STRAT_BTC','LSK_BTC','XVC_BTC','SC_BTC',
'DOGE_BTC','XBC_BTC','GNT_BTC','EMC2_BTC','CLAM_BTC','RIC_BTC','SYS_BTC',
'DCR_BTC','STEEM_BTC','ZRX_BTC','GAME_BTC','VIA_BTC','NXC_BTC','NXT_BTC'
,'VRC_BTC','NAV_BTC','PINK_BTC','STORJ_BTC','ARDR_BTC','BCN_BTC','CVC_BTC',
'EXP_BTC','LBC_BTC','GNO_BTC','GAS_BTC','OMNI_BTC','XCP_BTC','NEOS_BTC',
'BURST_BTC','AMP_BTC','FLDC_BTC','FLO_BTC','SBD_BTC','BLK_BTC','BTCD_BTC',
'NOTE_BTC','GRC_BTC','PPC_BTC','BTM_BTC','XPM_BTC','NMC_BTC','PASC_BTC',
'NAUT_BTC','BELA_BTC','SJCX_BTC','HUC_BTC','RADS_BTC']

[LN4]

for ticker in tickers:
        CryptoDataCSV(ticker, 86400)

I now get the following error:

--------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () 1 for ticker in tickers: ----> 2 CryptoDataCSV(ticker, 86400)

in CryptoDataCSV(symbol, frequency) 7 url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0' 8 ----> 9 df = pd.read_json(url) 10 11 df.set_index('date',inplace=True)

~\Anaconda3\lib\site-packages\pandas\io\json\json.py in read_json(path_or_buf, orient, typ, dtype, convert_axes, convert_dates, keep_default_dates, numpy, precise_float, date_unit, encoding, lines) 352 obj = FrameParser(json, orient, dtype, convert_axes, convert_dates, 353 keep_default_dates, numpy, precise_float, --> 354 date_unit).parse() 355 356 if typ == 'series' or obj is None:

~\Anaconda3\lib\site-packages\pandas\io\json\json.py in parse(self) 420 421 else: --> 422 self._parse_no_numpy() 423 424 if self.obj is None:

~\Anaconda3\lib\site-packages\pandas\io\json\json.py in _parse_no_numpy(self) 637 if orient == "columns": 638 self.obj = DataFrame( --> 639 loads(json, precise_float=self.precise_float), dtype=None) 640 elif orient == "split": 641 decoded = dict((str(k), v)

~\Anaconda3\lib\site-packages\pandas\core\frame.py in init(self, data, index, columns, dtype, copy) 273 dtype=dtype, copy=copy) 274 elif isinstance(data, dict): --> 275 mgr = self._init_dict(data, index, columns, dtype=dtype) 276 elif isinstance(data, ma.MaskedArray): 277 import numpy.ma.mrecords as mrecords

~\Anaconda3\lib\site-packages\pandas\core\frame.py in _init_dict(self, data, index, columns, dtype) 409 arrays = [data[k] for k in keys] 410 --> 411 return _arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype) 412 413 def _init_ndarray(self, values, index, columns, dtype=None, copy=False):

~\Anaconda3\lib\site-packages\pandas\core\frame.py in _arrays_to_mgr(arrays, arr_names, index, columns, dtype) 5494 # figure out the index, if necessary 5495 if index is None: -> 5496 index = extract_index(arrays) 5497 else: 5498 index = _ensure_index(index)

~\Anaconda3\lib\site-packages\pandas\core\frame.py in extract_index(data) 5533 5534 if not indexes and not raw_lengths: -> 5535 raise ValueError('If using all scalar values, you must pass' 5536 ' an index') 5537

ValueError: If using all scalar values, you must pass an index

I don't know if read_json understands URLs, but when I tried getting results through requests, I got "invalid currency pair". — cs95
@COLDSPEED did it indicate which currency pair is erring? i have run it with the shorter list and it works fine. any suggestions of a rewrite would be appreciated. — Mike O
Let me know if my answer helped. On trying with your smaller list, I was able to get results. — cs95

cs95 cs95 · Accepted Answer · 2017-11-07T17:45:22

I just tested your data, and it appears that some of your currency pairs do not work at all, returning a json of the form:

{"error":"Invalid currency pair."}

When this is returned, pd.read_json throws an error, because it can't convert this to a dataframe.

The simplest workaround is to use a try-except brace and handle any non-working tickers.

broken_tickers = []

for t in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair={}&end=9999999999&period={}&start=0'.format(t, 86400)

    try:
        df = pd.read_json(url)
    except ValueError:
        broken_tickers.append(t)
        continue

    df.set_index('date')
    df.to_csv('{}.csv'.format(t))

I've gotten rid of the function, I didn't really feel it necessary here but you can add it back in.

Received “ValueError: If using all scalar values, you must pass an index” in Python

1 Answers