Objects stored by riak-java-client end up as raw json when read by riak-python-client?

Question

I might be confused about something, but when I store a custom object from the Java Riak client and then try to read that object using the Python Riak client, I end up with a raw json string instead of a dict.

However, if I store a the object in python, I am able to output a python dictionary when fetching that object.

I could simply use a json library on the python side to resolve this, but the very fact that I am experiencing this discrepancy makes me think that I am doing something wrong.

On the Java side, this is my object:

class DocObject 
{
    public String status; // FEEDING | PERSISTED | FAILED | DELETING
    public List<String> messages = new ArrayList<String>();
}

class PdfObject extends DocObject
{
    public String url;
    public String base_url;
}

This is how I am storing that object in Riak:

public void feeding(IDocument doc) throws RiakRetryFailedException {
    PdfObject pdfObject = new PdfObject();

    pdfObject.url = doc.getElement("url").getValue().toString();
    pdfObject.base_url = doc.getElement("base_url").getValue().toString();
    pdfObject.status = "FEEDING";

    String key = hash(pdfObject.url);

    pdfBucket.store(key, pdfObject).execute();
}

And this is what I am doing in Python to fetch the data:

# Connect to Riak.
client = riak.RiakClient()

# Choose the bucket to store data in.
bucket = client.bucket('pdfBucket')

doc = bucket.get('7909aa2f84c9e0fded7d1c7bb2526f54')

doc_data = doc.get_data()

print type(doc_data)

The result of the above python is:

<type 'str'>

I am expecting that to be <type 'dict'>, just like how the example here works: http://basho.github.com/riak-python-client/tutorial.html#getting-single-values-out

I am perplexed as to why when the object is stored from Java it is stored as a JSON string and not as an object.

I would appreciate if anybody could point out an issue with my approach that might be causing this discrepancy.

Thanks!

If you retrieve and compare two objects created through the Python and Java clients, are there any differences in metadata, e.g. Content-Type? — Christian Dahlqvist
As noted in the answer below, yes there was a difference between the Content-type, where the python client produced "application/json" whereas the Java client produced "application/json; charset=UTF-8". According to the answer and the issue reported on github, that is the problem. Spot on suggestion. — chaimp

Brian Roach Brian Roach · Accepted Answer · 2013-03-19T15:08:42

It would appear you've found a bug in our Python client with the HTTP protocol/transport.

Both the version you're using and the current one in master are not decoding JSON properly. Myself and another dev looked into this this morning and it appears to stem from an issue with charset parameter being returned from Riak with the content-type as Christian noted in his comment ("application/json; charset=UTF-8")

We've opened an issue on github (https://github.com/basho/riak-python-client/issues/227) and will get this corrected.

In the mean time the only suggestion I have is to decode the returned JSON string yourself, or using the 1.5.2 client (latest stable from pypy) and the Protocol Buffers transport:

client = riak.RiakClient(port=8087, transport_class=riak.RiakPbcTransport)

it will return the decoded JSON as a dict as you're expecting.

Objects stored by riak-java-client end up as raw json when read by riak-python-client?

1 Answers