Feedback on practical limitations for JSON?

Question

I'm at a critical decision point in my application design. It's an ASP.NET web application that uses REST to request information about various products. Some products have varying ProductIDs depending on their attributes/variations. For example, a user might be interested in Rogaine. A REST request about Rogain might return a response containing several variations - a 1 month supply, a 3 month supply or a 4 month supply. Each of these variations have a different ProductID, a different regular price, different sale price and a multitude of different attributes. Each one has a different set of 'features', different images, etc. The amount of attributes for each product can be quite a lot.

In most instances there are only a couple of product variations. Sometimes a half dozen, sometimes a dozen or two. In these instances, JSON can handle my requirements easily.

But take, for example, a dress. It might be available in a couple of dozen colors. It's also available in 6 different sizes. You wouldn't want to display each of these on their own page. It's much more user friendly to display this product on a single page and present these size and color options for the user to select. Perhaps they want to order a small size in a red color.

In the example above, where there are almost 100 combinations of options. At this point, I think JSON becomes an impractical choice for me. For each unique combination, there are several links to images (each image displays the product in a different color). Each product has a list price, a regular price and a sale price, amount saved. Each has shipping attributes. Each has its own list of features which can be paragraphs of text for each product.

Ok, my point is made - this is a lot of data to be stuffing in a JSON string and there will be a delay to lookup details when a color or size is changed.

I like the way Amazon presents choices. Here is a link of such a situation:

http://www.amazon.com/American-Apparel-Jersey-Chemise-Small-Navy/dp/B003ILSHQ2/

I've looked in the source page and don't see where they are storing each product's details on the client side. If you hover each color swatch you'll see that all the details change on the page. Price, shipping, large image, features. Virtually everything changes.

I see the "Loading..." indicators once in a while, but in most cases it's so fast that you don't see any signs of client/server communication.

How are they doing this without reloading the page entirely and without storing this information on the client? What technology are they using, does anyone know?

Trust me, a round trip to send another REST request each time a product variation is changed would be way too expensive. And I already have ALL of the data for every product in my first REST response. The data is not on my server or in my control and I really don't want to save it on my server.

Since I already have all of the data, I'd like to store it so it can be used like Amazon does, but I'd like to hand it off to the client. It would be perfect to use a JSON string and in most cases, I can... but not effectively/efficiently in all.

Is there a way to index a JSON string to make it quicker for large amounts of data? Is it just too much for JSON? What other options might I have to use with JQuery/Javascript?

sounds more like a problem of how to organize data than how much data. Example on amazon stores data in js variables. Look at colorImages object as example — charlietfl
I considered that too. Breaking the data into objects. Yes, you will definitely see where they stored some of the data in variables, but there are 77 unique combinations of that product (5 sizes and 21 colors). I do not find any object where there are 77 different product IDs. I don't find any object with a bunch of prices or a bunch of shipping options. Images, yes. But alot of it isn't in the client page - from what I can see. I've spent hours looking at it and playing around. — rwkiii
there are a lot of ways to handle it, very difficult to asses without seeing data and how UI needs to work — charlietfl
The link demonstrates how I'd like the UI to work. Really, pretty much just like that. As for seeing the data, well, the link above is a page created from as complex XML file that's about 100k in size. I think your initial comment has merit and it has me more seriously considering multiple objects. The size/color options (attributes) are what determine the product ID. Once the product ID is determined it could be used as a key for each object - pricing, shipping, images, etc. I kinda like that and think it would work a lot quicker than putting all data in one object. — rwkiii
you are completely over reacting to size of json, your issue has more to do with how to structure your data — charlietfl

Jasper Mogg Jasper Mogg · Accepted Answer · 2012-06-23T00:26:05

I'm currently writing an app where I get a load of JSON from various sources and then store them in localStorage. They're a set of Facebook, twitter and rss feeds. I build an index of the feeds by a unique ID assigned by the application and store that as an array, with each ID associated with a string denoting type i.e. FBK/TWI/RSS and a string with the title of the feed.

I then store an array of arrays for each feed containing data on the posts fetched from them.

With 5Mb to play with, this is a feasible setup. If your data you need to store doesn't exceed that, which I can imagine it doesn't, it would work for you too. On change of page you can clear the storage and download your next product object.

It's hard to imagine a 50kB object exceeding the combined power of JSON + localStorage :-) Though I'm sure I'm about to be told why I'm wrong haha!

Feedback on practical limitations for JSON?

2 Answers