Handling two different data with Crossfilter and make chart

Question

I have two data, one with time dependent and one not.

The data which is time dependent would be like below.

year,  name    , value
2001, "AAAAAAA", 200
2002, "AAAAAAA", 300
2003, "AAAAAAA", 400
2001, "BBBBBBB", 150
2002, "BBBBBBB", 250
2000, "CCCCCCC", 500
2001, "CCCCCCC", 600
2002, "CCCCCCC", 550
2001, "DDDDDDD", 100
2002, "DDDDDDD", 300

On the other hand, I have data independent to time which describes property of names.

 name      ,  type
"AAAAAAAA", "red"
"BBBBBBBB", "blue"
"CCCCCCCC", "green"
"DDDDDDDD", "red"

One way to handle data would be to hold data as following.

year,  name    , value,  type
2001, "AAAAAAA", 200  , "red"
2002, "AAAAAAA", 300  , "red"
2003, "AAAAAAA", 400  , "red"
2001, "BBBBBBB", 150  , "blue"
2002, "BBBBBBB", 250  , "blue"
2000, "CCCCCCC", 500  , "green"
2001, "CCCCCCC", 600  , "green"
2002, "CCCCCCC", 550  , "green"
2001, "DDDDDDD", 100  , "red"
2002, "DDDDDDD", 300  , "red"

If you want to handle data on crossfilter, is the above data format the best way to handle it? Or could you handle it separately which is more data efficient?

Related to the data format, how could I make chart with type in the X-axis and average of the names value's average in the Y-axis. Just using type as a dimension and calculating average would end up in averaging all result at once. I assume it would be normal to calculate average first with names then average again afterward.

With Crossfilter you will usually want to denormalize your tables, so the suggestion you make about how to organize your data in a single table looks correct to me. — Ethan Jewett
OK. Suppose I have some png in base64 format for each data (such as logos for company), and I want to make them popup by hovering on data point, should I make it a large data with base64 overall or would there be any workaround? — adamist521
I'd just store a reference to the PNG in your Crossfilter and then look up the PNG based on that reference when you want to display it. Putting big chunks of text into Crossfilter is not what it was designed for. — Ethan Jewett

Gordon Gordon · Accepted Answer · 2016-05-26T17:57:43

Regarding the average-of-averages, I'd suggest something like this (untested).

Basically, keep a running sum and count for each name, and then when you need the average-of-averages, calculate all the averages and sum them, and divide by the number of names.

group.reduce(function(p, v) { // add
    p.sum[v.name] = (p.sum[v.name] || 0) + v.value;
    p.count[v.name] = (p.count[v.name] || 0) + 1;
    return p;
}, function(p, v) { // remove
    p.sum[v.name] -= v.value;
    p.count[v.name]--;
    return p;
} function() { // init
    return {sum: {}, count: {}};
});

chart.valueAccessor(function(kv) {
    var names = Object.keys(kv.value.sum),
        sumavg = 0;
    names.forEach(function(name) {
        sumavg += kv.value.sum[name] / kv.value.count[name];
    });
    return sumavg / names.length;
});

Handling two different data with Crossfilter and make chart

1 Answers