dc.js crossfilter without reduce

Question

Is crossfilter manipulating my data?

Background

I have performed all the processing I need server side and just want to graph exactly what comes down the json pipe. So far I've get the graph working exactly how I want it to except for it seems my data is being manipulated.

Here's my crossfilter code:

ndx = crossfilter(rData);
runDimension = ndx.dimension(function (d) { return [+d.series, +d.norm_1]; });
runGroup = runDimension.group();
runGroup.reduceSum(function (d) { return d.value;});

Note: norm_1 is unique

Issues

Basically I'm noticing two issues:

I know for a fact that all my data will be between -1 and 1 (I've run several checks to test this), BUT when graphing it I see it dips down to -1.4 in some places.
My server sends exactly 1000 rows of data, but by breakpointing some of the dc.js code I can see it's only graphing 752 rows.

More Evidence

On my chart I've set the valueAccessor and added some checks to test the values going out of bounds, and I can see very clearly it goes out:

        .valueAccessor(function (d) {
            if (d.value > 1 || d.value < -1) {
                console.log(d);
            }
            return d.value;
        })

The data from the server requires a small amount formatting before going into crossfilter (it comes down as a table and needs to be split into series objects). I used this as an opportunity to test whether the data goes out of bounds, and I can see clearly it stays within bounds:

    for (var i = 0; i < $scope.remoteData.rows.length; i++) {
        for (var j = 0; j < $scope.remoteData.labels.length; j++) {
            var label = $scope.remoteData.labels[j];
            var value = $scope.remoteData.rows[i][label];
            if (value > 1 || value < -1) {
                console.log({
                    label: label,
                    i: i,
                    series: j,
                    norm_1: $scope.remoteData.rows[i].norm_1,
                    value: value,
                });
            }
            rData.push({
                series: j,
                norm_1: $scope.remoteData.rows[i].norm_1,
                value: value
            })
        }
    }

Discussion

I suspect my problems have something to do with:

runGroup.reduceSum(function (d) { return d.value;});

Is this function adding together certain data points?

Gordon Gordon · Accepted Answer · 2016-02-17T19:52:32

Sounds like you have some rows for which [+d.series, +d.norm_1] is not unique. And yes any rows with the same key will be added with reduceSum.

I'd suggest making your dimension key be something that's really unique.

If you don't have a unique key, with a little more work you could use the array indices themselves as the dimension key. It will mean you have to use both key and value accessors everywhere to look back in the original array.

Something like:

ndx = crossfilter(d3.range(0, rData.length));
runDimension = ndx.dimension(function(d) { return d; })
runGroup = runDimension.group().reduceSum(function(d) { 
    return rData[d].value;
})

chart.keyAccessor(function(kv) { return rData[kv.key].x; })
     .valueAccessor(function(kv) { return rData[kv.key].y; })

dc.js crossfilter without reduce

1 Answers