Crossfilter showing negative numbers on dc.js with no negative numbers in the dataset

Question

I am using CrossFilter along with dc.js to create 4 different bar charts and allow a user to change the data by using the brush feature on the charts, so when the user changes the brush on one chart the others dynamically changes.

This is all working for me at the minute bar one interesting issue, it looks like CrossFilter or dc.js are putting negative values on the graphs but only when certain sections of a graph is selected.

So as you can see from the image when I select an area of a chart that seems to have no values this shows up negative values in the other charts.

enter image description here

I have four items in my data, date, a type (string), value (number) and grouped value (this is the value grouped into smaller chunks of values of 50)

And then I have 4 dimensions on each piece of data and 4 groups and these are supplied to the charts.

There is never any negative values in my data so how can my charts be showing negative values?

I am new to CrossFilter and dc.js so I'm not sure of the best approach or the best snippets of code to show here, if there is something else I should share please let me know, I really need help trying to understand why there are negative numbers in my graphs.

EDIT: Adding code

Here is an example of my data:

[{"Id":"1","InDate":"31/10/2015","Type":"New","Category":"cat1","Value":"1.400874145"}, {"Id":"2","InDate":"21/10/2014","Type":"Old","Category":"cat2","Value":"0"}]

I read it in using the d3.csv function from a CSV file. I have triple checked for negative values in the CSV file.

Here is how I set up the dimensions:

var ndx = crossfilter(data);

var parseDate = d3.time.format("%d/%m/%Y").parse;

data.forEach(function(d) {
    d.date = parseDate(d.InDate);
    d.valueGrouped = createValueGrouping(parseFloat(d.Value));
});

var dateDim = ndx.dimension(function(d) {return d.date;});
var typeDim = ndx.dimension(function(d) {return d.Type;});
var valueGroupDim = ndx.dimension(function(d) {return d.valueGrouped;});
var categoryDim = ndx.dimension(function(d) {return d.Category;});

Here is the function that creates the valueGrouped attribute:

function createValueGrouping(value){
    return 50 * Math.round(value/50);
}

Finally here is how I set up the groups:

var timelineGroup = dateDim.group().reduceSum(function(d) {return parseFloat(d.Value);});
var typeGroup = typeDim.group().reduceSum(function(d) {return parseFloat(d.Value);});
var valueGrouped = valueGroupDim.group().reduceSum(function(d) {return parseFloat(d.Value);});
var categoryGroup = categoryDim.group().reduceSum(function(d) {return parseFloat(d.Value);});

EDIT 2: Adding JSFiddle

Fiddle - JSFiddle

You probably have a bug in your Crossfilter grouping code, but you'll need to share your code for us to help you. — Ethan Jewett
@EthanJewett - thanks, I have added my code showing a snippet of the data, how I set up the dimensions and then the groups, I then use these dimensions and groups in the most basic manner with dc.js but if you need to see some chart code let me know and I will add it. — Donal Rafferty
Sorry I didn't get back to it sooner. I'm glad it was resolved! @Gordon is the go-to person for this stuff. — Ethan Jewett

Gordon Gordon · Accepted Answer · 2015-03-24T17:11:25

It looks like these are infinitesimal negative numbers, which would probably indicate a glitch caused by floating point numbers not canceling out perfectly.

When you are adding floating point numbers with greatly varying magnitude, the operations are not necessarily associative or distributive, which means that if you add numbers in a different order you will get different results.

https://en.m.wikipedia.org/wiki/Floating_point#Accuracy_problems

Since crossfilter is liable to subtract out values in a different order from how they were added, and the order of addition is not necessarily defined either, this kind of problem often arises.

I would suggest creating a fake group wrapping your data and setting the values to zero when they are very close:

function snap_to_zero(source_group) {
    return {
        all:function () {
            return source_group.all().map(function(d) {
                return {key: d.key, 
                        value: (Math.abs(d.value)<1e-6) ? 0 : d.value};
            });
        }
    };
}

Wrap your original group with this function wherever you're seeing the problem:

chart.group(snap_to_zero(valueGrouped))

Crossfilter showing negative numbers on dc.js with no negative numbers in the dataset

1 Answers