Replace crossfilter data, restore dimensions and groups

7
votes

I'm using dc.js to render a nice bubble chart of a dataset. Underlying dc.js is crossfilter.

I'd like to smoothly refresh my chart with new data from the server. This issue on Github makes it clear that it is possible to do this by:

  1. deleting all the data from the crossfilter
  2. adding in the new data
  3. calling dc.redrawAll().

I've got this working but in order to delete all the data, you first have to clear all the filters (because crossfilter.remove only removes the records matching the current filter).

I'd like to 'remember' how my data was filtered before, so I can reconstruct the filter again once I've replaced all the data. I'm willing to get into the guts of the crossfilter code, but any pointers would be helpful.

Additionally: if anyone knows a way of updating crossfilter data based on a unique key, that'd be gold dust!

2
javascriptdc.jscrossfilter
Are you only adding data? Then I'd say use crossfilter.add([data]). If you need to remove data as well, I don't have much to offer, but you might want to take a look at this issue and comment if it sounds like the right direction: github.com/square/crossfilter/issues/109 – Ethan Jewett
@EthanJewitt I'm not adding or removing data. I'm updating existing data, each row uniquely keyed. Or does add have the facility to update based on a key? – LondonRob
Gotcha - then you do need to remove the old data and add new data (no ability to update existing keys as far as I know). That capability doesn't really exist at the moment as you will have to remove all filters in order to remove your data. Personally, I think this is something that needs to be addressed in Crossfilter, and there is some discussion on that issue about how to address it, but I don't think anyone has had time to tackle it. – Ethan Jewett

2 Answers

1
votes

Thanks to @londonrob's Answer all data is now removed from the index. Here is a more functional approach for the resetting of data.

// reset the filter for a dimension
function resetDimensionFilter (dimension) {
  dimension.filter(null);
}

// reset filters for all given dimensions, 
// remove all data from index and
// return empty index
function resetData(ndx, dimensions) {
  // Clear all filters from dimensions, because `ndx.remove` 
  // only removes records matching the current filters.
  dimensions.forEach(resetDimensionFilter);

  // Remove all data from the cross filter
  ndx.remove();
}
3
votes

Here's what I ended up hacking together. It works perfectly well, although I'm sure it's ludicrously inefficient because all the dimensions have to be created from scratch:

var _xfilter = crossfilter({x:1, y:2},{x:3, y:4}),
    _dimensions = [];

_dimensions.push(_xfilter.dimension(function(d) { return d.x; });

// Unfilters all the given dimensions, removes all data
// from xf and adds newData to xf.
var _xfilter_reset = function(xf, dimensions, newData) {
    var i;
    for (i = 0; i < dimensions.length; i++) {
        // Clear all filters from this dimension.
        // Necessary because xf.remove only removes records
        // matching the current filter.
        dimensions[i].filter(null);
    }
    xf.remove(); // Remove all data from the crossfilter
    xf.add(newData);
    return xf;
}

// Resets the global crossfilter object and reapplies all
// current dc.js chart filters.
var _refresh_data = function(data) {
    var i, j,
        chart, oldFilters,
        allCharts = dc.chartRegistry.list();

    _xfilter = _xfilter_reset(_xfilter, _dimensions, data);     

    // Reset all filters using dc.js
    for (i = 0; i < allCharts.length; i++) {
        chart = allCharts[i];
        oldFilters = chart.filters(); // Get current filters
        chart.filter(null); // Reset all filters on current chart
        for (j = 0; j < oldFilters.length; j++) {
            // Set all the oldFilters back onto the chart
            chart.filter(oldFilters[j]);
        }
    }
    dc.redrawAll();
}