Efficient data storage, partitioning, and selection in d3

Question

I'm having trouble seeing the big picture for some potentially massive refactoring. I'm looking for a solution (if one exists) that solves two problems simultaneously. I haven't coded in d3 for a long time and was never fluent to begin with.

I've created a simple demo of MCMC. It looks like this: demo pic

All the data exist in an array thetas that has dimensions nChains x nIters, or 2 x ... (with the chain length increasing with each mouse click).

Now I want to show a burn-in, which will be the same length for each chain. For the left plot, the burn-in will show, in different colors (e.g., orange and red) for each chain, the first biLength points. Effectively, the first biLength points that are already there will just be colored differently. On the histogram on the right, I want those points to have their own corresponding bars.

Should I create a separate array to hold the burn-in, effectively plotting two containers of data on each plot? (Is there an example of how to do this?) Alternatively, is there some way to keep the data in thetas but handle subcomponents differently? (Example?)

Here's some sample code to show how I'm currently plotting thetas:

// Left plot update
function movePoints ()
    { 
    var tmp = chart.selectAll(".vis1points").data(thetas[0], function (d, i) { return i; } );
    tmp
        .enter().insert("svg:circle")
        .attr("class", "vis1points")
        .attr("cx", function (d, i) { return xx(i); })
        .attr("cy", yy)
        .attr("r", 3)
    tmp.transition()
        .duration(10)
        .attr("cx", function (d, i) { return xx(i); })
        .attr("cy", yy)
    tmp.exit()
        .transition().duration(10).attr("r",0).remove();

    if ( nChains == 2 ) {
        var tmp2 = chart.selectAll(".vis1points2").data(thetas[1], function (d, i) { return i; } );
        tmp2
        .enter().insert("svg:circle")
        .attr("class", "vis1points2")
        .attr("cx", function (d, i) { return xx(i); })
        .attr("cy", yy)
        .attr("r", 3)
        tmp2.transition()
        .duration(10)
        .attr("cx", function (d, i) { return xx(i); })
        .attr("cy", yy)
        tmp2.exit()
        .transition().duration(10).attr("r",0).remove();
    }
    };

The axes currently take thetas as input to infer the range and domain limits.

For the histogram on the right:

 // Creation involves... 
    var dataLength = thetas[0].length;
    this.histBars = this.vis.selectAll("rect.vis2bars")
        .data(that.histVals.counts[0])
        .enter().append("svg:rect")
        .attr("class", "vis2bars")
        .attr("x", function (d, i) { return that.xx2(i); }) 
        .attr("y", function (d, i) { return that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[0])/dataLength)); })    
        .attr("height",function (d, i) { return that.yy2(0)-that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[0])/dataLength)); })
        .attr("width", Math.floor(innerWidth2/numBins));
    this.histBars.transition()
        .attr("y", function (d, i) { return that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[0])/dataLength)); })    
        .attr("height",function (d, i) { return that.yy2(0)-that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[0])/dataLength)); })
        .duration(dur);

And the update:

    this.update = function( dur2 ) {
    var dataLength = thetas[0].length;
    var tmp2 = this.histBars.data(this.histVals.counts[0]); 
    var tmp3;
    if ( nChains > 1 ) { // fix
        tmp3 = this.histBars2.data(this.histVals.counts[1]);
    }
    var that = this;
    tmp2
        .transition()
        .delay( 0 ) // could tweak...
        .duration(dur2 - 10)
        .attr("y", function (d, i) { return that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[0])/dataLength)); })    
        .attr("height",function (d, i) { return that.yy2(0)-that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[0])/dataLength)); }); 

    // Add update for second data set
    if ( nChains > 1 ) {
        tmp3
        .transition()
        .attr("y", function (d, i) { return that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[1])/dataLength)); })    
        .attr("height",function (d, i) { return that.yy2(0)-that.yy2(that.maxDensity*(d/dataLength)/(d3.max(that.histVals.counts[1])/dataLength)); }); 
    }

Barnabé Monnot Barnabé Monnot · Accepted Answer · 2013-09-03T09:58:51

You can dynamically change the color of your circles when updating them, for example inside here :

tmp.transition()
    .duration(10)
    .attr("cx", function (d, i) { return xx(i); })
    .attr("cy", yy)
    .attr("fill", function(d) {
         if (d.isBiLengthPoint) {
              return "orange";
         } else {
              return "blue";
         }
     });

I assume that for the moment you are coloring your circles based on your CSS, but for the effect you want I really think that loading them dynamically is a better solution.

Or you could change the class of your circles dynamically and have two CSS classes declaration

tmp.transition()
    .attr("class", function(d) {
         if (d.isBiLengthPoint) {
              return "vis1points-bilength";
         } else {
              return "vis1points";
         }
     })
    .duration(10)
    .attr("cx", function (d, i) { return xx(i); })
    .attr("cy", yy)

and in your CSS

 vis1points {
     fill: blue;
 }
 vis1points-bilength {
     fill: orange;
 }

Efficient data storage, partitioning, and selection in d3

1 Answers