dc Charts not interacting with each other (with crossfilter on server side)

2018-06-11 09:35:54

I'm pretty new to crossfilter and dc, currently trying to build interactive dashboard with them. The data I have would easily go up to around 200MB and my browser would just crash when I do the crossfilter on the client side, so after some search online I decided to run crossfilter on the server and dc for charting on client side, and I'm following the work here dc.js-server-side-crossfilter However, the dc charts are not interacting with each other.

So from what I understand the basic components are, server-side app.js handles Ajax calls sent from the client-side app.js, and perform required filtering then send the dimension and groups in JSON format back to the client-side, then we will create fake dimensions and groups and feed them into dc chart, then each user clicks/drag on the chart will then again create another Ajax call, the whole process repeats again.

Here I post the important parts in somewhat pseudo-code as the original is messy to look at,

Server-side app.js

var app = express();

// define variables that will get passed
var dimensions = {};
var groups     = {};

// handle the ajax call coming from client-side
app.get("/refresh", function(req, res, next){
    var results = {};
    filter = req.query["filter"] ? JSON.parse(req.query["filter"]) : {} 
    for(group_i in groups){
        var group = groups[group_i];
        if(filter[group_i]){
            dimensions[group_i].filter(filter[group_i]);
        }
        results[group_i]= {values:group.all(), 
                           top: group.top(1)[0].value,
                          };
    }
    // send result back in JSON
    res.writeHead(200, { 'content-type': 'application/json' });
    res.end((JSON.stringify(results)));
});

load data into memory {
    // do crossfiltering
    var cf = crossfilter(data);

    // dimensions
    var dateDim = cf.dimension(function(d) {return d['timestamp'];});
    var typeDim = cf.dimension(function(d) {return d['destination_type'];});

    // groups    
    var numByDate = dateDim.group();
    var numByType = typeDim.group();   

    // put these dimensions and groups into corresponding variables
    dimensions.dateDim = dateDim;
    dimensions.typeDim = typeDim;

    groups.numByDate = numByDate; 
    groups.numByType = numByType; 
}

Client side app.js

var timeChart = dc.barChart("#pvs-time-chart");

var filteredData = {};
var queryFilter = {};

var refresh = function(queryFilter){
    // make ajax call
    d3.json("/refresh?filter="+JSON.stringify(queryFilter), function(d){
        filteredData = d;
        dc.redrawAll();
    });
}

var fakeDateDim = {
    filter: function(f){
        if(f){
            queryFilter["dateDim"]=f;
            refresh(queryFilter);
        }
    },
    filterAll: function(){

    }
};

var fakeNumByDate = {
    all: function(){
        try{
            var dateFormat = d3.time.format("%Y-%m-%d");
            filteredData["numByDate"].values.forEach(function (d) {
                d['key'] = dateFormat.parse(d['key']);
            });
        }
        catch(err) {
            // console.log(err);
        }
        return filteredData["numByDate"].values;
    },

    order: function(){

    },

    top: function(){

    }
 };


// Make charts
timeChart
  .width(1200)
  .height(250)
  .margins({top: 10, right: 50, bottom: 30, left: 50})
  .dimension(fakeDateDim)
  .group(fakeNumByDate)
  .transitionDuration(500)
  .x(d3.time.scale().domain([minDate, maxDate]))
  .elasticY(true)
  .renderHorizontalGridLines(true)
  .yAxis().ticks(4);

timeChart
  .filterHandler(function(dimension, filters){
    if (filters) {
      dimension.filter(filters);
    }
    else {
      dimension.filter(null);
    };
   return filters; 
   });

// Similarly for typeChart
var typeChart = dc.rowChart("#dest-type-row-chart");
...


function init(){
    d3.json("/refresh?filter={}", function(err, d){
        filteredData = d;
        dc.renderAll();
    });
}

init();

I'm really stuck with why these two charts are not interacting with each other, any suggestions? (My apologies if they are obvious things)

Edit :

The data is loaded from a mongodb database.

The two plots are all shown correctly, so I don't think there is issue with the data itself and how they are grouped.

There are no errors being thrown. Server side debug shows (with each user click/drag):

GET /refresh?filter={%22dateDim%22:[[%222017-08-13T21:10:30.937Z%22,%222017-
09-08T16:02:52.755Z%22]],%22typeDim%22:
[%22Player%22,%22Schedule%22,%22Other%22,%22Game%20Stats%22]} 200 8.333 ms - 
-

client side has no error shown.

The issue is, when you create two dc charts on selected dimensions and groups, they are supposed to interact/change according to the user action on either one of the two charts, but the way it's set up now, user action (clicks/drag) on either of the chart does not trigger the change to the other chart at all, the bar chart can still be clicked but nothing changes.