Performance alternatives

GoJS
1

Hi!

In the last few days I’ve been trying to speed up my application. I need to display about 40k nodes, and it’s taking about 40 seconds, in which the browser stays completely frozen.
My first attempt was to remove all customization from the node template, which reduced the time by half. Still, 20 seconds is not acceptable, so I tried to use virtualization. In this approach, the initial load is pretty quick, but I lost some features that are essential for the application (expand/collapse hierarchy, change node’s parent, localize node).

So my doubt is: Is 20~40 seconds the default time for loading this amount of nodes without virtualization? Is it possible to replicate behaviours like expand/collapse and change node’s superior with virtualization? If so, is there any examples for that? In the worst case scenario, is it possible to prevent the browser from freezing, so I can display a progress bar animation?

Thanks,

2

40 seconds for 40,000 nodes sounds reasonable. Alas, JavaScript provides no mechanisms for parallel computations except in Web Workers – but those operate in different address spaces, so the objects built up there cannot be used by the main browser “thread”.

Rather than building (or reading) the whole model and then assigning it to the diagram all at once, you could try setting the diagram’s model to an empty model with the properties that you want. Then you could try loading 1000 nodes at a time in a setTimeout function. I’m assuming that you have node locations for all of your data in the model data, so no layout is required while you are loading – set the Layout.isInitial and .isOngoing both to false on your Diagram.layout.

3

Hi walter! Thanks for the reply!

Well, I think that using an interval should be a great solution! However, I’m still facing a problem:
I got my setTimeout function to work, and it performs quickly on start, but on each iteration it slows down as the array of nodes gets bigger.
When I set the .isInitial and .isOngoing both to false, my diagram disappears, but I don’t have the nodes locations. If I adapt a virtualization example for calculating its positions, considering that I have a very dynamic diagram (it’s possible to change node’s parent, add childs and delete nodes), should I recalculate it at every operation, or it would be possible to restore the default behavior once it get loaded? Is it possible to avoid calculating their positions manually and keep the setTimeout strategy somehow?

4

So you’re calling Model.addNodeDataCollection, or a bunch of calls to Model.addNodeData, in each timer call?

Maybe don’t set Diagram.layout until after all of your nodes have been loaded.

You better not be using LayeredDigraphLayout, because that can’t handle even several thousand nodes. Even ForceDirectedLayout might be too slow for that many nodes, although you can change its parameters.

5

I was updating my nodeDataArray in each iteration, like self.diagram.model.nodeDataArray = […]
So I replaced it for addNodeDataCollection and now each call takes the same time to complete! This allows me to build a progress bar, which is great!!

The only problem is: in my diagram, for some reason, when a child node is inside a different batch from its parent, they never get “linked”, resulting in multiple nodes that act like root, even when all timeout calls are done. Do you know what I could be doing wrong?

My timeout function looks like this:
`

    var quantity = 1000;
    var allPositions = this.simulationMonth.listSimulationPositions();
    self.diagram.model.nodeDataArray = [];

    var i = 0;
    function includeOnView(diagram, allPositions){
      console.log('iteration: ' + (i++));
      var start = diagram.model.nodeDataArray.length;
      var end = start + quantity;
      diagram.model.addNodeDataCollection(allPositions.slice(start, end > allPositions.length ? allPositions.length : end));
      if (allPositions.length > diagram.model.nodeDataArray.length){
        setTimeout(function(){
          includeOnView(diagram, allPositions);
        }, 900);
      }
    }

    includeOnView(self.diagram, allPositions);

    self.diagram.startTransaction('centralize');
    self.diagram.alignDocument(go.Spot.Center, go.Spot.Center);
    self.diagram.commitTransaction('centralize');

and some configurations:

`

layout: {
  treeStyle: go.TreeLayout.StyleLastParents,
  arrangement: go.TreeLayout.ArrangementHorizontal,
  sorting: go.TreeLayout.SortingAscending,
  nodeSpacing: 100,angle: 90,
  layerSpacing: 35,
  alternateAngle: 90,
  alternateLayerSpacing: 35,
  alternateAlignment: go.TreeLayout.AlignmentBus
},
diagram: {
  validCycle: go.Diagram.CycleDestinationTree,
  maxSelectionCount: 1,

  allowZoom: true,
  allowDelete: false,

  scrollMode: go.Diagram.InfiniteScroll,

  'animationManager.isEnabled': false,

  'toolManager.hoverDelay': 100,
  'undoManager.isEnabled': false
}

`

6

Yes, replacing the whole Model.nodeDataArray with ever increasing length Arrays would cause that N^2 behavior.

I think you can greatly decrease the setTimeout delay from 900 to 1.

I suspect that each time you call Model.add… you need to do so within a transaction. But I’m not sure that’s the only problem. I assume you originally just set model.nodeDataArray = this.simulationMonth.listSimulationPositions()?

7

Thank you so much for the help so far!

Yes, it was my original code. In this way, all the parent links are created, but I get fully frozen for the entire processing.

I also wrapped the “addNodeDataCollection” inside a transaction like the following, but like you said, probably this is not the only problem. I’ve tried using transactions from both “diagram” and “diagram.model”. I’ve noticed that some nodes are correctly placed below their parents, but not all of them.

`

function includeOnView(diagram, allPositions){
  var start = diagram.model.nodeDataArray.length;
  var end = start + quantity;
  diagram.startTransaction('update');
  diagram.model.addNodeDataCollection(allPositions.slice(start, end > allPositions.length ? allPositions.length : end));
  diagram.commitTransaction('update');
  
  if (allPositions.length > diagram.model.nodeDataArray.length){
    setTimeout(function(){
      includeOnView(diagram, allPositions);
    }, 1);
  }
}

includeOnView(self.diagram, allPositions);

`

8

For any of the missing link cases, are any of the tree child nodes being loaded in a transaction before the tree parent node?

I’m wondering if it doesn’t support missing referents between transactions.

9

Hi Walter!

I think we finally found a solution! I’ve tested pushing all nodes at once using addNodeDataCollection, but the links were still missing. Then I realized that the order inside the array does matter, even if its inside the same transaction.

Ordering the array according to the hierarchy level solved the “missing links” problem. Now I can work on a loading bar to improve the user experience. =D

Thanks for all help!!

10

No, within a transaction, it’s OK to have temporary unresolved references between nodes, or from links to nodes. It’s just that those unresolved references are discarded at the end of the transaction (i.e. missing Links), even though the data objects continue to hold the references.

However, I’ve confirmed that at least in simple cases involving a TreeModel, the unresolved references can be resurrected as Links in the Diagram by calling Diagram.updateAllRelationshipsFromData.

      var m = new go.TreeModel();
      m.nodeDataArray = [
        { key: 4, parent: 3, text: "n4" },
        { key: 3, parent: 1, text: "n3" }
      ];
      diagram.model = m;

This produces two nodes connected by a link from “n3” to “n4”.

Then later:

      var m = diagram.model;
      m.startTransaction();
      m.addNodeDataCollection([
        { key: 2, parent: 1, text: "n2" },
        { key: 1, text: "n1" }
      ]);
      diagram.updateAllRelationshipsFromData();
      m.commitTransaction("added two");

And this now produces the correct graph: “n1” ==> “n2”, “n1” ==> “n3”, “n3” ==> “n4”.

Without the call to updateAllRelationshipsFromData, the graph has the correct nodes and links except not the link from “n1” to “n3”, as you’ve discovered.

11

Great!!! You were right! Calling this method makes unnecessary to previously sort my array of nodes!