[1.8] Updating model from server data has inconsistent events

estesbubba · July 21, 2020, 6:57pm

In an older version of our product which is using 1.8.38 we have a very random bug when updating the model with new server data. This is the code that updates the model.

        me.nodes = me.buildNodes();
        me.links = me.buildLinks();
        me.goDiagram.model = new go.GraphLinksModel(me.nodes, me.links);

Most of the time this triggers a model ChangedEvent with e.oldValue === ‘Initial Layout’ but every once in a while it doesn’t. I’ve tried all kinds of things including wrapping the above in a transaction but the GoJS says not to update model while in a transaction.

In 2.1 I changed how we do it but these model calls aren’t in 1.8 so no help.

      diagram.startTransaction(tx);
      model.mergeNodeDataArray(nextProps.nodeDataArray);
      model.mergeLinkDataArray(nextProps.linkDataArray);
      diagram.commitTransaction(tx);

So in 1.8 what is the best way to replace an existing Diagram’s model with completely new data? My modelChangedListener performs logic on e.oldValue === ‘Initial Layout’ because our older apps link data doesn’t include start & end points, so when ‘Initial Layout’ doesn’t happen the diagram is incorrect.

Simple example of what the links look like.

This is what it looks like when I don’t get the ‘Initial Layout’.

walter · July 21, 2020, 7:45pm

What you did in 1.8 is the normal way to replace a diagram with a new one for all versions. Just construct the model and then assign Diagram.model.

Looking for an “Initial Layout” Transaction ChangedEvent should be OK. But you could just as well implement an “InitialLayoutCompleted” DiagramEvent listener, which is more commonly done.

You call the Model.mergeNodeDataArray method when you have a copy of the model data, including the whole Array and you want the model to be updated, finding existing nodes that have the same key and modifying them and maybe adding or removing a few nodes based on their keys.

One difference is that replacing the whole model by setting Diagram.model causes the various “Initial…” behaviors, whereas just modifying a few node data or link data (or even all of them) does not result in any “initial” behavior.

estesbubba · July 21, 2020, 9:00pm

I was hoping that I was doing something wrong because tracking this down is killing me!

I added this to my code.

        goDiagram.addDiagramListener('InitialLayoutCompleted', function (e) {
            console.log('InitialLayoutCompleted')
        });

And also this in my model ChangedEvent

console.log('Initial Layout', e.oldValue === 'Initial Layout');

So “InitialLayoutCompleted” always gets called on a server refresh but every once in a while “Initial Layout” does not which is where my problems occur. Any idea on what could cause this or should both always happen on a model update?

Our ModelChangedListener does a lot of work and several years ago allowed me to remove a lot of our diagram listeners.

walter · July 21, 2020, 9:25pm

In looking over the code I don’t see a way for the “InitialLayoutCompleted” DiagramEvent to be raised without raising the "Initial Layout** ChangedEvent.

Unless there is an exception causing the stack to be unwound in either the “InitialLayoutCompleted” or the “LayoutCompleted” DiagramEvent or the internal code for updating the scrollbars.

estesbubba · July 21, 2020, 9:49pm

This was happening with 1.6 so I decided to update to 1.8 earlier today but same problem. It could take 1 refresh or 13 before the problem happens which makes it really hard to track down.

This is my logging on a good refresh. The [Log] K {} is in the code where I updated the points.

[Log] InitialLayoutCompleted (AbstractModelerController.js, line 130)
[Log] Initial Layout – true (AbstractModelerController.js, line 171)
[Log] K {__gohashid: 1575, J: false, o: Array, I: 8, Lb: null, …} (AbstractModelerController.js, line 275)
[Log] updateNode (ext-all-rtl-debug.js, line 9184, x3)
[Log] updateLink (ext-all-rtl-debug.js, line 9184, x3)

This is my logging on a bad refresh.

[Log] InitialLayoutCompleted (AbstractModelerController.js, line 130)
[Log] updateNode (ext-all-rtl-debug.js, line 9184, x2)
[Log] updateLink (ext-all-rtl-debug.js, line 9184, x3)

There are no exception in the console. Is there some tracing I could turn on that might give some insight?

walter · July 21, 2020, 9:59pm

I don’t think there is any logging that would be useful to you.

I suppose you could try using the go-debug.js library, which will warn about more errors.

estesbubba · July 21, 2020, 10:08pm

I am using the debug library and it shows this on the refresh which is another thing I’m trying to track down. Could this be causing it? GoJS doesn’t like a transaction around the model change so I don’t know what do it here either.

[Log] Change not within a transaction: !d isTreeLeaf: 264:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 263:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 265:TEST  old: false  new: true (go-debug.js, line 35)

walter · July 22, 2020, 12:45am

What is happening when those warnings appear?

estesbubba · July 22, 2020, 12:54pm

Just updating the model using this code.

        me.nodes = me.buildNodes();
        me.links = me.buildLinks();
        Ext.log('updateModel');
        me.goDiagram.model = new go.GraphLinksModel(me.nodes, me.links);
        Ext.log('updateModel complete');

Here is the console for that.

[Log] updateModel (ext-all-rtl-debug.js, line 9184)
[Log] Change not within a transaction: !d isTreeLeaf: 264:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 263:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 265:TEST  old: false  new: true (go-debug.js, line 35)
[Log] updateModel complete (ext-all-rtl-debug.js, line 9184)

estesbubba · July 22, 2020, 12:59pm

If I wrap the model update with a transaction I get this.

[Error] Error: Do not replace a Diagram.model while a transaction is in progress. (anonymous function) (ext-all-rtl-debug.js:23305) (anonymous function) (ext-all-rtl-debug.js:7302)

walter · July 22, 2020, 1:11pm

Yes, the latter error makes sense because a Diagram only has an UndoManager through its Diagram.model, so replacing a model also replaces its undo manager, which would break the concept of a transaction replacing a model.

Those warnings about “Change not within a transaction” are not usually serious, but they are odd – I don’t know what might be causing that. Do you get those warnings consistently, whether or not you get your “Initial Layout” model ChangedEvent?

estesbubba · July 22, 2020, 1:21pm

Many clients have had this code in production for several years and I agree that change not within a transaction is not causing any problems. Just recently we had one bring up the multiple refreshes randomly causes links to lose points.

The transaction log happens on every refresh whether the “Initial Layout” fires or not. When it fires:

[Log] updateModel (ext-all-rtl-debug.js, line 9184)
[Log] Change not within a transaction: !d isTreeLeaf: 264:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 263:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 265:TEST  old: false  new: true (go-debug.js, line 35)
[Log] updateModel complete (ext-all-rtl-debug.js, line 9184)
[Log] InitialLayoutCompleted (AbstractModelerController.js, line 130)
[Log] Initial Layout – true (AbstractModelerController.js, line 171)
[Log] K {__gohashid: 2377, J: false, o: Array, I: 8, Lb: null, …} (AbstractModelerController.js, line 275)

When it doesn’t fire:

[Log] updateModel (ext-all-rtl-debug.js, line 9184)
[Log] Change not within a transaction: !d isTreeLeaf: 264:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 263:TEST  old: false  new: true (go-debug.js, line 35)
[Log] Change not within a transaction: !d isTreeLeaf: 265:TEST  old: false  new: true (go-debug.js, line 35)
[Log] updateModel complete (ext-all-rtl-debug.js, line 9184)
[Log] updateNode (ext-all-rtl-debug.js, line 9184)
[Log] InitialLayoutCompleted (AbstractModelerController.js, line 130)

The transaction warning does not happen when initially loading the model with data, only with each refresh.

walter · July 22, 2020, 1:26pm

OK, so those “Change not within a transaction” warnings can be ignored, since they happen in both cases.

What’s that “updateNode” line in the log? That looks suspicious and might explain the behavior of having that node’s connected links getting their routes invalidated.

estesbubba · July 22, 2020, 1:33pm

Here is the code for that.

    update: function(data) {
        var me = this,
            view = me.getView(),
            goDiagram = view.diagram.goDiagram,
            node = goDiagram.findNodeForData(view.getDiagramNodeData());

        Ext.log('updateNode');
        goDiagram.startTransaction('updateNode');
        view.buildNodeData();
        if (node) {
            Ext.iterate(view.nodeData, function(key, value) {
                goDiagram.model.setDataProperty(node.data, key, value);
            }, me);
        }
        goDiagram.commitTransaction('updateNode');
    }

walter · July 22, 2020, 1:39pm

Who calls this update function, and when? And why? Why are you not getting that “updateNode” log message every time?

What does buildNodeData do?

What property are you setting, and how would a change in that property affect your nodes – are there Bindings using that property as a source?

estesbubba · July 22, 2020, 2:21pm

This code was written 6 years ago so trying to remember what all it does. I do know that for links (above is for nodes) it updates the points since our DB doesn’t store start and end points, only midpoints. This code happens in the “Initial Layout” ChangedEvent so in the random times that doesn’t get called the buildNodeData and buildLinkData doesn’t happen. That’s why when the user does a refresh (updating from server) the links lose their midpoints.

estesbubba · July 23, 2020, 12:57pm

Walter if I comment out the code below my refresh problem doesn’t happen. Any idea why this code would cause the “Initial Layout” ChangedEvent to randomly not fire when Diagram.model is changed? Is there better way to implement what I’m doing below that might fix the problem?

        goDiagram.startTransaction('updateLink');
        view.buildLinkData();
        if (link) {
            Ext.iterate(view.linkData, function(key, value) {
                goDiagram.model.setDataProperty(link.data, key, value);
            }, me);
        }
        goDiagram.commitTransaction('updateLink');

walter · July 23, 2020, 1:22pm

Sorry, but I cannot tell what that code is supposed to do.

estesbubba · July 23, 2020, 1:34pm

It’s updating pieces of model data when the user makes changes in our properties. For example they can make a link hidden in our properties which is then changed in the diagram model. This code works just fine but for some reason causes a later “Initial Layout” ChangedEvent to not fire but the “InitialLayoutCompleted” DiagramListener still fires. It’s totally random and could happen after 1 refresh or take 13 refreshes.

So my are questions are:

Any reason calling model.setDataProperty() would randomly cause the ChangedEvent to not fire on a future model replace?
Any alternative ways to implement the functionality that might fix the problem?

walter · July 23, 2020, 1:39pm

You still haven’t described when the update function is being called.

It seems to me that your properties editor should make those changes to the model when the user makes them, not when loading a model.

Or are those properties supposed to modify all models when they are loaded? If so, then make those changes to the model data before setting Diagram.model, not after or during the loading process. Or maybe make changes to the templates before setting Diagram.model.