What should it do when there are five or more links connected to the port? If there are four, should they connect at each of the four sides, even though some might be on the "wrong" side relative to where the link should go?
I think you will need to override four GoPort methods: Get[From/To]Link[Point/Dir] and implement what you want from scratch. There are some example port classes that override these methods, but they do so for purposes different than yours. Still, you might find them instructive.