GoButtons are actually pretty simple GoGroups consisting of three objects: a Background (normally a GoRectangle), an Icon (normally null), and a Label (a GoText).
LayoutChildren just positions the Icon (if any) to the left of the Label, and positions the Background (if any) to fit around the Icon and Label, with margins given by the TopLeftMargin and BottomRightMargin properties.
It also overrides Paint to call PaintButton as well, which just draws a border just like ControlPaint.DrawBorder3D does.
Finally it also implements IGoActionObject, so that it can handle all mouse events (pre-empting the other tools), including autorepeating if the AutoRepeating property is true.
A simple example with some shapes as the Icon (actually a GoGroup holding several shapes), no Label and a transparent Background:
GoButton but = new GoButton();
but.Label = null; // get rid of standard GoText label
((GoRectangle)but.Background).Brush = null; // get rid of standard GoRectangle background brush
GoGroup g = new GoGroup();
g.Selectable = false; // don't let users select this or any of the shapes
GoRectangle r = new GoRectangle();
r.Selectable = false;
r.Brush = Brushes.LightSalmon;
r.Bounds = new RectangleF(10, 10, 20, 30);
g.Add(r);
GoEllipse e = new GoEllipse();
e.Selectable = false;
e.Brush = Brushes.LightBlue;
e.Bounds = new RectangleF(40, 10, 30, 30);
g.Add(e);
GoTriangle t = new GoTriangle();
t.Selectable = false;
t.Brush = Brushes.LightGreen;
t.A = new PointF(80, 10);
t.B = new PointF(90, 40);
t.C = new PointF(100, 10);
g.Add(t);
but.Icon = g; // the Icon is the group of three shapes
but.Position = new PointF(100, 100);
goView1.Document.Add(but);
I think the reason your button is always 10x10 on the inside is because that's the size of original Background, the GoRectangle created by GoButton.CreateBackground. And since you don't have either an Icon or a Label, it didn't have any reason to be resized.
You can override PaintButton if you want to draw the "button" borders differently. I do not recommend calling ControlPaint methods if you can avoid it.
Implementing an IGoActionObject is certainly one way to handle clicking in a non-standard manner, while allowing all other kinds of objects to be handled in the normal manner.
An alternative is to implement your own tool that is first on the list of GoView.MouseDownTools, so that it takes precedence over all other tools, if it wants to. That's basically what GoToolAction is doing.