Being "planar" means that a polygon must be flat. This is easy to do if you never use UPPITCH, DOWNPITCH, RIGHTROLL, or LEFTROLL between POLYSTART and POLYEND.

This is legal:

POLYSTART
REPEAT 4 [FORWARD 100 RIGHT 90] ; a square is planar
POLYEND

This is NOT legal:

POLYSTART
REPEAT 3 [
  FORWARD   100
  RIGHT     90
]
FORWARD   50
UPPITCH   90  ; error: these instruction move
FORWARD   50  ;        out of the polygon's plane
DOWNPITCH 90  ;
FORWARD   50  ;
DOWNPITCH 90  ;
FORWARD   50  ;
UPPITCH   90  ;
FORWARD   50
RIGHT     90
POLYEND

If you want to create a polygon with a "bend" in it that leaves the plane, you can break up your polygon into simpler polygons that are coplanar.