Not neccessarily a black flag offense and usually only punished if a position is gained illegally. You won't gain any time by cutting across gravel or grass.