Introduce an Asymmetric switch in Vision Settings. Only asymmetric parts can have vision offsets. The Vision Offsets fields are disabled in the UI, if asymmetric is off. This should prevent accidental entering or pressing the Detect button (confusing it with Test Alignment).
