Something is off there. The for loop in vdev_probe() does continue in case of failures, but is intended to walk through all 4 labels, so why do we fail?

Yes, but not for all failures. If a label is found in the loop but later the associated data found invalid the vdev_probe() will fail even if there would be another label with better data.

In D20319#442667, @p.bruenn_beckhoff.com wrote:

Yes, but not for all failures. If a label is found in the loop but later the associated data found invalid the vdev_probe() will fail even if there would be another label with better data.

right, I see. I suggest to rework this solution even more; to something like:

vdev_probe()
{

 for (i = 0; i < VDEV_LABELS; i++) {
   vdev_read_label(i, &ub, &nvl);
   check if received  ub and nvl  are better than current.
}
if we have no good ub and nvl
  fail.

}

and vdev_read_label should read the specific label, perform initial tests and return good data.

This way we do not have to play the games with those fancy constants and games with inserting/deleting spa and vdev data.

Agreed, a sane vdev_probe() would be nice. But I have totally no idea how you want to omit cleaning up partially initialized spa/vdev objects without rewriting the whole file. Either some of the current error paths are dead code or things like vdev_init_from_nvlist() have to be replaced.

In D20319#443365, @p.bruenn_beckhoff.com wrote:

Agreed, a sane vdev_probe() would be nice. But I have totally no idea how you want to omit cleaning up partially initialized spa/vdev objects without rewriting the whole file. Either some of the current error paths are dead code or things like vdev_init_from_nvlist() have to be replaced.

I do not think that rewriting the whole file is needed:) And true, we probably need some configuration release during the process.