Bug 274103

This is a horror story with production server that hanged and did not reboot. This was a rare case that one can't predict and it related to logic inside geom_gmirror so I think  it is actual for all version of FreeBSD.

Background:
There was two SAS HDD: Toshiba and Seagate ST1800MM0129.
Both are GPT partitioned and three UFS paritition are mirrored on each other.

Failure:
There was an error from Seagate disk logged in /var/messages. Error seemed serious. Because of no ability to physically remove Seagate from system there  was a decision do logically detach it from system.
It've been realized by series of command like "gmirror detach". There was a long time of each command. One command hangs, other command seems complete but mirror status stayed the same. Because of this there was a decision to reboot.

After reboot system hangs during geom_mirror initialization.
Also tried a FreeBSD 13-STABLE USB-stick but result was the same.

Cause:
(this is mostly reconstruction because there was no time to explore problem meaningfully)

After reboot (reinitialization?) Seagate ST1800MM0129 goes to read-only mode because of internal hardware failure. It appear in camcontrol(?) with flag MEDIUM: WRITE_PROTECTED. Geom_mirror unexpect this case and attach it as usual as part of a mirror. But with the first attempt to write to it hangs.


Summary:

geom_mirror must check each mirror component for WRITE_PROTECTED flag and do not attach such a component to a mirror (at least during initialization)

P.S. Solution for people in this situation: use Seagate SeaChest tools for FreeBSD to engage self-wiping process on failure disk. Disk will become clean and geom_mirror will initialize successfully.