Bug 247696

Summary:	Boot fails when attempting to mount with gmirror disk from different o/s release
Product:	Base System	Reporter:	Barry Friedman <bfriedman>
Component:	kern	Assignee:	freebsd-bugs (Nobody) <bugs>
Status:	Closed FIXED
Severity:	Affects Only Me
Priority:	---
Version:	12.1-RELEASE
Hardware:	Any
OS:	Any

Description Barry Friedman 2020-07-01 19:19:22 UTC

In order to upgrade a server running 10.4 using gmirror,  shutdown the system and removed one disk.  Installed the disk on another machine and proceeded to upgrade to 12.1-release-p2.  Installed a new disk on 10.4 system and rebuilt the mirror.
Once the 12.1 system build was stable did a gmirror remove on the rebuilt mirror on 10.4 and removed the spare disk. Brought the spare disk over to the new system and attempted a boot with both disks present.  The boot produced a series of  "init getting kqueues resource limit: Invalid argument" messages.  When I attempted to login the login failed and returned to the login: prompt,  Syslog showed the Invalid argument message.  Disabling the spare disk, 12.1 booted as normal. Attempted to boot the 10.4 system which was on the spare disk and this failed to mount /.

Brought the spare disk back to the 10.4 system and the system booted.

I believe there is some kind of a mismatch between the 10.4 gmirror disk and what 12.1 is expecting.  

At this point I have both systems running, should anyone like to have me troubleshoot this.  My plan at the moment is to disable the gmirror on the 10.4 system and mount both disks stand alone.  Then re-format the spare.  I would also like to destroy the 12.1 mirror and proceed to rebuild the mirror from scratch although I am not sure that that is safe.

Setting component to kern although it might be gnu.

Comment 1 Conrad Meyer freebsd_committer

2020-07-01 21:58:05 UTC

What are you actually trying to do here?  You managed to update an (EOL 10.4) system to 12.1 successfully.

What are you doing after that?  What do you expect to happen?  What do you observe?

Comment 2 Barry Friedman 2020-07-02 00:18:25 UTC

In case it wasn't clear the intent was to use the spare disk rebuilt on 10.4 to mirror the 12.1 system disk.  I only have 3 X 2 tb disks and no longer need to support the 10.4 system with a mirror.

What I observed is that it's not possible to successfully boot the 12.1 system which has a status of 
> gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ada0 (ACTIVE)

When only the system disk is connected and the system boots as normal.

With the 10.1 disk connected it boots but I cannot login due to errors of 

 "init getting kqueues resource limit: Invalid argument" messages. and the system returns to the login prompt.  

I want to break the mirror on 12.1 which carried over the gmirror status of the disk that had been taken from the 10.4 system at the start of this process
and rebuild it with the spare disk from 10.4.
I had thought that I could simply gmirror insert the disk as after a failure.

Looking over the report I sent I can't be sure whether I did a gmirror forget
or gmirror remove.  In any case the 10.1 system  shows the same gmirror status as above.

Any advice would be appreciated.

Comment 3 Barry Friedman 2020-07-02 00:55:46 UTC

Mounted the disk on 10.1 and did a gpart -f destroy ada1
Moved the disk over and all is well.