I migrated a zpool (mirror) from two 1T drives to two 3T drives. It had two (also mirrored) log devices. I detached both mirror devices, one at a time, and then replaced both drives, one at a time with autoexpand turned on. Then I re-attached the log devices in the same mirrored configuration. The pool is healthy and seems to be working normally, but I can't detach devices from it anymore. Any time I try to detach any of the four devices, the system becomes completely unresponsive. The software version is the same. The only configuration difference is that the new pool has 4k blocks versus 512b and also has a different pool name. After a forced reboot, pool is marked as degraded and I can plug the device back in and wait for a resilver. The device doesn't become detached. I can't produce a vmcore because the system becomes completely unresponsive but doesn't crash.
what is the output from: 1. zpool status 2. uname -a
Also: 3. zdb -C <pool>
I was able to take a disk out using "zpool split" and add disks from former pool to it. I'm in a middle of a resilver right now, but I'll try to reprocude the problem and provide the info when I'm done.
$ zpool status pool: data state: ONLINE scan: scrub repaired 75K in 6h34m with 0 errors on Sun Dec 20 12:43:26 2015 config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ada1 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 logs mirror-2 ONLINE 0 0 0 ada6p1 ONLINE 0 0 0 ada7p1 ONLINE 0 0 0 errors: No known data errors pool: tank state: ONLINE scan: resilvered 583G in 1h56m with 0 errors on Thu Jan 7 11:03:58 2016 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada4p2 ONLINE 0 0 0 ada5p2 ONLINE 0 0 0 logs mirror-1 ONLINE 0 0 0 ada6p2 ONLINE 0 0 0 ada7p2 ONLINE 0 0 0 errors: No known data errors $ uname -a FreeBSD lifanovbsd0 11.0-CURRENT FreeBSD 11.0-CURRENT #21 r293241M: Wed Jan 6 13:21:02 EST 2016 root@lifanovbsd0:/usr/obj/usr/src/sys/GENERIC-NODEBUG amd64 $ zdb -C data: version: 5000 name: 'data' state: 0 txg: 12863548 pool_guid: 2244539855095666375 hostid: 657389954 hostname: 'lifanovbsd0' vdev_children: 3 vdev_tree: type: 'root' id: 0 guid: 2244539855095666375 children[0]: type: 'mirror' id: 0 guid: 9136418650500458576 metaslab_array: 37 metaslab_shift: 32 ashift: 9 asize: 500103118848 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 820204606399244410 path: '/dev/ada0' phys_path: '/dev/ada0' whole_disk: 1 DTL: 481 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 7129954888383586426 path: '/dev/ada1' phys_path: '/dev/ada1' whole_disk: 1 DTL: 480 create_txg: 4 children[1]: type: 'mirror' id: 1 guid: 22857558361533772 metaslab_array: 34 metaslab_shift: 32 ashift: 9 asize: 500103118848 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 3541792554089895143 path: '/dev/ada2' phys_path: '/dev/ada2' whole_disk: 1 DTL: 483 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 4600530648211267864 path: '/dev/ada3' phys_path: '/dev/ada3' whole_disk: 1 DTL: 482 create_txg: 4 children[2]: type: 'mirror' id: 2 guid: 10125232546100156377 metaslab_array: 8299 metaslab_shift: 27 ashift: 12 asize: 17175150592 is_log: 1 create_txg: 9083333 children[0]: type: 'disk' id: 0 guid: 9706229731141400493 path: '/dev/ada6p1' phys_path: '/dev/ada6p1' whole_disk: 1 DTL: 2682 create_txg: 9083333 children[1]: type: 'disk' id: 1 guid: 4287673589667345344 path: '/dev/ada7p1' phys_path: '/dev/ada7p1' whole_disk: 1 DTL: 2681 create_txg: 9083333 features_for_read: com.delphix:hole_birth com.delphix:embedded_data tank: version: 5000 name: 'tank' state: 0 txg: 13891 pool_guid: 1331125925218803251 hostid: 657389954 hostname: 'lifanovbsd0' vdev_children: 2 vdev_tree: type: 'root' id: 0 guid: 1331125925218803251 create_txg: 4 children[0]: type: 'mirror' id: 0 guid: 9985890284685236925 whole_disk: 0 metaslab_array: 34 metaslab_shift: 34 ashift: 12 asize: 2748774350848 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 18200602675754198144 path: '/dev/ada4p2' phys_path: '/dev/ada4p2' whole_disk: 1 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 9057482964507241068 path: '/dev/ada5p2' phys_path: '/dev/ada5p2' whole_disk: 1 DTL: 40 create_txg: 4 resilver_txg: 13092 children[1]: type: 'mirror' id: 1 guid: 1504801081987587436 metaslab_array: 68 metaslab_shift: 27 ashift: 12 asize: 17175150592 is_log: 1 create_txg: 13887 children[0]: type: 'disk' id: 0 guid: 14079142470155627886 path: '/dev/ada6p2' phys_path: '/dev/ada6p2' whole_disk: 1 create_txg: 13887 children[1]: type: 'disk' id: 1 guid: 2344469679409857908 path: '/dev/ada7p2' phys_path: '/dev/ada7p2' whole_disk: 1 create_txg: 13887 features_for_read: com.delphix:hole_birth com.delphix:embedded_data
The problem pool was "tank", but I can't reproduce the problem anymore after doing a "zpool split" and reconnecting the rest of the old pool devices to it. I was able to successfully detach ada5p2 and then reattach it.
Thanks Nikolai, closing as no reproducible. If anyone has the same issue again please feel free to re-open.