Hi, i have nanopct4 with zfs on sdcard and clone git repository # mmcsd0: Error indicated: 4 Failed mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout panic: Memory modified after free 0xffff0000f2ce1000(131072) val=65766964 @ 0xffff0000f2ce1000 cpuid = 4 time = 1652189895 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x30 vpanic() at vpanic+0x174 panic() at panic+0x44 trash_ctor() at trash_ctor+0x5c item_ctor() at item_ctor+0x68 abd_borrow_buf_copy() at abd_borrow_buf_copy+0x34 vdev_geom_io_start() at vdev_geom_io_start+0x1f0 zio_vdev_io_start() at zio_vdev_io_start+0x39c zio_nowait() at zio_nowait+0xe0 vdev_queue_io_done() at vdev_queue_io_done+0x1fc zio_vdev_io_done() at zio_vdev_io_done+0xbc zio_execute() at zio_execute+0x80 taskqueue_run_locked() at taskqueue_run_locked+0x178 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8 fork_exit() at fork_exit+0x74 fork_trampoline() at fork_trampoline+0x14 KDB: enter: panic
KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x30 vpanic() at vpanic+0x174 panic() at panic+0x44 trash_ctor() at trash_ctor+0x5c item_ctor() at item_ctor+0x68 arc_buf_alloc_impl() at arc_buf_alloc_impl+0x560 arc_alloc_buf() at arc_alloc_buf+0x68 dbuf_read() at dbuf_read+0x7c8 dmu_buf_will_dirty_impl() at dmu_buf_will_dirty_impl+0x11c dmu_write_uio_dnode() at dmu_write_uio_dnode+0xd4 dmu_write_uio_dbuf() at dmu_write_uio_dbuf+0x48 zfs_write() at zfs_write+0x590 zfs_freebsd_write() at zfs_freebsd_write+0x44 VOP_WRITE_APV() at VOP_WRITE_APV+0xa4 vn_write() at vn_write+0x308 vn_io_fault_doio() at vn_io_fault_doio+0x4c vn_io_fault1() at vn_io_fault1+0x140 vn_io_fault() at vn_io_fault+0x18c dofilewrite() at dofilewrite+0x7c kern_writev() at kern_writev+0x50 sys_write() at sys_write+0x80 do_el0_sync() at do_el0_sync+0x524 handle_el0_sync() at handle_el0_sync+0x40 --- exception, esr 0x56000000 KDB: enter: panic [ thread pid 1448 tid 100285 ] Stopped at kdb_enter+0x40: undefined f902027f
this is was before crash mmcsd0: Error indicated: 4 Failed) mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout Solaris: WARNING: Pool 'nanopct4pool' has encountered an uncorrectable I/O failure and has been suspended.
What's the ashift of the pool? If it's 9, there is a recent bug fix which might be relevant: https://github.com/openzfs/zfs/commit/9209ea69bc03e7e9f678b2294da7a0317b5c9c5b
(In reply to Mark Johnston from comment #3) Hi, # zpool get all NAME PROPERTY VALUE SOURCE nanopct4pool size 11.5G - nanopct4pool capacity 10% - nanopct4pool altroot - default nanopct4pool health ONLINE - nanopct4pool guid 7980528985915604956 - nanopct4pool version - default nanopct4pool bootfs nanopct4pool/ROOT/freebsd-14.0-CURRENT local nanopct4pool delegation on default nanopct4pool autoreplace off default nanopct4pool cachefile - default nanopct4pool failmode wait default nanopct4pool listsnapshots off default nanopct4pool autoexpand off default nanopct4pool dedupratio 1.00x - nanopct4pool free 10.3G - nanopct4pool allocated 1.19G - nanopct4pool readonly off - nanopct4pool ashift 0 default
Next panic mmcsd0: Error indicated: 4 Failed mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout panic: Memory modified after free 0xffff0000ee6ed000(131072) val=46242023 @ 0xffff0000ee6ed000 cpuid = 4 time = 1652279802 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x30 vpanic() at vpanic+0x174 panic() at panic+0x44 trash_ctor() at trash_ctor+0x5c item_ctor() at item_ctor+0x68 arc_buf_alloc_impl() at arc_buf_alloc_impl+0x560 arc_read() at arc_read+0x1620 dbuf_read() at dbuf_read+0xbf4 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x264 dmu_buf_hold_array() at dmu_buf_hold_array+0x68 dmu_read_pages() at dmu_read_pages+0x9c zfs_freebsd_getpages() at zfs_freebsd_getpages+0x1e0 vnode_pager_getpages() at vnode_pager_getpages+0x30 vm_pager_get_pages() at vm_pager_get_pages+0x68 vm_fault() at vm_fault+0x544 vm_fault_trap() at vm_fault_trap+0x6c data_abort() at data_abort+0xe0 do_el0_sync() at do_el0_sync+0x128 handle_el0_sync() at handle_el0_sync+0x40 --- exception, esr 0x92000007 KDB: enter: panic [ thread pid 1279 tid 100310 ] Stopped at kdb_enter+0x40: undefined f902027f db>
next panic # mmcsd0: Error indicated: 4 Failed mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout mmcsd0: Error indicated: 1 Timeout panic: VERIFY3(0 == abd_cmp_buf(abd, buf, n)) failed (0 == -65) cpuid = 2 time = 1653060597 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x30 vpanic() at vpanic+0x174 spl_panic() at spl_panic+0x40 abd_return_buf() at abd_return_buf+0xe8 vdev_geom_io_done() at vdev_geom_io_done+0xc0 zio_vdev_io_done() at zio_vdev_io_done+0x148 zio_execute() at zio_execute+0x80 taskqueue_run_locked() at taskqueue_run_locked+0x178 taskqueue_thread_loop() at taskqueue_thread_loop+0xc8 fork_exit() at fork_exit+0x74 fork_trampoline() at fork_trampoline+0x14 KDB: enter: panic [ thread pid 0 tid 100182 ] Stopped at kdb_enter+0x40: undefined f907c27f db>
(In reply to Mark Johnston from comment #3) # zdb -C nanopct4pool: version: 5000 name: 'nanopct4pool' state: 0 txg: 73 pool_guid: 4341490173460547177 errata: 0 hostname: '' com.delphix:has_per_vdev_zaps vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 4341490173460547177 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 14273472532592614282 path: '/dev/diskid/DISK-6FA28943p2' whole_disk: 1 metaslab_array: 64 metaslab_shift: 28 ashift: 12 asize: 4206100480 is_log: 0 create_txg: 4 com.delphix:vdev_zap_leaf: 129 com.delphix:vdev_zap_top: 130 features_for_read: com.delphix:hole_birth com.delphix:embedded_data