src/stand/libsa/zfs/zfsimpl.c defines features_for_read, which contains a set of whitelisted zpool features. If any other read-only-incompatible feature is active on a boot pool (that is, enabled and in use) the boot loader will fail to boot from the pool. With ZoR, it is easy to shoot oneself in the foot by accidentally activating a read-only-incompatible feature. Currently these include redaction_bookmarks and redacted_datasets, activated by creating redaction bookmarks and receiving redacted send streams. It would be nice if zfs redact and zfs receive commands checked the feature activation and warned the (unsuspecting) user about this if the pool is a boot pool. Suggested message for zfs redact: WARNING: A redaction bookmark was created on a boot pool. Currently FreeBSD cannot boot from a pool with redaction bookmarks or redacted datasets. To undo this change, run: zfs destroy zroot/test/base#redacted To identify all redaction bookmarks and redacted snapshots on the pool, run: zfs get -rHp -t bookmark,snapshot -o name redact_snaps zroot
I haven't yet attempted to reproduce this, but the described behaviour of enabling a read-only-incompatible feature upon receiving a dataset sounds quite wrong. I do see the following in recv_begin_check_feature_flags_impl(): 543 /* 544 * Receiving redacted streams requires that redacted datasets are 545 * enabled. 546 */ 547 if ((featureflags & DMU_BACKUP_FEATURE_REDACTED) && 548 !spa_feature_is_enabled(spa, SPA_FEATURE_REDACTED_DATASETS)) 549 return (SET_ERROR(ENOTSUP)); so it seems that we are indeed checking this. Perhaps that's not sufficient somehow. Of course, ideally we could handle these features in the loader.
(In reply to Mark Johnston from comment #1) Feature flags are tri-state: * disabled - the feature cannot be used in the pool. * enabled (but inactive) - the feature can be used, but is not currently in use. * active (implies enabled) - the feature can be used, and is in use. The quoted piece of code prevents pools with redacted_datasets feature disabled from receiving redacted streams, whereas the bug is about receiving redacted streams in a pool with the feature already enabled (but inactive), resulting in the feature state transitioning from enabled to active. The boot loader rejects the pool if a non-whitelisted read-only-incompatible feature is active; enabled is fine.
(In reply to Eugene M. Kim from comment #2) > Feature flags are tri-state: I meant feature properties (of zpool), not feature flags (of zstream). Sorry for the confusion!