Bug 239702

Summary: sysutils/openzfs bectl activate … cannot promote … not a cloned filesystem … did not successfully activate boot environment
Product: Ports & Packages Reporter: Graham Perrin <grahamperrin>
Component: Individual Port(s)Assignee: Ryan Moeller <freqlabs>
Status: New ---    
Severity: Affects Some People CC: 0mp, allanjude, freqlabs, kevans
Priority: --- Flags: bugzilla: maintainer-feedback? (kmoore)
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232488

Description Graham Perrin freebsd_committer freebsd_triage 2019-08-07 18:48:27 UTC
With ZoF: 

----

…
Updating '/usr/src':
At revision 350684.
root@momh167-gjp4-8570p:~ # bectl create r350684
root@momh167-gjp4-8570p:~ # bectl activate r350684
cannot promote 'copperbowl/ROOT/r350684': not a cloned filesystem
did not successfully activate boot environment r350684
root@momh167-gjp4-8570p:~ # date ; uname -v 
Wed Aug  7 19:01:53 BST 2019
FreeBSD 13.0-CURRENT r350368 GENERIC 
root@momh167-gjp4-8570p:~ # bectl list -as
BE/Dataset/Snapshot                               Active Mountpoint Space Created

r350368
  copperbowl/ROOT/r350368                         N      /          60.7G 2019-07-27 09:56
  r350368@2019-06-16-08:24:26-0                   -      -          4.11G 2019-06-16 08:24
  r350368@2019-07-05-11:42:18-0                   -      -          8.55G 2019-07-05 11:42
  r350368@2019-07-13-23:44:48-0                   -      -          7.61G 2019-07-13 23:44
  r350368@2019-07-16-13:57:25-0                   -      -          5.43G 2019-07-16 13:57
  r350368@2019-07-27-09:56:16-0                   -      -          7.91G 2019-07-27 09:56
  r350368@2019-08-07-19:01:01-0                   -      -          0     2019-08-07 19:01

r350684
  copperbowl/ROOT/r350684                         R      -          8K    2019-08-07 19:01

r349033
  copperbowl/ROOT/r349033                         -      -          29.9G 2019-06-14 16:04
  r349033@2019-06-14-16:04:33-0                   -      -          4.84G 2019-06-14 16:04

r350027
  copperbowl/ROOT/r350027                         -      -          2.45M 2019-07-16 13:57

r343663
  copperbowl/ROOT/r343663                         -      -          25.2G 2019-02-01 18:04
  r343663@2019-02-01-18:04:56                     -      -          5.16G 2019-02-01 18:04
root@momh167-gjp4-8570p:~ # bectl list | sort
BE      Active Mountpoint Space Created
r343663 -      -          25.2G 2019-02-01 18:04
r349033 -      -          29.9G 2019-06-14 16:04
r350027 -      -          2.45M 2019-07-16 13:57
r350368 N      /          60.7G 2019-07-27 09:56
r350684 R      -          8K    2019-08-07 19:01
root@momh167-gjp4-8570p:~ # bectl activate r350368
cannot promote 'copperbowl/ROOT/r350368': not a cloned filesystem
did not successfully activate boot environment r350368
root@momh167-gjp4-8570p:~ # bectl list | sort
BE      Active Mountpoint Space Created
r343663 -      -          25.2G 2019-02-01 18:04
r349033 -      -          29.9G 2019-06-14 16:04
r350027 -      -          2.45M 2019-07-16 13:57
r350368 NR     /          60.7G 2019-07-27 09:56
r350684 -      -          8K    2019-08-07 19:01
root@momh167-gjp4-8570p:~ # zpool get bootfs copperbowl
NAME        PROPERTY  VALUE                    SOURCE
copperbowl  bootfs    copperbowl/ROOT/r350368  local
root@momh167-gjp4-8570p:~ # bectl activate r350684
cannot promote 'copperbowl/ROOT/r350684': not a cloned filesystem
did not successfully activate boot environment r350684
root@momh167-gjp4-8570p:~ # zpool get bootfs copperbowl
NAME        PROPERTY  VALUE                    SOURCE
copperbowl  bootfs    copperbowl/ROOT/r350684  local
root@momh167-gjp4-8570p:~ # bectl activate r350368
cannot promote 'copperbowl/ROOT/r350368': not a cloned filesystem
did not successfully activate boot environment r350368
root@momh167-gjp4-8570p:~ # pkg query '%o %v %R' openzfs openzfs-kmod
sysutils/openzfs 2019072200 poudriere
sysutils/openzfs-kmod 2019072200_1 poudriere
root@momh167-gjp4-8570p:~ # grep zfs /boot/loader.conf
# zfs_load="YES"
openzfs_load="YES"
root@momh167-gjp4-8570p:~ # grep zfs /etc/rc.conf
zfs_enable="YES"
root@momh167-gjp4-8570p:~ # kldstat | grep zfs
 7    1 0xffffffff82837000   554028 openzfs.ko
root@momh167-gjp4-8570p:~ # nano /boot/loader.conf

…

root@momh167-gjp4-8570p:~ # grep zfs /boot/loader.conf
zfs_load="YES"
# openzfs_load="YES"
root@momh167-gjp4-8570p:~ # shutdown -r +2
Shutdown at Wed Aug  7 19:12:31 2019.
shutdown: [pid 19292]
root@momh167-gjp4-8570p:~ #                                                                                
*** System shutdown message from root@momh167-gjp4-8570p ***                 

System going down in 2 minutes
Comment 1 Graham Perrin freebsd_committer freebsd_triage 2019-08-07 18:51:54 UTC
With zfs.ko in lieu of openzfs.ko :

----

root@momh167-gjp4-8570p:~ # date ; uname -v ; uptime
Wed Aug  7 19:23:01 BST 2019
FreeBSD 13.0-CURRENT r350368 GENERIC 
 7:23PM  up 8 mins, 2 users, load averages: 0.14, 0.46, 0.31
root@momh167-gjp4-8570p:~ # kldstat | grep zfs
 6    1 0xffffffff8282b000   458710 zfs.ko
root@momh167-gjp4-8570p:~ # bectl activate r350684
successfully activated boot environment r350684
root@momh167-gjp4-8570p:~ # shutdown -r +2
Shutdown at Wed Aug  7 19:25:39 2019.
shutdown: [pid 2713]
root@momh167-gjp4-8570p:~ #                                                                                
*** System shutdown message from root@momh167-gjp4-8570p ***                 

System going down in 2 minutes
Comment 2 Kyle Evans freebsd_committer freebsd_triage 2019-11-26 02:24:32 UTC
So, I think this is currently an unsupported setup and beadm should be used for OpenZFS testing. bectl/libbe are inherently reliant on the in-tree userland zfs bits corresponding with the kmod being used, so attempting to mix and match like this is a recipe for sadness. =-(

We could perhaps coordinate to produce a bectl build in sysutils/openzfs for those wanting to test it early and/or wanting to keep any existing workflows, but I wouldn't expect major issues from libbe being transplanted over to OpenZFS libs.
Comment 3 Graham Perrin freebsd_committer freebsd_triage 2020-01-11 07:59:02 UTC
(In reply to Kyle Evans from comment #2)

> So, I think this is currently an unsupported setup …

I'm surprised, given the progression from beadm to bectl. 

With bectl integral to FreeBSD, I think it should be supported sooner rather than later. 

https://www.freebsd.org/cgi/man.cgi?query=bectl&manpath=FreeBSD+11.3-RELEASE

https://www.freebsd.org/cgi/man.cgi?query=bectl&manpath=FreeBSD+12.1-RELEASE

>> bectl is based on beadm(1) and was implemented as a project for the 2017
>> Summer of Code, along with libbe(3).

(I can't recall _exactly_ why I stopped using beadm, but I do recall that the reasons were prtactical, not esoteric.)
Comment 4 Graham Perrin freebsd_committer freebsd_triage 2020-01-11 08:59:06 UTC
… sorry, to avoid misunderstanding: 

* the (slight) surprise is from my perspective as a supporter of OpenZFS. 

Whilst I would like to be a reasonably early adopter/tester, my current use case does not _require_ OpenZFS. 

I'm happy to have the software installed but not in use: 

# date ; uname -irsKU
Sat Jan 11 08:49:59 GMT 2020
FreeBSD 13.0-CURRENT GENERIC-NODEBUG 1300072 1300072
# pkg query '%o %v %R' openzfs openzfs-kmod
sysutils/openzfs 2019101600 FreeBSD
sysutils/openzfs-kmod 2019101600 FreeBSD
# pkg info openzfs-kmod | grep nstall
Installed on   : Thu Oct 24 15:03:19 2019 BST
# kldstat | grep zfs
 2    1 0xffffffff82103000   3a6180 zfs.ko
#
Comment 5 Graham Perrin freebsd_committer freebsd_triage 2020-09-04 23:01:10 UTC
bectl: cannot promote 'zroot/ROOT/r364030-OpenZFS2': not a cloned filesystem 

<https://lists.freebsd.org/pipermail/freebsd-current/2020-August/076896.html>
Comment 6 Allan Jude freebsd_committer freebsd_triage 2020-09-21 19:07:50 UTC
Is bectl actually trying to do a promote on the dataset?
Comment 7 Kyle Evans freebsd_committer freebsd_triage 2020-09-21 19:11:36 UTC
(In reply to Allan Jude from comment #6)

Yes, non-temporary activate does promote on activation. AFAIK beadm does the same, but this is libbe built against illumos-libzfs on new OpenZFS.