Bug 252417

Summary:	mergemaster: deprecation and removal plan
Product:	Base System	Reporter:	Enji Cooper <ngie>
Component:	bin	Assignee:	Enji Cooper <ngie>
Status:	In Progress ---
Severity:	Affects Some People	CC:	0mp, antoine, chris, emaste, grahamperrin, jakub_lach, jhb, vsasjason, vvd, yasu, zarychtam
Priority:	---
Version:	CURRENT
Hardware:	Any
OS:	Any
See Also:	https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252132 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247519 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252054 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274967 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233872 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202664 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192577 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144442 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278412

Description Enji Cooper freebsd_committer

2021-01-04 19:24:35 UTC

mergemaster(8) has been deprecated for several releases and its replacement--etcupdate(8)--has been present in production systems since at least 9.x/10.x.

Reasons for deprecating/removing mergemaster(8) are as follows:
1. It doesn't have a maintainer.
2. It's not being actively developed.
3. It was made fragile with the move from svn to git (it relies on RCS tags like $FreeBSD$, which are no longer incremented when touched, requiring manual human intervention).

It's past time to replace mergemaster(8) with etcupdate(8).

This bug will track the following items:
* Reconciling all missing features (if any) between etcupdate(8) and mergemaster(8) so the transition can be relatively painless.
* Replacing mergemaster(8) with etcupdate(8) in documentation (in base and docs). Affected areas [in base] in my svn checkout (which is admittedly dated) appear to be as follows:
$ grep -rIl mergemaster .
contrib/sendmail/FREEBSD-upgrade
release/tools/vmimage.subr
release/scripts/mm-mtree.sh
release/Makefile
share/man/man8/beinstall.8
share/man/man7/build.7
targets/pseudo/userland/Makefile.depend
tools/build/beinstall.sh
usr.sbin/etcupdate/etcupdate.8
usr.sbin/Makefile
usr.sbin/mergemaster/...
Makefile
UPDATING
$

* Adding appropriate messaging in the utility to note the deprecation/removal plan.

Comment 1 Vladimir Druzenko freebsd_committer

2021-01-05 00:52:03 UTC

Please, write an "article/howto" with description of the migration from mergemaster to etcupdate.
Fore example I want to see an equivalent of the "mergemaster -iUF".

Comment 2 jakub_lach 2021-02-01 21:36:23 UTC

(In reply to VVD from comment #1)

Yes, I'm currently lookig for a etcupdate drop-in replacement for 'mergemaster -Ui'

Comment 3 John Baldwin freebsd_committer

2021-02-02 16:43:51 UTC

etcupdate's default behavior is to always do the equivalent of -Ui. etcupdate keeps the previous "unmodified" version of each /etc file around in /var/db/etcupdate/current (this is what enables it to do 'etcupdate diff' for example as it runs a diff of files in /etc against those unmodified copies). On each update, etcupdate generates a new tree and then does a 3-way merge copying over any new files or updating changed files for which the file in /etc matches the old version (equivalent to -Ui). In addition, if you have local changes in a file in /etc, it uses diff3 to generate a 3-way merge similar to 'svn up' or 'cvs up', etc. and if the changes do not conflict, it installs the updated file with the new changes merged in but local changes kept. If the changes conflict, it saves a version with conflict markers for the user to manually resolve. At the end of 'etcupdate', it lists any conflicts you manually need to resolve. For those, you run 'etcupdate resolve' which gives you options to either keep your version, install the new version only, or invoke an editor to resolve the changes.

In my experience, manual conflicts are quite rare and most of the time after an upgrade you just run 'etcupdate' after 'make installworld' and it auto-updates any files you have without requiring any user interaction.

Brand new installs come with etcupdate bootstrapped, so if you install a new VM, etc. you can use 'etcupdate' after your first 'make installworld' and it will work fine.

If you have done several worlds with 'mergemaster' instead, then etcupdate's database in /var/db/etcupdate is stale. You can overwrite the database by doing 'etcupdate extract' _before_ you do an 'svn up' or 'git pull'. To check to see what local diffs etcupdate thinks you have, run 'etcupdate diff' (it outputs a patch).

Comment 4 jakub_lach 2021-02-03 14:41:13 UTC

(In reply to John Baldwin from comment #3)

Great summary, thanks. Only thing I feel is not stressed enough - before first use of etcupdate you need to run 'etcupdate extract' to bootstrap etcupdate _before_ updating /usr/src (you need to have src aligned to the already running system).

Comment 5 Antoine 2021-03-21 13:18:25 UTC

Upgrading jail/etc was simple with mergemaster -xxx -D /jail_root_filesystem/ from inside the host containing /usr/src.
Now it seems that /usr/src has to be mounted then etcupdate executed from inside each jail. 
Any other easiest way ?

Comment 6 Christos Chatzaras 2021-03-21 13:30:03 UTC

(In reply to Antoine from comment #5)

I use these commands:

cd /usr/src
etcupdate -p -D /home/jail/redis
make installworld DESTDIR=/home/jail/redis
etcupdate -D /home/jail/redis
make -DBATCH_DELETE_OLD_FILES delete-old DESTDIR=/home/jail/redis
make -DBATCH_DELETE_OLD_FILES delete-old-libs DESTDIR=/home/jail/redis

Comment 7 commit-hook freebsd_committer

2021-11-10 08:28:28 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e641c29a006ae9f528f196386052355b42a53d75

commit e641c29a006ae9f528f196386052355b42a53d75
Author:     Dries Michiels <driesm@FreeBSD.org>
AuthorDate: 2021-02-07 19:25:31 +0000
Commit:     Dries Michiels <driesm@FreeBSD.org>
CommitDate: 2021-11-10 08:18:42 +0000

    UPDATING: Change update procedure to use etcupdate(8) over mergemaster(8)

    This commit aligns the steps in UPDATING with the steps from the
    handbook which already prefers etcupdate(8). While here also remove a
    dubious comment.

    PR:                     252417
    Reviewed by:            ceri
    Approved by:            philip (mentor), imp
    Differential Revision:  https://reviews.freebsd.org/D28062

 UPDATING | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

Comment 8 Graham Perrin freebsd_committer

2021-11-11 07:13:26 UTC

(In reply to commit-hook from comment #7)

> https://cgit.FreeBSD.org/src/commit/?id=e641c29a006ae9f528f196386052355b42a53d75


Thank you, and a suggestion: also, insert a brief entry at 
<https://cgit.freebsd.org/src/tree/UPDATING?id=e641c29a006ae9f528f196386052355b42a53d75#n29> (most recent line 29): 



20211111:
        Where previously the COMMON ITEMS section showed use of 
        mergemaster(8), there's now use of etcupdate(8).



– or words to that effect. For this transition to be not overlooked. 

Thanks

Comment 9 Mateusz Piotrowski freebsd_committer

2021-11-23 10:46:48 UTC

*** Bug 255184 has been marked as a duplicate of this bug. ***

Comment 10 Ed Maste freebsd_committer

2023-03-03 01:44:36 UTC

commit 398b12691b4f1e7a73afca4c051c182bfcaf9671 (HEAD -> main)
Author: Ed Maste <emaste@FreeBSD.org>
Date:   Wed Jan 22 16:04:03 2020 -0500

    mergemaster: add deprecation notice to man page
    
    Reviewed by:    bapt, gjb, kevans (all earlier)
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D23327

Comment 11 commit-hook freebsd_committer

2023-03-06 00:32:18 UTC

A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=ab53efd14237b71c3c0e4e212ada6526ef14402c

commit ab53efd14237b71c3c0e4e212ada6526ef14402c
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2020-01-22 21:04:03 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-03-06 00:30:57 +0000

    mergemaster: add deprecation notice to man page

    PR:             252417
    Reviewed by:    bapt, gjb, kevans (all earlier)
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D23327

    (cherry picked from commit 398b12691b4f1e7a73afca4c051c182bfcaf9671)

 usr.sbin/mergemaster/mergemaster.8 | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

Comment 12 commit-hook freebsd_committer

2023-03-06 13:03:34 UTC

A commit in branch releng/13.2 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5fa16e3c50c5083acd52a61a76f726c759c82ffd

commit 5fa16e3c50c5083acd52a61a76f726c759c82ffd
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2020-01-22 21:04:03 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-03-06 13:02:01 +0000

    mergemaster: add deprecation notice to man page

    PR:             252417
    Reviewed by:    bapt, gjb, kevans (all earlier)
    Approved by:    re (cperciva)
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D23327

    (cherry picked from commit 398b12691b4f1e7a73afca4c051c182bfcaf9671)
    (cherry picked from commit ab53efd14237b71c3c0e4e212ada6526ef14402c)

 usr.sbin/mergemaster/mergemaster.8 | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

Comment 13 Graham Perrin freebsd_committer

2023-03-11 19:23:42 UTC

(In reply to commit-hook from comment #12)

Mention 5fa16e3c50c5083acd52a61a76f726c759c82ffd in release notes?

Comment 14 Ed Maste freebsd_committer

2023-03-12 03:05:44 UTC

(In reply to Graham Perrin from comment #13)
See https://reviews.freebsd.org/D39016

Comment 15 commit-hook freebsd_committer

2023-03-23 12:51:12 UTC

A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8ebda77aa1de531003e6aa123db0a2b35f618cd1

commit 8ebda77aa1de531003e6aa123db0a2b35f618cd1
Author:     Ed Maste <emaste@FreeBSD.org>
AuthorDate: 2020-01-22 21:04:03 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2023-03-23 12:49:43 +0000

    mergemaster: add deprecation notice to man page

    PR:             252417
    Reviewed by:    bapt, gjb, kevans (all earlier)
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D23327

    (cherry picked from commit 398b12691b4f1e7a73afca4c051c182bfcaf9671)
    (cherry picked from commit ab53efd14237b71c3c0e4e212ada6526ef14402c)

 usr.sbin/mergemaster/mergemaster.8 | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

Comment 16 Ed Maste freebsd_committer

2023-05-24 17:46:58 UTC

Mergemaster doc references to update:

documentation/content/en/books/handbook/cutting-edge/_index.adoc
documentation/content/en/books/handbook/jails/_index.adoc
website/content/en/advocacy/whyusefreebsd.adoc

Comment 17 commit-hook freebsd_committer

2024-04-17 17:07:04 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/doc/commit/?id=e5f0ee69e64a568dab4ee71d994d6231c79f6b21

commit e5f0ee69e64a568dab4ee71d994d6231c79f6b21
Author:     Minsoo Choo <minsoochoo0122@proton.me>
AuthorDate: 2024-04-17 16:41:18 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2024-04-17 17:06:09 +0000

    handbook: Remove mergemaster usage in update process

    PR:             252417
    Reviewed by:    emaste
    Differential Revision: https://reviews.freebsd.org/D43119

 .../en/books/handbook/cutting-edge/_index.adoc     | 29 +++-------------------
 1 file changed, 3 insertions(+), 26 deletions(-)

Comment 18 Ed Maste freebsd_committer

2024-04-17 17:13:37 UTC

documentation/content/en/books/handbook/jails/_index.adoc was addressed by 612b7cc1721224c494c5b2600188e1508bb5611b

Comment 19 Ed Maste freebsd_committer

2024-04-17 17:32:05 UTC

website/content/en/advocacy/whyusefreebsd.adoc was addressed by bba34a39cd8e017b27f0fa144f09d2b7e8f6e2bb