Bug 214545 - freebsd-update erroneously reports mandoc.db databases need updating
Summary: freebsd-update erroneously reports mandoc.db databases need updating
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.0-RELEASE
Hardware: Any Any
: --- Affects Some People
Assignee: Ed Maste
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-15 20:12 UTC by Jason Stames
Modified: 2022-12-14 16:12 UTC (History)
21 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jason Stames 2016-11-15 20:12:31 UTC
After patching my servers, I have noticed that after a couple of days, freebsd-update thinks the following files still need to be patched which cause additional "[server_name] security updates" emails to be generated each morning.

/usr/share/man/mandoc.db
/usr/share/openssl/man/mandoc.db

I have been told that these databases are periodically updated, which means their hash values will change (https://forums.freebsd.org/threads/58553/#post-334803). Should freebsd-update ignore changes to these files or are these files in the wrong location?

I have also noticed that the "freebsd-update IDS" command produces hash warnings for the above listed files three different times. All other files are only listed once.

<output>
$ freebsd-version
11.0-RELEASE-p3

$ sudo freebsd-update fetch
Password:
Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 11.0-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be updated as part of updating to 11.0-RELEASE-p3:
/usr/share/man/mandoc.db
/usr/share/openssl/man/mandoc.db

$ sudo freebsd-update IDS
Password:
Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 11.0-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
[output from other files removed]
/usr/share/man/mandoc.db has SHA256 hash 5f346b11b7ba8281a33963ead7bf914816cb4c274af2b5c5735da2f6183cc8eb, but should have SHA256 hash 095526d080f7f94768d14d8fd52ea1411d478cf0c12c82f693540266ea1bf3a5.
/usr/share/man/mandoc.db has SHA256 hash 5f346b11b7ba8281a33963ead7bf914816cb4c274af2b5c5735da2f6183cc8eb, but should have SHA256 hash 17c15936fb3ff4879ebd066c7210b2dca671f2d864852523c2e76b4853858420.
/usr/share/man/mandoc.db has SHA256 hash 5f346b11b7ba8281a33963ead7bf914816cb4c274af2b5c5735da2f6183cc8eb, but should have SHA256 hash 8f9f39a2355f27840219be2da88ae5e77b5142fbab677ea2c43b342c2e7f7394.
/usr/share/openssl/man/mandoc.db has SHA256 hash e2f1ac0efeedc35a6076f84831cafed08a8705a08528870f43d15b5b70e26dd7, but should have SHA256 hash 3def5f84de882f7031351edcc888e8680058e494fad532f5e4aaf1680eea8fa7.
/usr/share/openssl/man/mandoc.db has SHA256 hash e2f1ac0efeedc35a6076f84831cafed08a8705a08528870f43d15b5b70e26dd7, but should have SHA256 hash 3f6e5ca36cb7098d89430a523e501b50b9437e69ef0bc19fc43d36eb0cf73f74.
/usr/share/openssl/man/mandoc.db has SHA256 hash e2f1ac0efeedc35a6076f84831cafed08a8705a08528870f43d15b5b70e26dd7, but should have SHA256 hash 814d38c9b5ea4665fc1275707332c8c093650a3c67f5946f5cf5aa986749f552.
</output>
Comment 1 Glen Barber freebsd_committer freebsd_triage 2016-11-15 20:23:17 UTC
This is planned for an Errata Notice, with the caveat that mandoc.db will be updated again (one final time), after which subsequent updates to the file will not occur.
Comment 2 Ed Maste freebsd_committer freebsd_triage 2016-11-26 03:40:09 UTC
> I have been told that these databases are periodically updated

FYI, it's not that they are updated (which would justify including them in a SA or EN), it's that the generated db files change when rebuilt even from the same, unchanged, input.

As of r309183 in stable/11 they won't change on rebuild so as Glen says this change needs to be merged to the releng/11.0 branch, and then the next SA or EN after that will be the last one that has an extraneous change in these files.
Comment 3 Petr Fischer 2016-12-19 15:24:13 UTC
+1 - issue still exist in 11.0-RELEASE-p5
Comment 4 Ed Maste freebsd_committer freebsd_triage 2016-12-19 15:28:15 UTC
We still need to bring r309183 from stable/11 into releng/11.0 in an Errata Notice.
Comment 5 lumiwa 2017-01-21 21:58:13 UTC
I have still on 11.0-RELEASE-p7 (amd64)
Comment 6 everytrash 2017-01-30 13:04:15 UTC
Same error! (11.0-RELEASE-p7 (amd64))
Comment 7 Petr Fischer 2017-01-30 19:07:45 UTC
You also have "freebsd-update cron" in cron on production servers? Nice feature.

Time to change "Affects Some People" -> "Affects Many People"
Comment 8 Petr Fischer 2017-02-25 23:24:02 UTC
After errata with mandoc info, I upgraded to 11.0-RELEASE-p8.

Today, my cron freebsd-update send me message:

The following files will be updated as part of updating to 11.0-RELEASE-p8:
/usr/share/man/mandoc.db
/usr/share/openssl/man/mandoc.db

Again.
Comment 9 Xin LI freebsd_committer freebsd_triage 2017-02-26 07:50:15 UTC
Resolved by FreeBSD-EN-17:04.mandoc.
Comment 10 Xin LI freebsd_committer freebsd_triage 2017-02-26 07:51:19 UTC
(In reply to Petr Fischer from comment #8)
The ordering have been changed by the recent mandoc and this is therefore expected.
Comment 11 Ed Maste freebsd_committer freebsd_triage 2017-02-27 15:36:53 UTC
See PR217389 - we are using the wrong makewhatis when generating mandoc.db.
Comment 12 Pierre Guinoiseau 2017-03-06 06:09:17 UTC
It doesn't seem to be fixed, I'm still receiving updates for those files a week after upgrading to 11.0-RELEASE-p8:

The following files will be updated as part of updating to 11.0-RELEASE-p8:
/usr/share/man/mandoc.db
/usr/share/openssl/man/mandoc.db
Comment 13 Petr Fischer 2017-03-06 17:35:20 UTC
Yes. This issue is definitely not FIXED. Question is, who reads comments in closed issues :(
Comment 14 Jason Stames 2017-03-06 20:59:24 UTC
Petr,

I will re-open this bug if bug 217389 is marked as closed/fixed and this issue still persists.
Comment 15 Josh 2017-04-22 12:03:44 UTC
This is still happening after updating to 11.0-RELEASE-p9.

Wouldn't it be more correct to leave this as open and add a "depends on" for the other bug, then verify after that bug is fixed?
Comment 16 Ed Maste freebsd_committer freebsd_triage 2017-04-24 17:13:12 UTC
Reopen, the issue in the headline still exists although one bug that contributed to it has been fixed.
Comment 17 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2017-04-25 10:58:01 UTC
Can someone please update sba and sbi so they have the correct version of mandoc, and / or make mandoc a build tool?
Comment 18 Ed Maste freebsd_committer freebsd_triage 2017-04-25 20:37:35 UTC
mandoc is a build tool, the problem is that it's run by installworld (etc/Makefile afterinstall -> share/man/Makefile makedb) which uses existing host tools. These are normally copied aside before use (ITOOLS); makewhatis is not included in that list but the problem would persist even if it were.

I have a review open to have freebsd-update invoke makewhatis after installing the updates: https://reviews.freebsd.org/D10482

By itself this would not solve the issue, but would allow us to stop shipping mandoc.db.
Comment 19 Josh 2017-06-10 10:32:58 UTC
This is still happening after 11.0-RELEASE-p10.

A workaround that seems to work is excluding these files via /etc/freebsd-update.conf:

IgnorePaths /usr/share/man/mandoc.db /usr/share/openssl/man/mandoc.db

Of course, this change means they won't get updated when they are actually supposed to, and I'm not sure of any other consequences or side-effects.

For me it's better than getting a crapload of email all the time from systems that think they need updates when they really don't (or worse, automatically updating and rebooting).
Comment 20 Petr Fischer 2018-08-12 18:58:41 UTC
Still happening even on 11.2-RELEASE-p1
Comment 21 commit-hook freebsd_committer freebsd_triage 2019-01-30 19:20:11 UTC
A commit references this bug:

Author: emaste
Date: Wed Jan 30 19:19:14 UTC 2019
New revision: 343589
URL: https://svnweb.freebsd.org/changeset/base/343589

Log:
  freebsd-update: regenerate man page database after update

  These are currently not reproducible because they're built by the
  makewhatis on the freebsd-update build host, not the one in the tree.
  Regenerate after update, and later we can avoid including it in
  freebsd-update data.

  PR:		214545, 217389
  Reviewed by:	delphij
  MFC after:	1 month
  Sponsored by:	The FreeBSD Foundation
  Differential Revision:	https://reviews.freebsd.org/D10482

Changes:
  head/usr.sbin/freebsd-update/freebsd-update.sh
Comment 22 commit-hook freebsd_committer freebsd_triage 2019-03-05 18:10:54 UTC
A commit references this bug:

Author: emaste
Date: Tue Mar  5 18:10:07 UTC 2019
New revision: 344799
URL: https://svnweb.freebsd.org/changeset/base/344799

Log:
  MFC r343589: freebsd-update: regenerate man page database after update

  These are currently not reproducible because they're built by the
  makewhatis on the freebsd-update build host, not the one in the tree.
  Regenerate after update, and later we can avoid including it in
  freebsd-update data.

  PR:		214545, 217389
  Reviewed by:	delphij
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  stable/12/
  stable/12/usr.sbin/freebsd-update/freebsd-update.sh
Comment 23 commit-hook freebsd_committer freebsd_triage 2019-03-05 18:12:06 UTC
A commit references this bug:

Author: emaste
Date: Tue Mar  5 18:11:37 UTC 2019
New revision: 344800
URL: https://svnweb.freebsd.org/changeset/base/344800

Log:
  MFC r343589: freebsd-update: regenerate man page database after update

  These are currently not reproducible because they're built by the
  makewhatis on the freebsd-update build host, not the one in the tree.
  Regenerate after update, and later we can avoid including it in
  freebsd-update data.

  PR:		214545, 217389
  Reviewed by:	delphij
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  stable/11/
  stable/11/usr.sbin/freebsd-update/freebsd-update.sh
Comment 24 hdeleeuw 2020-01-16 10:53:16 UTC
Still happens in 11.5-RELEASE-p5
Comment 25 Ed Maste freebsd_committer freebsd_triage 2020-08-25 23:54:21 UTC
All supported releases now have r343589, and mandoc.db can be excluded from future updates.
Comment 26 Ed Maste freebsd_committer freebsd_triage 2022-12-14 16:12:13 UTC
mandoc.db is now excluded from updates

https://github.com/freebsd/freebsd-update-build/commit/c4bad8d73b47388ce44045a1f531b24258a7c207