Bug 235456
| Summary: | [maintainer update] japanese/mh: Fix man pages breakage on FreeBSD 11.x and later | ||
|---|---|---|---|
| Product: | Ports & Packages | Reporter: | WATANABE Kazuhiro <CQG00620> |
| Component: | Individual Port(s) | Assignee: | Kurt Jaeger <pi> |
| Status: | Closed FIXED | ||
| Severity: | Affects Some People | CC: | pi, schwarze |
| Priority: | --- | Flags: | pi:
maintainer-feedback+
pi: merge-quarterly+ |
| Version: | Latest | ||
| Hardware: | Any | ||
| OS: | Any | ||
| Attachments: | |||
Created attachment 201672 [details]
A patch for japanese/mh
Created attachment 201673 [details]
The groff output of mh(1) man page without the patch
It is created by the following commands:
/usr/bin/zcat /usr/local/man/man1/mh.1.gz | tbl | groff -S -P-h -Wall -mtty-char -man -Tascii -P-c | col -b > mh.1.groff_without_patch.txt
Created attachment 201674 [details]
The mandoc output of mh(1) man page without the patch
It is created by the following commands:
/usr/bin/zcat /usr/local/man/man1/mh.1.gz | mandoc | col -b > mh.1.mandoc_without_patch.txt
Created attachment 201675 [details]
The mandoc output of mh(1) man page with the patch
Created attachment 201676 [details]
poudriere testport log on FreeBSD 12.0 without the patch
Created attachment 201677 [details]
poudriere testport log on FreeBSD 12.0 with the patch
To committer: Please commit the another patch which is reported in Bug 233463 if it has not been committed. It is highly required on FreeBSD 12.x. A commit references this bug: Author: pi Date: Sun Feb 3 10:51:16 UTC 2019 New revision: 492045 URL: https://svnweb.freebsd.org/changeset/ports/492045 Log: japanese/mh: Fix man pages, runtime error - man pages breakage on FreeBSD 11.x and later - runtime error due to lld 6.0 issue, fixed by using lld 7.0 PR: 233463, 235456 Submitted by: WATANABE Kazuhiro <CQG00620@nifty.ne.jp> (maintainer), nyan MFH: 2019Q1 Changes: head/japanese/mh/Makefile head/japanese/mh/files/patch-conf_doc_me2man.sed head/japanese/mh/files/patch-conf_doc_tmac.h head/japanese/mh/files/patch-conf_makefiles_doc A commit references this bug: Author: pi Date: Sun Feb 3 10:53:45 UTC 2019 New revision: 492046 URL: https://svnweb.freebsd.org/changeset/ports/492046 Log: MFH: r492045 japanese/mh: Fix man pages, runtime error - man pages breakage on FreeBSD 11.x and later - runtime error due to lld 6.0 issue, fixed by using lld 7.0 PR: 233463, 235456 Submitted by: WATANABE Kazuhiro <CQG00620@nifty.ne.jp> (maintainer), nyan Approved by: portmgr (unbreak blanket) Changes: _U branches/2019Q1/ branches/2019Q1/japanese/mh/Makefile branches/2019Q1/japanese/mh/files/patch-conf_doc_me2man.sed branches/2019Q1/japanese/mh/files/patch-conf_doc_tmac.h branches/2019Q1/japanese/mh/files/patch-conf_makefiles_doc From a mandoc(1) perspective, these are five different issues, so i'm commenting on them one by one, rather than all together. Let's start with issue (4). "\*(lq" and "\*(rq" are not macros, but expansions of predefined strings. I tried, but failed to reproduce any problem with them with -current mandoc. They seem to just work out of the box with both -T utf8 and -T ascii, both in small test files and in the context of the nmh(1) manual page itself. Can you provide a minimal test file exhibiting the problem, together with the exact command you type for formatting and the complete output you get from the formatter, and an explanation which output you would like to get instead? I can't reproduce issue (5) either. The unpaddable non-breaking space character "\ " appears to work exactly as expected. Can you provide a minimal test file exhibiting the problem, again with the exact formatting command used and the actual vs. the desired output? Besides, your workaround for issue (5) is likely to cause misformatting of unrelated input lines because it removes the non-breaking property from "\ " regardless of context. Issue (3) was a real bug in mandoc, thanks for reporting it. Note that the whitespace after ".de De" is a tab character rather than a bunch of space characters. Fixed in mandoc.bsd.lv roff.c rev. 1.363: http://mandoc.bsd.lv/cgi-bin/cvsweb/roff.c#rev1.363 Issue (1) has been known as a missing feature for quite some time, see the second entry from the top in: http://mandoc.bsd.lv/cgi-bin/cvsweb/TODO?rev=HEAD It is low priority because manual pages needing it are extremely rare, on the order of about 0.01% of pages in the OpenBSD ports tree. Probably, i will eventually implement it, but i can't give an ETA. Issue (2) is a WONTFIX, the restriction on permissible include paths is in place for security reasons. Even if there weren't security issues with it, the .so request is too fragile for use in manual pages, so upstream should consider getting rid of it. (In reply to Ingo Schwarze from comment #10) > From a mandoc(1) perspective, these are five different issues, so i'm > commenting on them one by one, rather than all together. > > Let's start with issue (4). "\*(lq" and "\*(rq" are not macros, but > expansions of predefined strings. I tried, but failed to reproduce any > problem with them with -current mandoc. They seem to just work out of the > box with both -T utf8 and -T ascii, both in small test files and in the > context of the nmh(1) manual page itself. > > Can you provide a minimal test file exhibiting the problem, together with > the exact command you type for formatting and the complete output you get > from the formatter, and an explanation which output you would like to get > instead? Thanks for your folow-up(s). I've tested with the following procedure on FreeBSD 11.2 and 12.0. It's a sample document (sample.1): .TH EXAMPLE 1 .SH DESCRIPTION \*(lqThis paragraph is quoted.\*(rq .SH "SEE\ ALSO" example\-example(1) ...And run the following commands: $ LC_CTYPE=C mandoc sample.1 > sample_C.catman $ LC_CTYPE=UTF-8 mandoc sample.1 > sample_UTF-8.catman $ LC_CTYPE=ja_JP.eucJP mandoc sample.1 > sample_ja_JP.eucJP.catman $ LC_CTYPE=ja_JP.SJIS mandoc sample.1 > sample_ja_JP.SJIS.catman $ LC_CTYPE=ja_JP.UTF-8 mandoc sample.1 > sample_ja_JP.UTF-8.catman First of all, there is no problem which belongs to the issue (4) and (5) on FreeBSD 12.0, as you suggested. These issues seems to have been fixed. I don't know whether the difference is caused by the different version of mandoc (1.14.3 for 11.2-RELEASE and -stable, 1.14.4 for 12.0-RELEASE and -current). I had only tested these commands on 11.2, and had not considered the possibilities the different results between 11.2 and 12.0... * Correct output All locale listed above on 12.0. C, UTF-8 and ja_JP.UTF-8 locale on 11.2: ---------------------------------------------- DESCRIPTION "This paragraph is quoted." SEE ALSO example-example(1) ---------------------------------------------- (Omit the header and footer) These are the same output except ja_JP.UTF-8 locale. For ja_JP.UTF-8 it is used LEFT DOUBLE QUOTATION MARK and RIGHT DOUBLE QUOTATION MARK (U+201C and U+201D) instead of ASCII double quotes. See the attached files. * Incorrect output ja_JP.eucJP and ja_JP.SJIS locale on 11.2. These are losts double quotes and a escaped whitespace in a section header. It's the ja_JP.eucJP output: ---------------------------------------------- DESCRIPTION This paragraph is quoted. SEALSO example-example(1) ---------------------------------------------- Not SEEALSO but SEALSO. It's the following sequence: S <BS> S E <BS> E E <BS> E <BS> A <BS> A L <BS> L S <BS> S O <BS> O The extra <BS> follows to the last "E". The ja_JP.SJIS output is a bit different. See the attached files. Created attachment 201920 [details]
Correct mandoc output under C locale.
It is created by the following command:
$ LC_CTYPE=C mandoc sample.1 > sample_C.catman
Created attachment 201921 [details]
Correct mandoc output under ja_JP.UTF-8 locale.
It is created by the following command:
$ LC_CTYPE=ja_JP.UTF-8 mandoc sample.1 > sample_ja_JP.UTF-8.catman
Created attachment 201922 [details]
Incorrect mandoc output under ja_JP.eucJP locale on 11.2-RELEASE.
It is created by the following command on FreeBSD 11.2-RELEASE (mandoc 1.14.3):
$ LC_CTYPE=ja_JP.eucJP mandoc sample.1 > sample_ja_JP.eucJP.catman
Created attachment 201923 [details]
Incorrect mandoc output under ja_JP.SJIS locale on 11.2-RELEASE.
It is created by the following command on FreeBSD 11.2-RELEASE (mandoc 1.14.3):
$ LC_CTYPE=ja_JP.SJIS mandoc sample.1 > sample_ja_JP.SJIS.catman
(In reply to Ingo Schwarze from comment #11) > Besides, your workaround for issue (5) is likely to cause misformatting of > unrelated input lines because it removes the non-breaking property from "\ " > regardless of context. Indeed. Most of it appears in a SYNOPSYS section like this: ---------------------------------------------- SYNOPSIS - scan [+folder] [msgs] [-clear] [-noclear] [-form formatfile] [-format - string] [-header] [-noheader] [-width columns] [-reverse] + scan [+folder] [msgs] [-clear] [-noclear] [-form formatfile] + [-format string] [-header] [-noheader] [-width columns] [-reverse] [-noreverse] [-file filename] [-help] ---------------------------------------------- (- == 11.2 with substitution, + == 12.0 without substitution) But on FreeBSD 11.2-RELEASE, it is better to lost the non-breakng property than to lost the escaped whitespace. I will have remained the substitution code until FreeBSD releases that bundles old mandoc (version < 1.14.4) becomes end-of-life. |
On recent FreeBSD, japanese/mh's man pages has been broken with mandoc, the default man page formatter since 11.0-RELEASE. Because the man pages uses some features that is not supported by mandoc. (1) The man pages uses .fc macro which is not supported by mandoc. (2) Some man pages uses .so macro with an absolute file path. Mandoc's .so macro only supports a file in the current directory. (3) Local macro definitions has a trailing whitespace and comment. For example: .de De \" Defaults section Mandoc ignores these definitions unless the trailing component has been removed. Mandoc's '.de' only accepts a macro name. (4) The man pages uses \*(lq and \*(rq macros which is not displayed anything with mandoc on a tty terminal other than UTF-8 locale. (5) Many of whitespace in a quoted strings is escaped by backslash. The escaped character is not displayed with mandoc. For example, "PROFILE\ COMPONENTS" is displayed 'PROFILECOMPONENTS' How to fix: (1) Remove the .fc macro and delimiter character. Replace the padding character to TAB character. (patch-conf_doc_tmac.h and patch-conf_doc_me2man.sed) (2) Replace the .so macro to the specified file contents. (patch-conf_makefiles_doc and an AWK script in Makefile) (3) Remove the trailing components from the macro name (patch-conf_doc_tmac.h). (4) Define local macros such as '.ds lq \&"'. This idea is taken from the top(1) man page. (patch-conf_doc_tmac.h) (5) 's/\\[ 0]/ /g' (patch-conf_doc_me2man.sed)