Bug 156081

Summary: troff falls with troff.core with UTF-8 man with incorrect symbols displaying for kernel UTF-8
Product: Documentation Reporter: Tim <timwork>
Component: Books & ArticlesAssignee: freebsd-doc (Nobody) <doc>
Status: Closed Overcome By Events    
Severity: Affects Only Me    
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   

Description Tim 2011-03-31 09:00:20 UTC
man pages for UTF-8 encoding at least for russian language cause troff
falling with troff.core, and all text is in incorrect character set for
kernel with embeded UTF-8 and UTF-8 locale specified (with xterm tty,
with LANG, etc.)

Fix: 

*roff system not understand standart unicode symbols, but man send to
*roff exactly UTF-8 without any transformations:

Principles of man -d for FreeBSD (with vim as example):
-----------------------
zcat /usr/local/man/ru.UTF-8/man1/vim.1.gz |\
tbl |\
groff -S -Wall -mtty-char -man -Tutf8 -dlocale=ru.UTF-8 |\
col | less
----------------------

In Linux there is preconv between zcat and *roff:

Principles of man -d vim for Ubuntu (as Linux example):
-------------------------------
zcat /usr/share/man/ru/man1/vim.1.gz |\
preconv -e UTF-8 |\
 tbl |\
nroff -mandoc -rLL=97n | less
-------------------------

preconv make \[uXXXX] char sequences for any UTF symbol and these sequences
understand *roff. In FreeBSD such preconv is absent. And this cause title effect.

Usage iconv such a way:
zcat vim.1.gz |iconv -f utf-8 -t koi8-r |nroff -mandoc -Tkoi8-r |iconv -t utf-8 -f koi8-r|less
is workaround of this problem, while nroff work correctly with koi8.
How-To-Repeat: 1. make kernel UTF-8 and UTF-8 locale
2. install vim
3. man vim
Comment 1 Baptiste Daroussin freebsd_committer freebsd_triage 2017-01-07 21:07:46 UTC
On FreeBSD 11+ man uses mandoc which supports utf8 so it is now fixed