In the default configuration of x11/xterm UTF-8 characters cause tabs
that occur after them to be misaligned according to their byte-length:
$ printf 'a\tb\n\302\254\tb\n'
$ printf 'a\tb\n\302\254\tb\n'
Calling `stty tabs' or `stty tab0' beforehand results in the expected
Confirmed. It's not isolated to the latest release. I've downgraded to 332 and observed the same behavior.
I've cc'ed Thomas from upstream.
I asked yesterday if the stty hardware tabstops were used or not, and Drake replied that the misbehavior was seen using the software (terminal-driver) tabs.
I don't see anything in my #333 changes which hints at this area.
(In reply to dickey from comment #2)
It happens in 332 as well. It was not introduced in changes between 332 and 333.
I'll have to investigate to pinpoint the problem. The example
appears to show that the terminal driver is assuming the character
0xac is using more than one cell. That would be the same effect
as if the print statement were modified to something like this:
printf 'a\tb\n\302\254\tb\n\302\254 b\n'
that is, 7 spaces between the two characters. If xterm's showing
that incorrectly, that's a problem with xterm (or the locale settings
which it uses for width computation).
Drake said this happened in 12-Current (which I've not installed).
ktrace(1) shows that x11/xterm does not receive any tab characters:
75446 xterm GIO fd 4 read 24 bytes
0x0000 6120 2020 2020 2020 620d 0ac2 ac20 2020 |a b.... |
0x0010 2020 2062 0d0a 2420 | b..$ |
Other terminals (vt(4) & x11/sterm) do receive tab characters, and
x11/xterm does when `stty tabs' is set. I have tried changing the
`c_oflags' setting in the x11/xterm source and changing various
tab-related settings in the `termcap' database but the default
behavior remained the same.
(In reply to dickey from comment #4)
It also happens in 11.2-RELEASE.
Drake's octal dump in comment #5 shows 6 spaces.
That has to be coming from the terminal driver,
which apparently is confused about the width of the
character. If it were sending 7 spaces, as I commented
before, and if xterm miscounted, that would be a bug in xterm.
Offhand, the only "recent" changes that I recall making in xterm
deal with the wrapping behavior, and the treatment of ambiguous-width
characters. This doesn't fall into that category (but there's always
the potential for bugs).
The default setting for tabs has long been to use the terminal driver,
since it's safer than assuming that some terminal handles hardware tabs,
and that the tab-stops have been initialized properly. (A quick check
of (u)rxvt, Terminal.app, iTerm2 shows the same default, for instance).
I can add something to the ttyMode resource to simplify initialization,
but that's a feature, not a bug-fix.
(In reply to dickey from comment #7)
Thank you for your analysis. I've added ed@ cc. Maybe he can shed some light into the terminal driver related issue.
This can be attributed to the fact that FreeBSD's TTY layer does not implement termios IUTF8, right?
iutf8 applies to input characters.
The example isn't explicit, but I read it as output behavior.
Drake may clarify.
(In reply to Ed Schouten from comment #9)
This happens because the configuration of x11/xterm in ports relies
on FreeBSD's TTY layer to expand tabs, but the TTY layer expands tabs
based on the column number which is updated with the byte count and so
does not support multi-byte characters.
To further clarify: this is about TTY output, and this would not be an
issue if FreeBSD's TTY layer supported multi-byte characters or if the
port was patched to disable `TAB3' when applicable.
Created attachment 196879 [details]
patch setting oflags to TAB0 by default
Screenshot showing stty behaviour
Created attachment 196880 [details]
Thank you for clarifying. I've seen that other terminals (gnome-terminal, xfce4-terminal) already use tab0 by default.
The attached quick patch seems to do the trick (see screenshot). Thomas, would it be acceptable to to add a ifdef __freebsd__ macro in main.c setting d_tio.c_oflag to TAB0?
A patch to the initialization seems like clutter.
If I added a tabs keyword to the ttyModes resource parsing,
then that could be patched in the system's app-defaults file.
Sounds good to me.
A commit references this bug:
Date: Thu Sep 20 09:19:07 UTC 2018
New revision: 480159
- Update to 336
- Pacify portlint
- Since FreeBSD terminal drivers are not aware of UTF-8, change default ttyModes
to tabs (tab0).
PR: 229682 (based on)