In the default configuration of x11/xterm UTF-8 characters cause tabs that occur after them to be misaligned according to their byte-length: Observed behavior: $ printf 'a\tb\n\302\254\tb\n' a b ¬ b Expected behavior: $ printf 'a\tb\n\302\254\tb\n' a b ¬ b Calling `stty tabs' or `stty tab0' beforehand results in the expected behaviour.
Confirmed. It's not isolated to the latest release. I've downgraded to 332 and observed the same behavior. I've cc'ed Thomas from upstream.
I asked yesterday if the stty hardware tabstops were used or not, and Drake replied that the misbehavior was seen using the software (terminal-driver) tabs. I don't see anything in my #333 changes which hints at this area.
(In reply to dickey from comment #2) It happens in 332 as well. It was not introduced in changes between 332 and 333.
I'll have to investigate to pinpoint the problem. The example appears to show that the terminal driver is assuming the character 0xac is using more than one cell. That would be the same effect as if the print statement were modified to something like this: printf 'a\tb\n\302\254\tb\n\302\254 b\n' that is, 7 spaces between the two characters. If xterm's showing that incorrectly, that's a problem with xterm (or the locale settings which it uses for width computation). Drake said this happened in 12-Current (which I've not installed).
ktrace(1) shows that x11/xterm does not receive any tab characters: 75446 xterm GIO fd 4 read 24 bytes 0x0000 6120 2020 2020 2020 620d 0ac2 ac20 2020 |a b.... | 0x0010 2020 2062 0d0a 2420 | b..$ | Other terminals (vt(4) & x11/sterm) do receive tab characters, and x11/xterm does when `stty tabs' is set. I have tried changing the `c_oflags' setting in the x11/xterm source and changing various tab-related settings in the `termcap' database but the default behavior remained the same.
(In reply to dickey from comment #4) It also happens in 11.2-RELEASE.
Drake's octal dump in comment #5 shows 6 spaces. That has to be coming from the terminal driver, which apparently is confused about the width of the character. If it were sending 7 spaces, as I commented before, and if xterm miscounted, that would be a bug in xterm. Offhand, the only "recent" changes that I recall making in xterm deal with the wrapping behavior, and the treatment of ambiguous-width characters. This doesn't fall into that category (but there's always the potential for bugs). The default setting for tabs has long been to use the terminal driver, since it's safer than assuming that some terminal handles hardware tabs, and that the tab-stops have been initialized properly. (A quick check of (u)rxvt, Terminal.app, iTerm2 shows the same default, for instance). I can add something to the ttyMode resource to simplify initialization, but that's a feature, not a bug-fix.
(In reply to dickey from comment #7) Thank you for your analysis. I've added ed@ cc. Maybe he can shed some light into the terminal driver related issue.
This can be attributed to the fact that FreeBSD's TTY layer does not implement termios IUTF8, right?
iutf8 applies to input characters. The example isn't explicit, but I read it as output behavior. Drake may clarify.
(In reply to Ed Schouten from comment #9) This happens because the configuration of x11/xterm in ports relies on FreeBSD's TTY layer to expand tabs, but the TTY layer expands tabs based on the column number which is updated with the byte count and so does not support multi-byte characters. To further clarify: this is about TTY output, and this would not be an issue if FreeBSD's TTY layer supported multi-byte characters or if the port was patched to disable `TAB3' when applicable.
Created attachment 196879 [details] patch setting oflags to TAB0 by default Screenshot showing stty behaviour
Created attachment 196880 [details] Screenshot Thank you for clarifying. I've seen that other terminals (gnome-terminal, xfce4-terminal) already use tab0 by default. The attached quick patch seems to do the trick (see screenshot). Thomas, would it be acceptable to to add a ifdef __freebsd__ macro in main.c setting d_tio.c_oflag to TAB0?
A patch to the initialization seems like clutter. If I added a tabs keyword to the ttyModes resource parsing, then that could be patched in the system's app-defaults file.
Sounds good to me.
A commit references this bug: Author: ehaupt Date: Thu Sep 20 09:19:07 UTC 2018 New revision: 480159 URL: https://svnweb.freebsd.org/changeset/ports/480159 Log: - Update to 336 - Pacify portlint - Since FreeBSD terminal drivers are not aware of UTF-8, change default ttyModes to tabs (tab0). PR: 229682 (based on) Changes: head/x11/xterm/Makefile head/x11/xterm/distinfo head/x11/xterm/files/ head/x11/xterm/files/patch-UXTerm.ad