Bug 211360 - ESC characters are not interpreted by /bin/sh
Summary: ESC characters are not interpreted by /bin/sh
Status: Closed Works As Intended
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.0-BETA1
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
Depends on:
Reported: 2016-07-25 06:49 UTC by Sergey
Modified: 2018-02-12 14:17 UTC (History)
4 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Sergey 2016-07-25 06:49:04 UTC

On FreeBSD 10.X /bin/sh performed interpreting of escape caracters like this one:
export PS1=$'\1\r\1\e[0;31m\1#\1\e[0m\1 '

on FreeBSD 11-beta1 it doesn't anymore. My shell prompt looks like this:


Comment 1 Jilles Tjoelker freebsd_committer 2016-07-29 14:38:19 UTC
The breaking change is in libedit, not sh itself. The libedit in FreeBSD 11.x uses a simple screen image array of wide characters indexed by the position, so characters c with wcwidth(c) == 0 are discarded. This includes not only control characters but also combining characters such as U+0308 combining diaeresis. Unix-like systems usually use precomposed characters such as U+00E4 latin small letter a with diaeresis which do not have this problem but precomposed characters do not exist for all combinations.

The libedit in FreeBSD 10.x uses a simple screen image array of bytes, all of which are assumed to be one cell wide. Therefore, control sequences confuse libedit's idea of the cursor position, and commands longer than the terminal width are not displayed properly if the prompt contains control sequences. Libedit has a feature to make this work (EL_PROMPT_ESC) but sh does not support it and I have not tested it.
Comment 2 James E Keenan 2016-10-18 20:45:17 UTC
Are there any plans to develop a workaround for this problem for FreeBSD-11 and later?

This came up for me -- a relatively new FreeBSD user -- when I went to use the same assignment to $PS1 in my .shrc on a F11 VM as I had used for several months on a F10.3 VM.  The problem was reported to the freebsd-questions mailing list in the thread starting here:  https://lists.freebsd.org/pipermail/freebsd-questions/2016-October/274025.html.

At first I assumed the problem was due to (a) difference between VirtualBox and VMware; and/or (b) my own ignorance.  But as to (b):  I've been setting up colored characters in my command-line prompt on Unix-like OSes since 2001.  And a contributor to that thread indicated he was having the problem in a "bare metal" environment (https://lists.freebsd.org/pipermail/freebsd-questions/2016-October/274041.html).

For reference:  uname -a
FreeBSD kid51.freebsd11 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

I can appreciate that, from one major OS release to the next, there need to be improvements in basic libraries that can have undesired side effects.  But, come on -- people have had color in their prompts for decades.  Why give it up now?  This is clearly a regression in user-friendliness.

What will it take to develop a workaround?

Thank you very much.
Jim Keenan
Comment 3 David Guyot 2016-10-20 09:27:19 UTC
It seems I'm also affected by this bug, but running fish: I was using a 10.3-RELEASE FreeBSD with the packaged fish 2.2.0 installed, without customization, and upgraded my system and packages to 11.0-RELEASE. fish were upgraded to 2.3.1, and it lost support of arrows and backspace keys: whenever I try to use them, I get this:
# Up arrow for autocompletion with the last command
penegal@Aethelthryth-FreeBSD~/usr/home/penegal> [A
# Right arrow for autocompletion with the last similar command, then left arrow to correct input
penegal@Aethelthryth-FreeBSD~/usr/home/penegal> ls[C[D

Tab still functions as usual; backspace produces no result, so I have to Ctrl+C to erase the line and restart the typing from the start. As I also use fish as the root shell, the problem persists when using sudo su, but not with su -; the problem is also present when using fish from vt, and the su tweak works all the same. bash doesn't have this problem and responds as usual to these keys, as do fish over SSH connections, so it's not keyboard-related.

James Keenan and another guy on the ML drew my attention to this report, which is disturbingly similar to my problem. Hope it is and that I'm not simply making noise.
Comment 4 David Guyot 2016-10-24 12:29:30 UTC
It seems that this problem affects fish deeper than just with keystrokes: when on a buggy fish CLI, shell variable setting is buggy and partly broken, but, when on a fully functional fish, the problem disappears and shell variable setting works as expected. I don't understand how the keystrokes and the variable problems are related, but they manifestly are. I don't give more details on these variable setting problems as they seem irrelevant, but, if they are relevant, feel free to ask me.
Comment 5 David Guyot 2016-10-25 14:20:53 UTC
Forget my last messages, the correct bug ID was #213013; sorry for the noise.
Comment 6 Graham Perrin 2016-10-30 19:51:06 UTC
Adding myself to the list of users of fish who may be affected by this bug. Referred from https://discourse.trueos.org/t/105/12

My environment: TrueOS Desktop. 

$ date ; freebsd-version ; uname -a
Sun 30 Oct 2016 19:49:03 GMT
FreeBSD momh167-gjp4-hpelitebook850g2-trueos.university.brighton.ac.uk 12.0-CURRENT FreeBSD 12.0-CURRENT #14 1717ae3(drm-next-4.7): Fri Oct 28 13:49:49 UTC 2016     root@gauntlet:/usr/obj/usr/src/sys/GENERIC  amd64
Comment 7 Luca Brassia 2018-02-12 14:17:09 UTC
Same bug's here.

FreeBSD 11.1-STABLE (amd64) on PROXMOX KVM with default Display, accessed with noVNC.