Bug 243807

Summary: CURRENT libedit breaks emacs tramp-mode
Product: Base System Reporter: John F. Carr <jfc>
Component: miscAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me CC: ashish, bapt, emacs, jhb, naito.yuichiro, trueos
Priority: ---    
Version: CURRENT   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
emacs-port.patch
none
emacs-port.patch none

Description John F. Carr 2020-02-02 16:42:07 UTC
Emacs tramp-mode uses ssh to move data and commands back and forth to use a local emacs to edit remote files.  It no longer works when the remote system is running FreeBSD CURRENT.  The problem is apparently caused by the new version of libedit.  Running 12.1 in a jail on CURRENT works, but copying in the CURRENT version of libedit reintroduced the problem.

I suspect this will turn out to be undesirable behavior that is not strictly speaking a bug, but the current behavior is suboptimal for reasons other than breaking tramp mode.

Emacs tramp mode opens an ssh connection to a remote machine.  It doesn't want to see input echoed back so it runs

stty -inlcr -onlcr -echo kill '^U' erase '^H'

This doesn't do anything useful if a shell is running in line editing mode (raw) instead of using the tty (cooked).  So tramp falls back to a hack to detect echoed input.  It sends "_echo" followed by a string of backspace characters.  "_echo" is unlikely to appear in program output.

Here is the next command after the initial stty:

_echo^H^H^H^H^Hstty icanon erase ^H cols 32767_echo^H^H^H^H^H

The groups of 5 ^H represent 5 backspace characters and the lone ^H in the middle is a two character sequence for stty.

The terminal output from a 12.1 system is

_echo^H ^H^H ^H^H ^H^H ^H^H ^Hstty icanon erase ^H cols 32767_echo^H ^H^H ^H^H ^H^H ^H^H ^H
#$ 

where again the middle ^H is a two character sequence and the others are control characters. There is a carriage return between the two lines.  "#$ " is the shell prompt set by tramp.

The terminal output from a CURRENT system is

_echo
#$ _ech ^H
#$ _ec ^H
#$ _e ^H
#$ _ ^H
#$  ^Hstty icanon erase ^H cols 32767_echo
#$ stty icanon erase ^H cols 32767_ech ^H
#$ stty icanon erase ^H cols 32767_ec ^H
#$ stty icanon erase ^H cols 32767_e ^H
#$ stty icanon erase ^H cols 32767_ ^H
#$ stty icanon erase ^H cols 32767 ^H
#$ 

with carriage returns between lines.  This does not make sense to emacs.  (And it is the suboptimial behavior I mentioned above, doing much more terminal output than necessary.)

I tried both /bin/sh and /bin/csh as shells and tramp didn't work with either.  I put set +V and set +E in my .profile thinking that would turn off line editing but there was no change.  Probably the shell still takes raw input through libedit even if vi and emacs style editing are off.

I browsed the large diff between 12 and head versions of libedit and didn't see anything obvious.  I don't have a NetBSD system to test on.  (NetBSD is the source of libedit.)

To reproduce run the emacs command
M-x load-library tramp
and try to open a file like
/ssh:user@host:/path

This should work when the remote host is 12.1 and hang if the remote host is CURRENT.
Comment 1 Conrad Meyer freebsd_committer freebsd_triage 2020-02-02 16:59:11 UTC
It seems odd that emacs tries to do this to a terminal emulator instead of opening an auxiliary ssh data channel over the same master connection. I know we can’t change tramp mode overnight but this seems like a misuse of ssh.
Comment 2 John F. Carr 2020-02-06 23:51:18 UTC
I reported this as an emacs bug too and a workaround will be in a future version of the Emacs tramp package: install a temporary .editrc to suppress line editing.
Comment 3 Yuichiro NAITO 2020-05-03 08:40:32 UTC
Hi, this PR insterests to me.

In my additional investigation, NetBSD doesn't have this problem,
because -E option of /bin/sh is not enabled by default.
So NetBSD's /bin/sh doesn't enter edit mode.
The -E option is enabled by default on FreeBSD.

I think emacs should execute "set +E" just before
"stty -inlcr -onlcr -echo kill '^U' erase '^H'".
I believe this will solve this problem.

I'm afraid that changing ~/.editrc affects to another /bin/sh processes
which are created while tramp-mode is running.

Some users are really running /bin/sh as a login shell.
If they run tramp-mode and login to the same server, they will be confused.
Comment 4 John Baldwin freebsd_committer freebsd_triage 2020-07-18 15:50:37 UTC
FYI that this is now broken on 12 stable as well.
Comment 5 Yuichiro NAITO 2020-07-21 00:07:54 UTC
Created attachment 216619 [details]
emacs-port.patch
Comment 6 Yuichiro NAITO 2020-07-21 00:10:24 UTC
(In reply to John Baldwin from comment #4)
Could you try `emacs-port.patch` on your Ports tree?
It will fix this problem.

I'm not sure where this problem is filed in the emacs bug tracking system.
Comment 7 Trenton Schulz 2020-07-22 13:17:54 UTC
(In reply to Yuichiro NAITO from comment #6)

Hi Yuichiro,

It looks like the Emacs bug on debbug as 39399 and is listed here:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39399

Perhaps you want to upstream the patch there as well?
Comment 8 Yuichiro NAITO 2020-07-27 04:21:14 UTC
(In reply to Trenton Schulz from comment #7)
Thanks for the link.

I sent my patch and explained my investigation.
Comment 9 Yuichiro NAITO 2020-07-30 04:19:54 UTC
Created attachment 216873 [details]
emacs-port.patch
Comment 10 Yuichiro NAITO 2020-07-30 04:24:45 UTC
Now tramp mode has been fixed in the tramp repository.

http://git.savannah.gnu.org/cgit/tramp.git/commit/?id=4c04a886b62263efa5d776538e5f6f6e4ff09fc2

My patch was rejected because it was not Posix standard.
But tramp-sh.el already has a same code `set +o emacs +o vi`,
it has been moved up to initialize code block before tramp hangs execution.

This code will be included future release.
If somebody wants to fix current emacs-26.3,
try `emacs-port.patch` that I updated to new one.

Oops, I missed obsoleting previous one!
Comment 11 Yuichiro NAITO 2020-08-03 02:18:47 UTC
emacs@
Could you please apply 'emacs-port.patch' to Ports tree?

I talked about this PR in FreeBSD workshop Tokyo on July 31.
The organizer hrs@ said that he wants to use this patch on FreeBSD Ports.
Comment 12 Ashish SHUKLA freebsd_committer freebsd_triage 2020-08-03 14:00:50 UTC
Thanks, looking at this.
Comment 13 commit-hook freebsd_committer freebsd_triage 2020-08-03 18:22:00 UTC
A commit references this bug:

Author: ashish
Date: Mon Aug  3 18:21:18 UTC 2020
New revision: 544097
URL: https://svnweb.freebsd.org/changeset/ports/544097

Log:
  - Add a patch to make Emacs' TRAMP mode work with 12-STABLE,
    and 13-CURRENT remote hosts

  PR:		243807
  Submitted by:	Yuichiro NAITO <nairo dot yuichiro at gmail dot com>
  Reported by:	John F. Carr <jfc at mit dot edu>

Changes:
  head/editors/emacs/Makefile
  head/editors/emacs/files/patch-lisp_net_tramp-sh.el
Comment 14 Ashish SHUKLA freebsd_committer freebsd_triage 2020-08-03 18:28:56 UTC
The diff seems to have worked as expected. Committed, thanks!