Bug 246029 - base termcap(5) is missing screen.* definitions
Summary: base termcap(5) is missing screen.* definitions
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Cy Schubert
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-04-29 13:34 UTC by Victor Sudakov
Modified: 2020-07-12 13:11 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (cy)


Attachments
New termcap entries (12.89 KB, patch)
2020-05-04 04:25 UTC, Cy Schubert
no flags Details | Diff
Add to end of termcap (801 bytes, text/plain)
2020-05-04 17:18 UTC, Cy Schubert
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Victor Sudakov 2020-04-29 13:34:21 UTC
The recent versions of screen use the "screen.xterm-256color" terminal type by default inside screen's windows. This terminal type is not known to FreeBSD (as recent as 12.1) which causes messages like this:

csh: using dumb terminal settings.

inside screen's windows, and failure of screen-oriented software (text editors etc) to work properly.

Workaround: add something like "screen.xterm-256color:tc=screen-256color" to /usr/share/misc/termcap and recompile it with cap_mkdb.

I really don't know if this should be considered a bug in sysutils/screen or a deficiency in FreeBSD's termcap in the base system.
Comment 1 Cy Schubert freebsd_committer 2020-04-29 14:09:44 UTC
Do you mean this one?

screen-256color|VT 100/ANSI X3.64 terminal with 256 colors:\
        :Co#256:pa#32767:\
        :AB=\E[48;5;%dm:AF=\E[38;5;%dm:tc=screen:
Comment 3 Victor Sudakov 2020-04-29 14:24:01 UTC
(In reply to Cy Schubert from comment #1)
I mean this one:


[vas@vas ~] grep term ~/.screenrc
term screen-256color
[vas@vas ~] echo $TERM
screen.xterm-256color
[vas@vas ~] 


Yes, it's adding ".xterm" to it.
Comment 4 Victor Sudakov 2020-04-29 14:29:36 UTC
See also https://bugzilla.redhat.com/show_bug.cgi?id=1357981
Comment 5 Cy Schubert freebsd_committer 2020-04-29 14:47:54 UTC
Red Hat does their own thing. We are not obligated to follow Red Hat's lead. They maintain their own set of patches and modifications in their SRPM files. Download their screen srpm and look for yourself.

We will not follow Red Hat's lead with screen.

BTW, Red Hat has deprecated screen and has removed it from RHEL 8. They have replaced it with tmux (which is also in the FreeBSD ports collection). Another reason not to follow Red Hat's lead. screen.xterm-256color will not be added to termcap.
Comment 6 Victor Sudakov 2020-04-29 14:57:52 UTC
OK, forget about RedHat. I already wish I had not mentioned it.

> screen.xterm-256color will not be added to termcap.

Then screen itself should be patched to maintain compatibility with FreeBSD, because the recent version of screen forces "screen.xterm-256color" in $TERM on the user, and there is no way to redefine it from screenrc or elsewhere.

Don't hurry to close the bug please. I'd like to hear another opinion on the matter.
Comment 7 Cy Schubert freebsd_committer 2020-04-29 14:59:13 UTC
Testing this on a RHEL 7 box at $JOB, Red Hat has renamed many of the entries in terminfo, replacing the first dash (-) with a dot (.). FreeBSD is not Linux. FreeBSD is not Red Hat. We will not follow Red Hat's lead.

BTW, screen is not in the yum repo for RHEL 8. I discovered that at $JOB last week. You must use tmux and install the screen compatibility on RHEL 8. Again, FreeBSD will not follow Red Hat's lead in this regard.
Comment 8 Victor Sudakov 2020-04-29 15:03:00 UTC
(In reply to Cy Schubert from comment #7)

Please forget already about RedHat. The problem addressed in this PR is that screen port does not work properly with FreeBSD. Period.
Comment 9 Cy Schubert freebsd_committer 2020-04-29 15:48:41 UTC
It works as documented and as delivered from the GNU screen upline.

If you feel strongly enough, patches please.
Comment 10 Cy Schubert freebsd_committer 2020-04-29 15:57:55 UTC
You can start by applying a patch to ${WRKSRC}/terminfo/screencap. Then verify it updates ${LOCALBASE}/share/misc/terminfo. However this will only work if the dependency is ncurses from ports (ncurses:port). Therefore you will need to make this an option when you submit your patch.

I will review your patch and commit it after review.
Comment 11 Cy Schubert freebsd_committer 2020-04-29 16:14:59 UTC
This is not a sysutils/screen issue. The bug submitter is asked submit a patch to add the Red Hat only terminfo definitions into the FreeBSD base termcap and and a patch to the devel/ncurses port to add the same definitions to its terminfo database.

Alternatively, as previously suggested, though less optimal, a patch to sysutils/screen to optionally add definitions to the terminfo database installed by devel/ncurses. One must remember that ports cannot update the base FreeBSD O/S therefore the port cannot update /etc/termcap.

It is suggested a new PR or a phabricator review (preferred) be used to submit the patches. The phabricator review will allow a greater number of FreeBSD developers to review the patches for inclusion into FreeBSD and devel/ncurses, or sysutils/screen.
Comment 12 Yuri Pankov 2020-04-29 16:40:27 UTC
Cy, the problem submitter describes is not about adding entries to termcap.  The problem seems to be that recent screen versions set the TERM to that weird "screen.xterm-256color" for them -- I was NOT able to reproduce it (i.e. for me it's "screen" by default and "xterm-256color" with "term xterm-256color" in ~/.screenrc) and just trying to clarify.
Comment 13 Cy Schubert freebsd_committer 2020-04-29 16:55:27 UTC
I'm not able to reproduce it here either. However I can reproduce it at $JOB on RHEL 7 and Fedora.

There may have been some Red Hatism that may have crept into screen.
Comment 14 Cy Schubert freebsd_committer 2020-04-29 17:09:38 UTC
I think I have a solution.

Can you provide the output of pkg info ncurses, please?
Comment 15 Cy Schubert freebsd_committer 2020-04-29 17:10:02 UTC
From both of you, please.
Comment 16 Yuri Pankov 2020-04-29 17:11:16 UTC
(In reply to Cy Schubert from comment #15)
No ncurses port installed, running pretty recent -CURRENT.
Comment 17 Cy Schubert freebsd_committer 2020-04-29 17:46:05 UTC
To get screen.anything you need devel/ncurses installed. devel/ncurses includes all the screen.* definitions in its terminfo.db file.

Without it, as the base FreeBSD doesn't have screen.*, you will not see it defined.

This suggests that the screen.xterm-256color that is supplied by devel/ncurses might be incorrect.
Comment 18 Cy Schubert freebsd_committer 2020-04-29 18:01:49 UTC
I have been able to reproduce this with devel/ncurses installed on -CURRENT. It works properly when devel/ncures is installed.

The question is, do we add this to the base termcap(5) or not? Or do we tell  users to install devel/ncurses for full functional behavior?
Comment 19 Yuri Pankov 2020-04-29 18:03:41 UTC
(In reply to Cy Schubert from comment #18)
We certainly could, but it would take everyone to run -CURRENT/-STABLE for this to work at the moment, so it's likely a better idea to just depend on devel/ncurses if that helps.
Comment 20 Cy Schubert freebsd_committer 2020-04-29 18:14:29 UTC
I'll consider adding (full function) to the options.

diff --git a/sysutils/screen/Makefile b/sysutils/screen/Makefile
index 15eaee1cf913..8b4c128d9686 100644
--- a/sysutils/screen/Makefile
+++ b/sysutils/screen/Makefile
@@ -31,8 +31,8 @@ SOCKETS_DESC=         Use new (4.2.1+) sockets for IPC (default)
 NAMED_PIPES_DESC=      Use legacy (4.0.3) named pipes for IPC (override)
 SYSTEM_SCREENRC_DESC=  Install system screenrc with helpful status line
 NCURSES_DEFAULT_DESC=  Depend on ncurses (ports if installed, otherwise base)
-NCURSES_BASE_DESC= Depend on ncurses in base
-NCURSES_PORT_DESC= Depend on devel/ncurses in ports
+NCURSES_BASE_DESC=   Depend on ncurses in base (limited termcap support)
+NCURSES_PORT_DESC=   Depend on devel/ncurses in ports (full terminfo support)
 
 NCURSES_DEFAULT_USES=  ncurses
 NCURSES_BASE_USES=     ncurses:base

For people who use pkg we can create a metaport which depends on devel/ncurses by default.

Adding support to base is probably out of the question unless we can convince a good number of the community that this is a good thing.
Comment 21 Cy Schubert freebsd_committer 2020-04-29 23:26:01 UTC
There doesn't appear to be much interest on freebsd-current@ or freebsd-ports@ regarding a metaport, The updated options descriptions should be enough. Making a dependency on devel/ncurses default will increase the port's footprint which will anger some who want minimal install.
Comment 22 Victor Sudakov 2020-04-30 02:11:28 UTC
(In reply to Yuri Pankov from comment #12)

> The problem seems to be that recent screen versions set the TERM to that weird "screen.xterm-256color" for them 

Yes, thanks for understanding.

> I was NOT able to reproduce it 

I think you should start screen from xterm, "Xfce terminal" or similar, that is from an X session. That is what I do.

When you run screen from the console, or a tty (ssh) session, it sets $TERM to just plain "screen".
Comment 23 Cy Schubert freebsd_committer 2020-04-30 02:13:45 UTC
(In reply to Victor Sudakov from comment #22)
Do you have devel/ncurses installed?
Comment 24 Victor Sudakov 2020-04-30 02:15:32 UTC
(In reply to Cy Schubert from comment #23)
> Do you have devel/ncurses installed?

No, I don't:

$ pkg info ncurses
pkg: No package(s) matching ncurses

And to reproduce the problem, start screen from within an xterm.
Comment 25 Victor Sudakov 2020-04-30 02:21:10 UTC
Why is the status of the PR FIXED?
Comment 26 Cy Schubert freebsd_committer 2020-04-30 02:25:46 UTC
There is nothing to do. I doubt anyone will welcome adding the screen.* entries into termcap.

To fix your problem, uninstall screen and reinstall it after installing devel/ncurses. Or you can select the NCURSES_PORT option using "make config install clean" to install devel/ncurses for you. That will add the screen.* definitions. Either way you need devel/ncurses installed.
Comment 27 Victor Sudakov 2020-04-30 02:38:49 UTC
(In reply to Cy Schubert from comment #26)

> There is nothing to do.
> I doubt anyone will welcome adding the screen.* entries into termcap.

I don't like this idea either, I just suggested it as a workaround.

> To fix your problem, uninstall screen and reinstall it after installing devel/ncurses.

Nope, just tried and it did not work. $TERM within screen is still screen.xterm-256color

Time to switch to tmux probably.
Comment 28 Victor Sudakov 2020-04-30 02:44:58 UTC
I found more details.

If your current $TERM is "xterm-256color" and you start screen, $TERM within screen will be "screen.xterm-256color".

If your current $TERM is just "xterm" and you start screen, $TERM within screen will be "screen" as usual.

I think this PR is far from CLOSED or FIXED.
Comment 29 Cy Schubert freebsd_committer 2020-04-30 03:15:27 UTC
with devel/ncurses installed TERM=screen.xterm-256color, which is correct. No curses apps break. If you think otherwise, submit a patch, please.
Comment 30 Victor Sudakov 2020-04-30 04:21:35 UTC
(In reply to Cy Schubert from comment #29)
> with devel/ncurses installed TERM=screen.xterm-256color, which is correct.

No, it is not correct. Please ssh from inside screen to another FreeBSD host with this value of $TERM, and observe errors:

tcsh: The terminal database could not be opened.
tcsh: using dumb terminal settings.
# echo $TERM
screen.xterm-256color
#
# vi /etc/motd 
vi: No terminal database found
# 

etc etc.
Comment 31 Cy Schubert freebsd_committer 2020-04-30 04:36:55 UTC
slippy$ TERM=xterm-256color
slippy$ 

slippy<0>$ echo $TERM
screen.xterm-256color
slippy<0>$ vi /etc/motd


FreeBSD 13.0-CURRENT (BREAK) #175 r350162M: Fri Jul 19 20:54:56 PDT 2019


Access to, or unauthorized use of data on this computer by any person other
than authorized person(s) or owner(s) of an account is strictly prohibited
and may result in legal action against such person.

~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
/etc/motd: unmodified, readonly: line 1
[slippy][                     (0*$ slippy)                     ][04/29/20 21:36]
Comment 32 Cy Schubert freebsd_committer 2020-04-30 04:45:07 UTC
slippy$ TERM=xterm-256color
slippy$ ssh cwsys
Last login: Wed Apr 29 21:43:14 2020 from slippy
FreeBSD 13.0-CURRENT (BREAK) #1078 r360479M: Wed Apr 29 15:45:48 PDT 2020

Access to, or unauthorized use of data on this computer by any person other
than authorized person(s) or owner(s) of an account is strictly prohibited
and may result in legal action against such person.

cwsys$ echo $TERM
xterm-256color
cwsys$ 

cwsys<0>$ echo $TERM
screen.xterm-256color
cwsys<0>$ 

FreeBSD 13.0-CURRENT (BREAK) #183 r350162M: Fri Jul 19 20:58:22 PDT 2019

Access to, or unauthorized use of data on this computer by any person other
than authorized person(s) or owner(s) of an account is strictly prohibited
and may result in legal action against such person.

~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
/etc/motd: unmodified, readonly: line 1
[cwsys][                      (0*$ cwsys)                      ][04/29/20 21:44]
Comment 33 Cy Schubert freebsd_committer 2020-04-30 05:02:13 UTC
Put this into your .cshrc on the target system.

eval `tset -Q -s -m screen.xterm-256color:xterm-256color`
Comment 34 Victor Sudakov 2020-04-30 05:10:12 UTC
(In reply to Cy Schubert from comment #32)
I don't have 13-CURRENT, but a simple test for me on 12.1:

$ freebsd-version 
12.1-RELEASE-p4
$ setenv TERM screen.xterm-256color
csh: The terminal database could not be opened.
csh: using dumb terminal settings.
$
Comment 35 Victor Sudakov 2020-04-30 05:18:09 UTC
(In reply to Cy Schubert from comment #33)
> tset

Could be used as a workaround!

But if I have to modify .cshrc on all target systems (!) I'd probably just conditionally redefine the value of $TERM

However it's not the right path IMHO. It would be more correct to teach screen to always obey the term setting in screenrc and never add anything from itself.
Comment 36 Cy Schubert freebsd_committer 2020-04-30 13:34:54 UTC
I'll have a short-term solution later this week. Someone else will work on a long term strategy. It will require buildworld/installworld.
Comment 37 Thomas E. Dickey 2020-05-02 13:00:17 UTC
It's not a new issue.  ncurses has had screen.whatever entries since 2001:

https://invisible-island.net/ncurses/terminfo.src.html#t20010331

It's a documented feature in screen, which is useful for working around limitations of screen handling diverse terminals.  It would be more of an issue with tmux if tmux had more than a few users where the underlying terminal isn't xterm or something trying to imitate it.  (A large fraction of tmux's bug reports deal with that "trying").

On the other hand, FreeBSD's termcap entries generally don't agree with ncurses (we could go on at length about that).  It works well enough for the most common types, of course, but the limitation to termcap means that a large chunk of functionality isn't available in the first place, and then there are quirks like vt100-color which don't appear to have a quick solution.

The recommended solution for FreeBSD would be to install the ncurses port, and rely on that.  On a vanilla install, I'd just override TERM=screen (which works well for casual use).
Comment 38 Thomas E. Dickey 2020-05-02 13:18:07 UTC
By the way, the comments about "Redhat-only" should be ignored.
No Redhat developer contributed to this issue in any way.
Comment 39 Victor Sudakov 2020-05-03 17:24:27 UTC
(In reply to Thomas E. Dickey from comment #37)
> The recommended solution for FreeBSD would be to install the ncurses port,
> and rely on that. 

Install where? I do not see how the presence of the ncurses port is any of help when there is no termcap entry for "screen.xterm-256color":

vas@test4:~ % pkg info -o ncurses
ncurses-6.1.20190525           devel/ncurses
vas@test4:~ % setenv TERM screen.xterm-256color
tcsh: The terminal database could not be opened.
tcsh: using dumb terminal settings.
vas@test4:~ % 


You probably meant something else.
Comment 40 Thomas E. Dickey 2020-05-03 19:37:26 UTC
Perhaps.  I generally use what I build for development.

Checking... the base libncursesw isn't configured to allow it to read terminfo.
The source is capable of doing this.  If the base were configured that way,
it would add 10Kb to the library size (524Kb vs 514Kb).  To put that into
perspective, this webpage when saved to an html-file takes up 70Kb.
Comment 41 Thomas E. Dickey 2020-05-03 19:42:43 UTC
Then again, I configure ncurses with more features, so FreeBSD's library is
a little smaller than 500Kb.  The 10Kb difference is what you'd see if you
built ncurses with the different configurations.  Making that change to the
base would result in better integration with the ncurses port...
Comment 42 Cy Schubert freebsd_committer 2020-05-03 20:04:33 UTC
There is another problem, not related to sysutils/screen and not addressed by devel/ncurses. Open a screen session. Then ssh/rsh/telnet to a target server. Any base (/bin, /usr/bin, /sbin, /usr/sbin) utility or any app linked against the base ncurses will not understand termcap entries used by screen linked against devel/ncurses.

To do:

1. Test using screen linked against base only. I suspect the problem is worse.
2. Add the additional screen.* entries to base termcap. This will resolve the issue for now.
3. Develop a utility to convert terminfo entries in devel/ncurses to termcap in base. (bapt@ has agreed put this on his todo, while I work on the first two.)

#3 could be resolved by using infotocap(1) from devel/ncurses.

I'll try to tackle this sometime this week.
Comment 43 Victor Sudakov 2020-05-04 02:04:47 UTC
(In reply to Cy Schubert from comment #42)
> 1. Test using screen linked against base only. I suspect the problem is worse.

Does not seem to help, please see below. I just started screen from an XFCE Terminal: 

$ ldd /usr/local/bin/screen 
/usr/local/bin/screen:
        libncursesw.so.8 => /lib/libncursesw.so.8 (0x8002b1000)
        libutil.so.9 => /lib/libutil.so.9 (0x800312000)
        libulog.so.0 => /lib/libulog.so.0 (0x800329000)
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x80032d000)
        libc.so.7 => /lib/libc.so.7 (0x80034e000)
        libmd.so.6 => /lib/libmd.so.6 (0x800747000)
$ echo $TERM
screen.xterm-256color
$
$ pkg which  /usr/local/bin/screen
/usr/local/bin/screen was installed by package screen-4.8.0
$
Comment 44 Victor Sudakov 2020-05-04 02:08:36 UTC
Excuse me, I'm afraid you words do not make sense to me any more. Probably it's my poor English. The problem is very simple: screen when started from an xterm sets the $TERM variable in its windows to "screen.xterm-256color" and there is no way to change that even with the "term" screenrc command.

The only workaround known to me is to start screen like that:

env TERM=screen screen -S foo

Then it will have $TERM=screen inside its windows.
Comment 45 Victor Sudakov 2020-05-04 02:09:30 UTC
The last message was a reply to Comment 41 from Thomas.
Comment 46 Cy Schubert freebsd_committer 2020-05-04 04:25:23 UTC
Created attachment 214083 [details]
New termcap entries

Apply this patch. Buildworld/installworld on the target system (the system you will ssh to - not ssh from).
Comment 47 Victor Sudakov 2020-05-04 13:16:01 UTC
(In reply to Cy Schubert from comment #46)
What FreeBSD version is the patch for? Does not apply to /usr/src of 12.1-RELEASE: 

# patch -p1 -C < /root/attachment.diff
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- b/share/termcap/termcap
|+++ b/share/termcap/termcap
--------------------------
Patching file share/termcap/termcap using Plan A...
Hunk #1 failed at 2788.
1 out of 1 hunks failed while patching share/termcap/termcap
done
Comment 48 Cy Schubert freebsd_committer 2020-05-04 13:37:02 UTC
It appears termcap has diverged in -CURRENT. I'll try to simplify the patch to just one entry, the the entry in this PR.
Comment 49 Cy Schubert freebsd_committer 2020-05-04 17:18:46 UTC
Created attachment 214116 [details]
Add to end of termcap

Add this to the end of share/termcap/termcap. Buildworld and installworld.
Comment 50 Victor Sudakov 2020-07-12 12:51:31 UTC
Why are buildworld and installworld necessary? Would it not be sufficient to run "cap_mkdb /usr/share/misc/termcap" ?
Comment 51 Cy Schubert freebsd_committer 2020-07-12 13:11:29 UTC
Correct but that is simpler for some people.
Comment 52 Cy Schubert freebsd_committer 2020-07-12 13:11:45 UTC
Adding this to termcap has been rejected by the person who maintains termcap. He will work on a more comprehensive update to termcap.