Bug 30778

Summary: libncurses/termcap.c does not handle escaped colons
Product: Base System Reporter: Bernd Luevelsmeyer <bdluevel>
Component: binAssignee: ru <ru>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 4.4-STABLE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
file.diff none

Description Bernd Luevelsmeyer 2001-09-24 07:00:00 UTC
Using a terminal 'wy60', the initialization with 'tset' results in the messages
"TERMCAP", line 0, terminal 'wy60-43': enter_alt_charset_mode but no exit_alt_charset_mode
"TERMCAP", line 0, terminal 'wy60-43': enter_alt_charset_mode but no acs_chars

This can be helped by omitting the 'as=' entry from the terminal description.

How-To-Repeat:   %tset wyse60
Comment 1 ru freebsd_committer freebsd_triage 2001-09-24 13:05:43 UTC
I once had the Esprit 400 terminal that worked in wy60k emulation mode.
On it, the sequence to turn off an alternate character set was \EH^C.
Can you please test this patch:

1.  Apply the patch
2.  Reinstall termcap
3.  Set TERM=wy60k
4.  tput as; echo ABC; tput ae; echo ABC

You should see pseudographic or alternate-charset characters on the
first line, and ABC on the second.  Let me know if it works for you.

Index: termcap.src
===================================================================
RCS file: /home/ncvs/src/share/termcap/termcap.src,v
retrieving revision 1.109
diff -u -r1.109 termcap.src
--- termcap.src	2001/08/15 03:25:57	1.109
+++ termcap.src	2001/09/24 12:03:08
@@ -3489,7 +3489,7 @@
 # (Wyse Tech, svd@bigboy.wyse.com) and Douglas Good (doug@cli.com).
 # Merged by Paul Vojta (vojta@math.berkeley.edu).
 wy60|wyse60|Wyse 60 in native mode:\
-	:ae=\EH^C:al=4\EE:am:as=\EH^B:bl=^G:bs:bt=\EI:bw:cd=100\Ey:ce=\Et:\
+	:ae=\EH^C:al=4\EE:am:as=\EH^B:ae=\EH^C:bl=^G:bs:bt=\EI:bw:cd=100\Ey:ce=\Et:\
 	:cl=100\E*:cm=\E=%+ %+ :co#80:ct=\E0:da:db:dc=11\EW:dl=5\ER:do=^J:\
 	:ds=\EF\r:ei=\Er:fs=\r:ho=^^:hs:if=/usr/share/tabset/std:im=\Eq:ip=3:\
 	:is=\Ed$\EcD\E'\Er\EH^C\Ed/\EO\Ee1\Ed*\E`@\E`9\E`1^N^T\EwG\Ew0^^^K:\

On Mon, Sep 24, 2001 at 07:55:28AM +0200, Bernd Luevelsmeyer wrote:
> 
> Using a terminal 'wy60', the initialization with 'tset' results in the messages
> "TERMCAP", line 0, terminal 'wy60-43': enter_alt_charset_mode but no exit_alt_charset_mode
> "TERMCAP", line 0, terminal 'wy60-43': enter_alt_charset_mode but no acs_chars
> 
> This can be helped by omitting the 'as=' entry from the terminal description.
> 
> >How-To-Repeat:
>   %tset wyse60
> 
> >Fix:
> --- /usr/share/misc/.vimbk/termcap.vimbk	Mon Sep 10 04:43:40 2001
> +++ /usr/share/misc/termcap	Mon Sep 24 07:30:31 2001
> @@ -3519,7 +3519,7 @@
>  # (Wyse Tech, svd@bigboy.wyse.com) and Douglas Good (doug@cli.com).
>  # Merged by Paul Vojta (vojta@math.berkeley.edu).
>  wy60|wyse60|Wyse 60 in native mode:\
> -	:ae=\EH^C:al=4\EE:am:as=\EH^B:bl=^G:bs:bt=\EI:bw:cd=100\Ey:ce=\Et:\
> +	:ae=\EH^C:al=4\EE:am:bl=^G:bs:bt=\EI:bw:cd=100\Ey:ce=\Et:\
>  	:cl=100\E*:cm=\E=%+ %+ :co#80:ct=\E0:da:db:dc=11\EW:dl=5\ER:do=^J:\
>  	:ds=\EF\r:ei=\Er:fs=\r:ho=^^:hs:if=/usr/share/tabset/std:im=\Eq:ip=3:\
>  	:is=\Ed$\EcD\E'\Er\EH^C\Ed/\EO\Ee1\Ed*\E`@\E`9\E`1^N^T\EwG\Ew0^^^K:\

-- 
Ruslan Ermilov		Oracle Developer/DBA,
ru@sunbay.com		Sunbay Software AG,
ru@FreeBSD.org		FreeBSD committer,
+380.652.512.251	Simferopol, Ukraine

http://www.FreeBSD.org	The Power To Serve
http://www.oracle.com	Enabling The Information Age
Comment 2 Bernd Luevelsmeyer 2001-09-24 15:09:52 UTC
Ruslan Ermilov wrote:
> 
> I once had the Esprit 400 terminal that worked in wy60k emulation mode.

I've got an original Wyse thingy, at least I think I have. There's
"datamax 2000" written on it, but the picture in the manual matches
the thing.


> On it, the sequence to turn off an alternate character set was \EH^C.
> Can you please test this patch:
> 
> 1.  Apply the patch
> 2.  Reinstall termcap

Patch was rejected by "patch" utility because of line offset; I did
it manually.


> 3.  Set TERM=wy60k

I understand you mean wy60? Because your patch changes wy60. And
there's no wy60k in the termcap file.


> 4.  tput as; echo ABC; tput ae; echo ABC
> 
> You should see pseudographic or alternate-charset characters on the
> first line, and ABC on the second.  Let me know if it works for you.

It works. The first line is like
  S S E
   H X X
(one line, characters consisting of two letters)

A mystery to me is, however, why the terminal type "wy60-43" still
outputs the two error lines. "wy60" doesn't!

I log in, and get the terminal type wy60-43. I get the two error
lines. I execute your tput command line. Characters are mangled
afterwards. Executing "reset" (typed blind) gives me letters again.
I then execute "export TERM=wy60" and the tput-line again, and it
works. "export TERM=wy60-43" gives me the errors, and the tput-line
again gives me mangled stuff again, which is cleaned up by "reset",


Greetings,
	Bernd
Comment 3 ru freebsd_committer freebsd_triage 2001-09-24 17:07:11 UTC
On Mon, Sep 24, 2001 at 04:09:52PM +0200, Bernd Luevelsmeyer wrote:
> Ruslan Ermilov wrote:
> > 
> > I once had the Esprit 400 terminal that worked in wy60k emulation mode.
> 
> I've got an original Wyse thingy, at least I think I have. There's
> "datamax 2000" written on it, but the picture in the manual matches
> the thing.
> 
There seems to be a bug in libncurses.

For yet unknown reason, it doesn't (sometimes) understand the escaped
colon (``:'') character embedded into the value.  For "wy60-25", I've
underlined the offendind symbols:

wy60-25|wyse60-25|wyse 60 80-column 25-lines:\
        :li#25:\
        :rs=500\E~!\E~4\E`\:\EeG\Ee)\EwG:tc=wy60:
                          ^^^^

The bug is somewhere in contrib/ncurses/ncurses/tinfo.


Cheers,
-- 
Ruslan Ermilov		Oracle Developer/DBA,
ru@sunbay.com		Sunbay Software AG,
ru@FreeBSD.org		FreeBSD committer,
+380.652.512.251	Simferopol, Ukraine

http://www.FreeBSD.org	The Power To Serve
http://www.oracle.com	Enabling The Information Age
Comment 4 Bernd Luevelsmeyer 2001-09-25 00:22:43 UTC
Ruslan Ermilov wrote:
[...]
> For yet unknown reason, it doesn't (sometimes) understand the escaped
> colon (``:'') character embedded into the value.  For "wy60-25", I've
> underlined the offendind symbols:
[...]

Thanks a lot for your investigation!

So we've got two unrelated errors then, one was mismatching as/ae in
termcap (which has been taken care of for the wyse terminals), and
the other an ncurses hickup on escaped colons.


> The bug is somewhere in contrib/ncurses/ncurses/tinfo.

I'll see if I can spot it. Don't expect too much, though; the topic
is new to me.

Could the same error happen for the reading of other, similar
databases, such as login.conf or printcap? They are built similarly,
so I assume code sharing has happened.


Greetings,
	Bernd
Comment 5 ru freebsd_committer freebsd_triage 2001-09-25 07:41:00 UTC
On Tue, Sep 25, 2001 at 01:22:43AM +0200, Bernd Luevelsmeyer wrote:
> Ruslan Ermilov wrote:
> [...]
> > For yet unknown reason, it doesn't (sometimes) understand the escaped
> > colon (``:'') character embedded into the value.  For "wy60-25", I've
> > underlined the offendind symbols:
> [...]
> 
> Thanks a lot for your investigation!
> 
> So we've got two unrelated errors then, one was mismatching as/ae in
> termcap (which has been taken care of for the wyse terminals), and
> the other an ncurses hickup on escaped colons.
> 
Nope, only the second.  "ae" is already put as the first entry for
"wy60", I just did not notice it.  :-)

> > The bug is somewhere in contrib/ncurses/ncurses/tinfo.
> 
> I'll see if I can spot it. Don't expect too much, though; the topic
> is new to me.
> 
> Could the same error happen for the reading of other, similar
> databases, such as login.conf or printcap? They are built similarly,
> so I assume code sharing has happened.
> 
No, but there are other terminals that are vulnerable to this.


Cheers,
-- 
Ruslan Ermilov		Oracle Developer/DBA,
ru@sunbay.com		Sunbay Software AG,
ru@FreeBSD.org		FreeBSD committer,
+380.652.512.251	Simferopol, Ukraine

http://www.FreeBSD.org	The Power To Serve
http://www.oracle.com	Enabling The Information Age
Comment 6 Jens Schweikhardt freebsd_committer freebsd_triage 2002-08-11 18:24:37 UTC
Responsible Changed
From-To: freebsd-bugs->ru

Ruslan, you've had quite some dialog with the originator. 
Would you mind taking care of a final resolution?
Comment 7 ru freebsd_committer freebsd_triage 2002-08-12 18:08:19 UTC
Responsible Changed
From-To: ru->freebsd-bugs

I know of a better person.
Comment 8 ru freebsd_committer freebsd_triage 2002-08-12 18:28:14 UTC
On Mon, Aug 12, 2002 at 10:13:36AM -0700, Ruslan Ermilov wrote:
> Old Synopsis: termcap problem with wyse-60 terminal
> New Synopsis: libncurses/termcap.c does not handle escaped colons
> 
> Responsible-Changed-From-To: ru->freebsd-bugs
> Responsible-Changed-By: ru
> Responsible-Changed-When: Mon Aug 12 10:08:19 PDT 2002
> Responsible-Changed-Why: 
> I know of a better person.
> 
Peter,

The FreeBSD replacement for contrib/ncurses/tinfo/read_termcap.c,
src/lib/libncurses/termcap.c, is subject to the following bug.

The colon (`:') is used to delimit capabilities.  To include the
colon as part of the value, it must be escaped.  (The wy60 entry
has this in its "rs" capability, for example.)

termcap.c capabilities tokenization code is not aware of escaped
colons, causing _nc_termcap to be initialized to a wrong value.

The bug is easily demonstrated by this command:

	env TERMCAP='term:c1=\:\Ev1:c2=v2:' tput -Tterm c1 | hd

This appears to fix the bug:

%%%
Index: termcap.c
===================================================================
RCS file: /home/ncvs/src/lib/libncurses/termcap.c,v
retrieving revision 1.5
diff -u -p -r1.5 termcap.c
--- termcap.c	21 Mar 2002 23:38:38 -0000	1.5
+++ termcap.c	12 Aug 2002 17:22:00 -0000
@@ -200,6 +200,7 @@ _nc_read_termcap_entry(const char *const
 		tcs = pd - 1;
 		for (;;) {
 			while ((tok = strsep(&ps, ":")) != NULL &&
+			       *(tok - 2) != '\\' &&
 			       (*tok == '\0' || *tok == '\\' || !isgraph(*tok)))
 				;
 			if (tok == NULL)
%%%


Cheers,
-- 
Ruslan Ermilov		Sysadmin and DBA,
ru@sunbay.com		Sunbay Software AG,
ru@FreeBSD.org		FreeBSD committer,
+380.652.512.251	Simferopol, Ukraine

http://www.FreeBSD.org	The Power To Serve
http://www.oracle.com	Enabling The Information Age
Comment 9 ru freebsd_committer freebsd_triage 2002-08-12 19:18:21 UTC
On Mon, Aug 12, 2002 at 09:37:59PM +0400, Andrey A. Chernov wrote:
> On Mon, Aug 12, 2002 at 10:30:03 -0700, Ruslan Ermilov wrote:
> >  
> >  The colon (`:') is used to delimit capabilities.  To include the
> >  colon as part of the value, it must be escaped.  (The wy60 entry
                                                       ^^^^^^^^^^^^^^
> >  has this in its "rs" capability, for example.)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> >From where you get assumption that \: is valid escape sequence? It should 
> be written as \072 instead.
> 
- by grepping over src/share/termcap/termcap.src (dg460-ansi, wy60,
  wy60-25, wy60-42, and wy60-43 have these),
- by looking over in the contrib/ncurses/ncurses/tinfo/comp_scan.c
  _nc_trans_string() code,
- by running the test with the ncurses' stock read_termcap.c,
- by seeing how tic(1) converts colons,
- by hearing the following from Thomas Dickey (ncurses maintainer):

:                                      ncurses handles the escaped colon
: fine, but how it's interpreted by FreeBSD's termcap library is another
: matter altogether.

- by POLA.

I know the termcap(5) mentions that `:' in the value should be written
as \072, I just think that not supporting `\:' is a bug, and the above
is a compiled list of why I think so.


Cheers,
-- 
Ruslan Ermilov		Sysadmin and DBA,
ru@sunbay.com		Sunbay Software AG,
ru@FreeBSD.org		FreeBSD committer,
+380.652.512.251	Simferopol, Ukraine

http://www.FreeBSD.org	The Power To Serve
http://www.oracle.com	Enabling The Information Age
Comment 10 Андрей Чернов 2002-08-12 19:22:57 UTC
On Mon, Aug 12, 2002 at 21:18:21 +0300, Ruslan Ermilov wrote:

> - by grepping over src/share/termcap/termcap.src (dg460-ansi, wy60,
>   wy60-25, wy60-42, and wy60-43 have these),
> - by looking over in the contrib/ncurses/ncurses/tinfo/comp_scan.c
>   _nc_trans_string() code,
> - by running the test with the ncurses' stock read_termcap.c,
> - by seeing how tic(1) converts colons,
> - by hearing the following from Thomas Dickey (ncurses maintainer):

I agree with you, please commit it.

-- 
Andrey A. Chernov
http://ache.pp.ru/
Comment 11 ru freebsd_committer freebsd_triage 2002-08-12 19:34:41 UTC
On Mon, Aug 12, 2002 at 10:22:57PM +0400, Andrey A. Chernov wrote:
> On Mon, Aug 12, 2002 at 21:18:21 +0300, Ruslan Ermilov wrote:
> 
> > - by grepping over src/share/termcap/termcap.src (dg460-ansi, wy60,
> >   wy60-25, wy60-42, and wy60-43 have these),
> > - by looking over in the contrib/ncurses/ncurses/tinfo/comp_scan.c
> >   _nc_trans_string() code,
> > - by running the test with the ncurses' stock read_termcap.c,
> > - by seeing how tic(1) converts colons,
> > - by hearing the following from Thomas Dickey (ncurses maintainer):
> 
> I agree with you, please commit it.
> 

I will wait for Peter's reply for some time.  If we decide to
commit this, the termcap(5) manpage needs to be fixed as well:

%%%
Index: termcap.5
===================================================================
RCS file: /home/ncvs/src/share/termcap/termcap.5,v
retrieving revision 1.21
diff -u -r1.21 termcap.5
--- termcap.5	8 Aug 2001 11:54:35 -0000	1.21
+++ termcap.5	12 Aug 2002 18:32:57 -0000
@@ -817,8 +817,9 @@
 .Sy \&\e\e .
 If it is necessary to place a
 .Sy \&:
-in a capability it must be escaped in
-octal as
+in a capability it must be escaped as
+.Sy \e:
+or be encoded as
 .Sy \&\e072 .
 If it is necessary to place a
 .Dv NUL
%%%


Cheers,
-- 
Ruslan Ermilov		Sysadmin and DBA,
ru@sunbay.com		Sunbay Software AG,
ru@FreeBSD.org		FreeBSD committer,
+380.652.512.251	Simferopol, Ukraine

http://www.FreeBSD.org	The Power To Serve
http://www.oracle.com	Enabling The Information Age
Comment 12 peter 2002-08-12 19:41:13 UTC
Ruslan Ermilov wrote:
> 
> --fdj2RfSjLxBAspz7
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: inline
> Content-Transfer-Encoding: quoted-printable
> 
> On Mon, Aug 12, 2002 at 10:22:57PM +0400, Andrey A. Chernov wrote:
> > On Mon, Aug 12, 2002 at 21:18:21 +0300, Ruslan Ermilov wrote:
> >=20
> > > - by grepping over src/share/termcap/termcap.src (dg460-ansi, wy60,
> > >   wy60-25, wy60-42, and wy60-43 have these),
> > > - by looking over in the contrib/ncurses/ncurses/tinfo/comp_scan.c
> > >   _nc_trans_string() code,
> > > - by running the test with the ncurses' stock read_termcap.c,
> > > - by seeing how tic(1) converts colons,
> > > - by hearing the following from Thomas Dickey (ncurses maintainer):
> >=20
> > I agree with you, please commit it.
> >=20
> I will wait for Peter's reply for some time.  If we decide to
> commit this, the termcap(5) manpage needs to be fixed as well:

Go ahead. This termcap.c comes from the freebsd version of libtermcap,
so the bug is also present in RELENG_3 and earlier as well.

Cheers,
-Peter
--
Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com
"All of this is for nothing if we don't go to the stars" - JMS/B5
Comment 13 ru freebsd_committer freebsd_triage 2002-08-12 20:13:43 UTC
State Changed
From-To: open->patched

Fixed in 5.0-CURRENT, lib/libncurses/termcap.c,v 1.6 and 
share/termcap/termcap.5,v 1.22. 


Comment 14 ru freebsd_committer freebsd_triage 2002-08-12 20:13:43 UTC
Responsible Changed
From-To: freebsd-bugs->ru
Comment 15 ru freebsd_committer freebsd_triage 2002-08-15 09:31:14 UTC
State Changed
From-To: patched->closed

Fixed in 4.6-STABLE, lib/libncurses/termcap.c,v 1.2.2.2 and 
share/termcap/termcap.5,v 1.12.2.6.