Apparently, /bin/csh (aka /bin/tcsh) is diddling termios flags, in particular the ECHO flag, for no apparently good reason and without ever even having been asked to do so. The result is that the /usr/bin/script program, when invoked with the -k option, will log every input character the user types in TWICE, but only if the user has SHELL set in the environment to either /bin/csh or to /bin/tcsh. This behavior does not seem to occur when and if SHELL is set to /bin/sh. Fix: I have no idea how to fix this. I tried to dredge into the tcsh source code, but it is just too imposing for me right now. So I'm sort-of hoping that somebody more familiarity with tcsh than me will dredge into it and find the problem. P.S. Strangely, when you run /usr/bin/script with SHELL set to /bin/tcsh and when you then run a program from the tcsh command prompt, that program will see a properly set group of termios bitflags, e.g. when and if it calls tcgetattr(). How-To-Repeat: setenv SHELL /bin/tcsh /usr/bin/script -k Then just type something... anything... and then exit the shell and then look at the typescript file to see the doubled characters. I further traced this problem down by inserting the following changes into /usr/bin/script. These show what the termios flags look like before and after the first call to select() within script.c, and thus they will show how once tcsh is fired up (as a child process) by script, it changes the termios flags on its pty, unsetting MANY of the bitflags, including ECHO. Note that if you run /usr/bin/script with these changes installed, and do so with SHELL set to /bin/sh then you will see that in that case, the termios bit flags DO NOT get diddled by the child shell process, and that the ECHO bitflag, in particular, does not get un-set, and thus, "script -k" works properly in that case. *** script.c.orig 2004-02-15 09:30:13.000000000 -0800 --- script.c 2010-11-11 16:41:15.000000000 -0800 *************** *** 92,95 **** --- 92,96 ---- fd_set rfd; int flushtime = 30; + int first_time = 1; aflg = kflg = 0; *************** *** 174,178 **** --- 175,184 ---- tv.tv_usec = 0; } + if (first_time & tcgetattr(master, &stt) >= 0) + fprintf (stderr, "Pre-select: 0x%08x\r\n", stt.c_lflag); n = select(master + 1, &rfd, 0, 0, tvp); + if (first_time & tcgetattr(master, &stt) >= 0) + fprintf (stderr, "Post-select: 0x%08x\r\n", stt.c_lflag); + first_time = 0; if (n < 0 && errno != EINTR) break;
On Thu, 11 Nov 2010, Ronald F.Guilmette wrote: >> Description: > > Apparently, /bin/csh (aka /bin/tcsh) is diddling termios flags, in particular > the ECHO flag, for no apparently good reason and without ever even having been > asked to do so. The result is that the /usr/bin/script program, when invoked > with the -k option, will log every input character the user types in TWICE, > but only if the user has SHELL set in the environment to either /bin/csh or > to /bin/tcsh. This behavior does not seem to occur when and if SHELL is set > to /bin/sh. /bin/sh now does this too. bash does this too (in at least versions 1.14.7 and 4.0.35). I think the mode switch in the shell is an unavoidable consequence of line editing in the shell, and script is depending on the line editing being done in the kernel. The old versions of /bin/sh that don't have the problem differ in not starting up in line editing mode. > P.S. Strangely, when you run /usr/bin/script with SHELL set to /bin/tcsh > and when you then run a program from the tcsh command prompt, that program > will see a properly set group of termios bitflags, e.g. when and if it > calls tcgetattr(). This also happens with at least bash, since bash is careful to virtualize the terminal state by context switching it to a different value only while in its line editor. I don't understand why clearing ECHO in line editing mode doubles the script -k output instead of suppressing it. Bruce
On Fri, 12 Nov 2010, Bruce Evans wrote: > On Thu, 11 Nov 2010, Ronald F.Guilmette wrote: > >>> Description: >> >> Apparently, /bin/csh (aka /bin/tcsh) is diddling termios flags, in >> particular >> the ECHO flag, for no apparently good reason and without ever even having Perhaps I shouldn't have replied to this, since my ISP was blacklisted. >> ... >> P.S. Strangely, when you run /usr/bin/script with SHELL set to /bin/tcsh >> and when you then run a program from the tcsh command prompt, that program >> will see a properly set group of termios bitflags, e.g. when and if it >> calls tcgetattr(). > > This also happens with at least bash, since bash is careful to virtualize > the terminal state by context switching it to a different value only > while in its line editor. > ... PS: probably starting the subshell in non-line-editing-mode would work, but this is no good in general since you want the subshell to run in normal mode. I don't know how to force non-line-editing-mode, especially using $SHELL. script even starts the shell with -i, which tends to imply line editing mode. I just remembered a related problem. I often mistype command lines, or do complicated editing of them involving many backspaces and other control characters to move around or search history. script makes a mess of this by showing all the control characters, even without -k. The only way to handle this right might be to have the line editor in script itself, or perhaps just between the kernel and script (the ile utility might work for this). Bruce
> From owner-freebsd-bugs@freebsd.org Fri Nov 12 03:46:22 2010 > Date: Fri, 12 Nov 2010 20:48:08 +1100 (EST) > From: Bruce Evans <brde@optusnet.com.au> > Subject: Re: bin/152154: /bin/csh & /bin/tcsh improperly diddle termios flags > > On Fri, 12 Nov 2010, Bruce Evans wrote: > > > I just remembered a related problem. I often mistype command lines, > or do complicated editing of them involving many backspaces and other > control characters to move around or search history. script makes a > mess of this by showing all the control characters, even without -k. This is _documented behavior_. It is, also, essentially inescapable. " BUGS Script places everything in the log file, including linefeeds and backspaces. This is not what the naive user expects. " It is also documented in the body of the description: " The script utility works best with commands that do not manipulate the screen. The results are meant to emulate a hardcopy terminal, not an addressable one. " Aside: some of us _count_ on script behaving that way -- capturing _every_ character sent out. Script has -no- way of knowing what is echoed 'editing' characters, and what is 'significant' output. *ALL* script 'guarantees' -- in the situation involving an 'addressable' terminal -- is that if you play back the logfile to the _same_ type of terminal is that 'what you see' on the screen on replay is "what you got' originally. True "WYSIWYG". <grin> > The only way to handle this right might be to have the line editor in > script itself ... That works _UNTIL_ you need to pass a 'line edit' character to an application. Heaven forbid you forget you have script running, and invoke a full-screen editor (or any-other full-scrren utility for that matter) that uses one of the line-edit characters as it's "exit" sequence. And, even doing _that_ does not help with the 'mess' created by rummaging through, say, the csh cmommand history. A script functionality integrated into whichever CLI (shell) one uses could eliminate the line-edit / history, etc. "mess" from the logfile, but that is 'no help' in dealing with _all_ the other programs that produce output "assuming" an 'addressable' display. How do you log a 'vi' session, for example? <evil grin> For a 'cleaner' typescript, set the TERM environment variable to 'dumb'.
Gentlemen, Thanks for the discussion of this problem. As someone else just pointed out to me (via the tcsh discussion list) _any_ shell that allows command line editing (not just tcsh but also bash and zsh) are going to put the pseudo terminal into character-at-a-time non-echo mode, and thus, one will get doubled characters in the typescript file when using "script -k" in conjunction with any such shell. So I guess we should say that the REAL non-feature (aka "bug") here is not really in tcsh but rather in the way that script(1) decides whether or not to write input characters to the typescript file when operating in the -k mode. Here is the code which involves that decision: if (cc > 0) { (void)write(master, ibuf, cc); if (kflg && (tcgetattr(master, &stt) >= 0) && ((stt.c_lflag & ECHO) == 0)) { (void)fwrite(ibuf, 1, cc, fscript); } } My feeling now is that I should withdraw/cancel this PR (bin/152154) which is on tcsh and instead file a new & different one on script(1), because it is clear now that -k, as implemented, just won't work in conjunction with shells that allow command line editing. (Can someone plesse instruct me how to cancel a PR? I've never done it before, and I'm ignorant.) I want to say also that I do believe that there is a solution to this problem. I cannot be 100% sure yet that there will never be any circumstances under which it might perhaps malfunction slightly, but my early (and so far minimal) testing has not shown any signs of any malfunctions, so I am hopeful that this may be a solution that will work OK, even for the shells that allow command line editing. So anyway, here's my idea... Right now, script(1) is simply writing 100% of all characters which the child shell (or its children) write to stdout/stderr into the typescript file. In the -k case however, this is augmented by script also writing any characters that are INPUT (to the stdin of the child shell) to the typescript file also, but only (as shown by the code above) when the the child shell (or any of its children) has set its pseudo-tty to _not_ ECHO. Clearly, as we now know, this approach won't work consistantly across all shells. So instead of this approach, I propose that script(1) be modified so that when -k mode is in effect, it will write 100% of all INPUT characters to the typescript file, but then each time it receives a block of characters that are comming _out_ from the shell, instead of writing 100% of those OUTPUT characters to the typescript file, script(1) would instead compare the buffer full of output bytes it just received (frm the child shell and/or its children) to the last buffer full of input bytes that script(1) sent down to the child shell. If the contents of the two buffers are equal, then the current block of output characters that are coming up from the child shell would simply be suppressed and NOT written to the typescript file, thus, in efffect, nixing any echoed characters which simply duplicate characters that were already added to the typescript file. A patch implementing the change I have just described is included below. I would appreciate any and all comments on it. I've just tried it out a little and it does seem to work OK, at least with tcsh. Of course, the whole idea here is, admittedly, based upon a presumption which may or may not be true in all circumstances and/or in all contexts, i.e. that any input from the _actual_ controlling tty... which will usually be, you know, one character at a time, especially for a hunt-and-pecker like me... will be echoed back before more input comes in, i.e. if it is going to be echoed back at all. And I am allowing here for even the super duper input rate that may come from doing, e.g. a cut-and-paste, where many characters may be read from the real terminal device in one glup. As long as those all get echoed back, immediately, in one single glup too, then this approach to implementing -k still works. (I did try some cut-and-pasting and that also seemed to work OK with the patche below.) The only thing that isn't all that clear to me (and that frankly, I haven't tested yet) is what would happen if input is coming from something non- interactive, like, you know, a file. I actually don't know what would happen in that case, but I suspect that this patch should work OK even for that case. I guess that the possibility that something might cause some characters (e.g control characters and/or terminal control sequences) not to "echo" back verbatim, as themselves, might also be an issue, but in this case I rather doubt that the patch below make things any worse/stranger than they already are, when/if such oddities are being logged in the (current) script-k+/bin/sh case. Regards, rfg P.S. My apologies to Bruce Evans about the local blacklisting of your ISP. It was nothing personal, believe me. Just got too much spam from them, that's all. Anyway, I have removed that local blacklist entry now so for the time being at least, please do feel free to mail me direct, if you are so inclined. *** script.c.orig 2004-02-15 09:30:13.000000000 -0800 --- script.c 2010-11-14 16:22:15.000000000 -0800 *************** *** 82,86 **** main(int argc, char *argv[]) { ! int cc; struct termios rtt, stt; struct winsize win; --- 82,86 ---- main(int argc, char *argv[]) { ! int icc = 0, occ; struct termios rtt, stt; struct winsize win; *************** *** 178,200 **** break; if (n > 0 && FD_ISSET(STDIN_FILENO, &rfd)) { ! cc = read(STDIN_FILENO, ibuf, BUFSIZ); ! if (cc < 0) break; ! if (cc == 0) (void)write(master, ibuf, 0); ! if (cc > 0) { ! (void)write(master, ibuf, cc); ! if (kflg && tcgetattr(master, &stt) >= 0 && ! ((stt.c_lflag & ECHO) == 0)) { ! (void)fwrite(ibuf, 1, cc, fscript); ! } } } if (n > 0 && FD_ISSET(master, &rfd)) { ! cc = read(master, obuf, sizeof (obuf)); ! if (cc <= 0) break; ! (void)write(STDOUT_FILENO, obuf, cc); ! (void)fwrite(obuf, 1, cc, fscript); } tvec = time(0); --- 178,201 ---- break; if (n > 0 && FD_ISSET(STDIN_FILENO, &rfd)) { ! icc = read(STDIN_FILENO, ibuf, BUFSIZ); ! if (icc < 0) break; ! if (icc == 0) (void)write(master, ibuf, 0); ! if (icc > 0) { ! (void)write(master, ibuf, icc); ! if (kflg) ! (void)fwrite(ibuf, 1, icc, fscript); } } if (n > 0 && FD_ISSET(master, &rfd)) { ! occ = read(master, obuf, sizeof (obuf)); ! if (occ <= 0) break; ! (void)write(STDOUT_FILENO, obuf, occ); ! if (!kflg || (occ != icc) || memcmp (obuf, ibuf, occ)) { ! (void)fwrite(obuf, 1, occ, fscript); ! icc = 0; ! } } tvec = time(0);
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped