As described in this thread: https://bugs.launchpad.net/ubuntu/+source/netcat-openbsd/+bug/544935 numerous people have, like me, encountered a serious problem with nc(1), specifically that it prematurely closes the connection to the remote network server being accessed as soon as it senses EOF on its stdin channel. As can been see by perusing the above web page, this problem can cause, and has caused much gnashing of teeth and tearing of hair in many quarters. It is certainly possible that some existing scripts and/or applications may perhaps depend on the current/existing behavior, and thus, it may perhaps not be considered wise to change the existing behavior at this (late?) point in time. However there are quite certainly many situations in which the different behavior (of gathering and outputting all output from the remote server before closing down) can be unambiguously useful, if not to say utterly irreplacable. For these reasons I propse the addition of a new "-q" option (no arguments) for nc(1) which, when used would change the program's behavior, causing it to wait for EOF from the remote network server before closing down and exiting. I provide implementation patches for exactly such a change below. How-To-Repeat: echo 193.0.6.139 | nc whois.ripe.net 43 ((note how the output produced by the above command pipeline gets chopped off prematurely)) Fix: Simple/trivial context diff patches (implementing -q) attached below. *** netcat.c 2012-12-03 10:55:02.000000000 -0800 --- netcat.c 2013-03-06 18:10:10.000000000 -0800 *************** *** 82,87 **** --- 82,88 ---- int FreeBSD_Oflag; /* Do not use TCP options */ char *Pflag; /* Proxy username */ char *pflag; /* Localport flag */ + int qflag; /* quit on remote EOF flag */ int rflag; /* Random ports flag */ char *sflag; /* Source Address */ int tflag; /* Telnet Emulation */ *************** *** 153,159 **** sv = NULL; while ((ch = getopt_long(argc, argv, ! "46DdEe:hI:i:jklnoO:P:p:rSs:tT:UuV:vw:X:x:z", longopts, NULL)) != -1) { switch (ch) { case '4': --- 154,160 ---- sv = NULL; while ((ch = getopt_long(argc, argv, ! "46DdEe:hI:i:jklnoO:P:p:qrSs:tT:UuV:vw:X:x:z", longopts, NULL)) != -1) { switch (ch) { case '4': *************** *** 224,229 **** --- 225,233 ---- case 'p': pflag = optarg; break; + case 'q': + qflag = 1; + break; case 'r': rflag = 1; break; *************** *** 823,829 **** if ((n = read(wfd, buf, plen)) < 0) return; else if (n == 0) { ! shutdown(nfd, SHUT_WR); pfd[1].fd = -1; pfd[1].events = 0; } else { --- 827,834 ---- if ((n = read(wfd, buf, plen)) < 0) return; else if (n == 0) { ! if (!qflag) ! shutdown(nfd, SHUT_WR); pfd[1].fd = -1; pfd[1].events = 0; } else {
----- Forwarded message from Fernando Apesteguía <fernando.apesteguia@gmail.com> ----- Date: Sat, 20 Jul 2013 17:58:52 +0200 From: Fernando Apesteguía <fernando.apesteguia@gmail.com> To: "Ronald F. Guilmette" <rfg@tristatelogic.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: bin/176713: [patch] nc(1) closes network socket too soon On Sat, Jul 20, 2013 at 4:31 PM, Ronald F. Guilmette <rfg@tristatelogic.com>wrote: > Could someone please take a look at this bug report (bin/176713) and > also at the simple patch that I provided to fix the problem? > > This is quite a serious problem, and my PR has been pending with no > action since Wed, 6 Mar 2013. > > > Regards, > rfg > > > P.S. Please note that in reality, I do not believe that it is necessary > to add a new -q option in order to preserve backwards compatability, > however > I proposed one because in my experience there is always going to be some > purist who will claim that *any* change in semantics, however small or > helpful, in *any* software ``may'' break something. In reality, the > problem in this can most easily be solved simply by removing the one > statement: > > shutdown(nfd, SHUT_WR); > > entirely from the source, rather than making it conditional on -q, as I > suggested in my PR, bin/176713. > It seems to work for me: Old version: $ echo 193.0.6.139 | nc whois.ripe.net 43 % This is the RIPE Database query service. % The objects are in RPSL format. % % The RIPE Database is subject to Terms and Conditions. % See http://www.ripe.net/db/support/db-terms-conditions.pdf with the patched version: echo 193.0.6.139 | ./nc -q whois.ripe.net 43 % This is the RIPE Database query service. % The objects are in RPSL format. % % The RIPE Database is subject to Terms and Conditions. % See http://www.ripe.net/db/support/db-terms-conditions.pdf % Note: this output has been filtered. % To receive output for a database update, use the "-B" flag. % Information related to '193.0.0.0 - 193.0.7.255' % Abuse contact for '193.0.0.0 - 193.0.7.255' is 'abuse@ripe.net' inetnum: 193.0.0.0 - 193.0.7.255 netname: RIPE-NCC descr: RIPE Network Coordination Centre org: ORG-RIEN1-RIPE descr: Amsterdam, Netherlands remarks: Used for RIPE NCC infrastructure. country: NL admin-c: JDR-RIPE admin-c: BRD-RIPE tech-c: OPS4-RIPE status: ASSIGNED PI source: RIPE # Filtered mnt-by: RIPE-NCC-MNT mnt-lower: RIPE-NCC-MNT organisation: ORG-RIEN1-RIPE org-name: Reseaux IP Europeens Network Coordination Centre (RIPE NCC) org-type: RIR descr: RIPE NCC Operations address: Reseaux IP Europeens Network Coordination Centre (RIPE NCC) P.O. Box 10096 1016 EB Amsterdam Netherlands phone: +31205354444 fax-no: +31205354445 admin-c: AP110-RIPE admin-c: MENN1-RIPE abuse-c: ops4-ripe mnt-ref: RIPE-NCC-MNT mnt-ref: RIPE-NCC-HM-MNT mnt-by: RIPE-NCC-HM-MNT source: RIPE # Filtered role: RIPE NCC Operations address: Singel 258 address: 1016 AB Amsterdam address: The Netherlands phone: +31 20 535 4444 fax-no: +31 20 535 4445 abuse-mailbox: abuse@ripe.net admin-c: JDR-RIPE admin-c: BRD-RIPE tech-c: GL7321-RIPE tech-c: JA47 tech-c: MENN1-RIPE tech-c: EMIL-RIPE tech-c: RCO-RIPE tech-c: APZ-RIPE tech-c: CNAG-RIPE tech-c: RPM-RIPE tech-c: SO2011-RIPE tech-c: RH5357-RIPE tech-c: DCW-RIPE tech-c: FMUL-RIPE nic-hdl: OPS4-RIPE mnt-by: RIPE-NCC-MNT source: RIPE # Filtered person: Brian Riddle address: IT Manager address: RIPE NCC - Operations address: Singel 258 address: 1016AB Amsterdam address: The Netherlands phone: +31 20 535 4444 fax-no: +31 20 535 4445 nic-hdl: BRD-RIPE source: RIPE # Filtered mnt-by: RIPE-NCC-MNT person: Jochem de Ruig address: RIPE NCC address: Singel 258 address: 1016AB Amsterdam address: The Netherlands phone: +31 20 535 4444 fax-no: +31 20 535 4445 nic-hdl: JDR-RIPE source: RIPE # Filtered mnt-by: RIPE-NCC-MNT % Information related to '193.0.0.0/21AS3333' route: 193.0.0.0/21 descr: RIPE-NCC origin: AS3333 mnt-by: RIPE-NCC-MNT source: RIPE # Filtered % This query was served by the RIPE Database Query Service version 1.66.3 (WHOIS2) Tested on: FreeBSD hammer 9.1-RELEASE-p3 FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > > P.P.S. I have just realized that yet one more critical enhancement for > nc is called for, and I will be filing a separate and additional PR for > that soon. I hope that someone will be able to take a look at that as > soon as it is filed. > Just out of curiosity. What is it about? ----- End forwarded message -----
----- Forwarded message from "Ronald F. Guilmette" <rfg@tristatelogic.com> ----- Date: Tue, 23 Jul 2013 16:48:33 -0700 From: "Ronald F. Guilmette" <rfg@tristatelogic.com> To: freebsd-hackers@freebsd.org Subject: Re: bin/176713: [patch] nc(1) closes network socket too soon In message <CAJ-Vmonk-8v9ej0w4qycNNBkieOeE9dL3BTVp6VqipxKH2jSUA@mail.gmail.com> Adrian Chadd <adrian@freebsd.org> wrote: >Right, and your patch just stops the shutdown(), right? The shutdown that occurs when EOF is encountered on stdin, yes. >Rather than >teaching nc to correctly check BOTH socket states before deciding to >close things. In effect, nc *is* currently "checking both sockets" and that is exactly the problem. It terminates (prematurely in some cases) whenever it sees an EOF _either_ from the remote host _or_ from stdin. My patch casuses nc to wait for EOF from the remote server before exiting, EVEN IF prior to the time it sees that EOF from the remote server it sees an EOF (first) on stdin. This code change demonstratably makes the functionality of nc better and more pragmatically useful in typical use cases. You appear to be proposing something else, but I'm sorry to say that I cannot decypher what, exactly you are attempting to propose. I have proposed specific code changes. If you have some different ones that you would like to propose, then I feel sure that everyone on the hackers list, including myself, would be interested to take a look at what you have in mind, and also what problem you are solving. >I'd personally rather see nc taught to check to see whether it can >possibly make ANY more progress before deciding to shut things down. I believe that that is exactly what the patch that I proposed does. I'm not sure why you feel otherwise. Look, there are only two scenarios... either (a) EOF arrives from stdin first or else (b) EOF arrives from the remote server first. My patch causes nc to continue gathering data from the remote server (and copying it all to stdout) in case (a). In case (b) there is no point in nc continuing to run (and/or continuing to read from stdin) if the remote server has shut down the connection. In this case, the data that nc might yet gather from its stdin channel has noplace to go! So whenever nc has sensed an EOF from the remote server it can (and should) immediately shut down... and that is exactly what it is _already_ programmed to do. So, what problem do you want to solve that is not solved by the patch that I already proposed? Also, with respect, if you think there really is some other problem, then proposing actual concrete patches to solve that other problem would perhaps allow folks, including myself, to better understand what it is that you are driving at. Regards, rfg _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" ----- End forwarded message -----
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped
Keyword: patch or patch-ready – in lieu of summary line prefix: [patch] * bulk change for the keyword * summary lines may be edited manually (not in bulk). Keyword descriptions and search interface: <https://bugs.freebsd.org/bugzilla/describekeywords.cgi>