Bug 25542
| Summary: | sh(1) null char in quoted string | ||
|---|---|---|---|
| Product: | Base System | Reporter: | val <val> |
| Component: | bin | Assignee: | freebsd-bugs (Nobody) <bugs> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | ||
| Priority: | Normal | ||
| Version: | 4.1-RELEASE | ||
| Hardware: | Any | ||
| OS: | Any | ||
Responsible Changed From-To: gnats-admin->freebsd-bugs over to bugs Responsible Changed From-To: gnats-admin->freebsd-bugs over to bugs Responsible Changed From-To: gnats-admin->freebsd-bugs over to bugs Responsible Changed From-To: freebsd-bugs->standards This might be standards@ fodder. Are NULs in this way defined/undefined behavior? Responsible Changed From-To: standards->freebsd-standards Misattributed PR. I've checked this on 6.2-REL and 7.0-PRE using: echo -e "\047O\000K\047" | od -c Result: 0000000 ' O \0 K ' \n 0000006 This is what I expect to come out and it shows, sh is handling NULL bytes correctly ATM. Please follow-up with your opinion , otherwise this PR can be closed. State Changed From-To: open->feedback Set to feedback. volker@vwsoft.com cannot reproduce this with recent builds of 6.X and 7.X, so if this is not a problem anymore we can close the ticket. State Changed From-To: feedback->closed Close as submitter e-mail address bounces. If you are the submitter and want to re-open this PR, please follow up with a new e-mail address so that the PR can be updated. Please also include further debugging information as we cannot reproduce the probelm on more recent FreeBSD versions. Thanks for the report! State Changed From-To: closed->open Reopen, this can still be recreated on 7.0: $ echo -e "echo \047O\000K\047" | /bin/sh | od -c 0000000 O K n 0000003 Considering that fixing this would be a lot of work and cannot be done completely (for example, argument strings and environment variables cannot contain '\0'), I think it is best to close this. sh(1) is meant to process text, not binary data. Trying to process binary data may or will also cause problems if the locale character set is set to UTF-8. As a clarification, this PR is about '\0' bytes in shell scripts, not about making the echo builtin produce '\0' characters. The latter feature works fine and is good. -- Jilles Tjoelker At Sat, 4 Apr 2009 14:41:43 +0200, Jilles Tjoelker wrote:
>Considering that fixing this would be a lot of work and cannot be done
>completely (for example, argument strings and environment variables
>cannot contain '\0'), I think it is best to close this.
I think, at least documentation issue mentioned in the original PR 25542
can be corrected easy. Here is the patch:
--- src/bin/sh/sh.1.orig 2007-12-05 17:29:07.000000000 +0300
+++ src/bin/sh/sh.1 2009-07-27 22:36:39.000000000 +0400
@@ -2381,4 +2381,6 @@
.Sh BUGS
The
.Nm
-utility does not recognize multibyte characters.
+utility does not recognize multibyte characters.
+ASCII <NULL> character in input strings, parameters etc. can be mishandled by
+.Nm .
Author: jilles Date: Sun Sep 20 21:42:38 2009 New Revision: 197371 URL: http://svn.freebsd.org/changeset/base/197371 Log: Mention that NUL characters are not allowed in sh(1) input. I do not consider this a bug because POSIX permits it and argument strings and environment variables cannot contain '\0' anyway. PR: bin/25542 MFC after: 2 weeks Modified: head/bin/sh/sh.1 Modified: head/bin/sh/sh.1 ============================================================================== --- head/bin/sh/sh.1 Sun Sep 20 19:32:10 2009 (r197370) +++ head/bin/sh/sh.1 Sun Sep 20 21:42:38 2009 (r197371) @@ -375,6 +375,10 @@ introduces a comment if used at the begi The word starting with .Ql # and the rest of the line are ignored. +.Pp +.Tn ASCII +.Dv NUL +characters (character code 0) are not allowed in shell input. .Ss Quoting Quoting is used to remove the special meaning of certain characters or words to the shell, such as operators, whitespace, keywords, _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" Responsible Changed From-To: freebsd-standards->jilles Take. Author: jilles Date: Sun Oct 4 17:16:11 2009 New Revision: 197748 URL: http://svn.freebsd.org/changeset/base/197748 Log: MFC r197371: Mention that NUL characters are not allowed in sh(1) input. I do not consider this a bug because POSIX permits it and argument strings and environment variables cannot contain '\0' anyway. PR: bin/25542 Modified: stable/7/bin/sh/ (props changed) stable/7/bin/sh/sh.1 Modified: stable/7/bin/sh/sh.1 ============================================================================== --- stable/7/bin/sh/sh.1 Sun Oct 4 16:30:33 2009 (r197747) +++ stable/7/bin/sh/sh.1 Sun Oct 4 17:16:11 2009 (r197748) @@ -375,6 +375,10 @@ introduces a comment if used at the begi The word starting with .Ql # and the rest of the line are ignored. +.Pp +.Tn ASCII +.Dv NUL +characters (character code 0) are not allowed in shell input. .Ss Quoting Quoting is used to remove the special meaning of certain characters or words to the shell, such as operators, whitespace, keywords, _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" State Changed From-To: open->patched I will merge this to stable/8 after 8.0 release. Author: jilles Date: Thu Oct 29 21:13:57 2009 New Revision: 198624 URL: http://svn.freebsd.org/changeset/base/198624 Log: MFC r197371: Mention that NUL characters are not allowed in sh(1) input. I do not consider this a bug because POSIX permits it and argument strings and environment variables cannot contain '\0' anyway. PR: bin/25542 Modified: stable/8/bin/sh/ (props changed) stable/8/bin/sh/sh.1 Modified: stable/8/bin/sh/sh.1 ============================================================================== --- stable/8/bin/sh/sh.1 Thu Oct 29 20:53:26 2009 (r198623) +++ stable/8/bin/sh/sh.1 Thu Oct 29 21:13:57 2009 (r198624) @@ -375,6 +375,10 @@ introduces a comment if used at the begi The word starting with .Ql # and the rest of the line are ignored. +.Pp +.Tn ASCII +.Dv NUL +characters (character code 0) are not allowed in shell input. .Ss Quoting Quoting is used to remove the special meaning of certain characters or words to the shell, such as operators, whitespace, keywords, _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" Responsible Changed From-To: jilles->freebsd-bugs I'm not interested in 6.x. State Changed From-To: patched->closed Previously merged to 8 and 7, and I've merged to 6 now too for completeness. Author: emaste Date: Sun Jan 17 14:11:42 2010 New Revision: 202484 URL: http://svn.freebsd.org/changeset/base/202484 Log: Log: MFC r197371: Mention that NUL characters are not allowed in sh(1) input. PR: bin/25542 Modified: stable/6/bin/sh/sh.1 Directory Properties: stable/6/bin/sh/ (props changed) Modified: stable/6/bin/sh/sh.1 ============================================================================== --- stable/6/bin/sh/sh.1 Sun Jan 17 13:49:18 2010 (r202483) +++ stable/6/bin/sh/sh.1 Sun Jan 17 14:11:42 2010 (r202484) @@ -370,6 +370,10 @@ introduces a comment if used at the begi The word starting with .Ql # and the rest of the line are ignored. +.Pp +.Tn ASCII +.Dv NUL +characters (character code 0) are not allowed in shell input. .Ss Quoting Quoting is used to remove the special meaning of certain characters or words to the shell, such as operators, whitespace, keywords, _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" |
/bin/sh improperly handles null characters within quoted strings. This behaviour is not documented in the man page, where the only char prohibited within single-quoted string is said to be the single quote itself. Fix: Avoid usage of the null(or whatever else hardcoded)-terminated strings in programs which deal with character strings. See awk or perl for the implementation details. Workaround: explicity state in the documentation that <NULL> character is prohibited for use in the shell parameters. How-To-Repeat: The command echo -e "echo \047O\000K\047" | /bin/sh yields only three bytes on stdout: "O", "K" and <LF>, whereas it must produce four of them: "O", <NULL>, "K" and <LF>, as the command echo -e 'O\000K' does. Another example is given by the following gawk script (utilizing the fact that Gnu awk calls /bin/sh to interpret the external system commands): #!/usr/bin/awk -f BEGIN{print system("echo 'O\000K'")}