After an upgrade from 12.2 to 13.0-BETA3, python's virtualenvwrapper stopped working with a sed error. Source seems to be a pipeline akin to this: > echo test/bin/activate | tr "\n" " " | sed "s|/bin/activate |/|g" | tr "/" "\n" | sed "s/^\s*$/d" sed: 1: "/^\s*$/d": RE error: trailing backslash (\) If I mount the 12.2 boot environment and replace the sed calls in the above with /tmp/be<RANDOM>/usr/bin/sed, it works, as it has on 12.2: > echo test/bin/activate | tr "\n" " " | /tmp/be_mount.6Hlw/usr/bin/sed "s|/bin/activate |/|g" | tr "/" "\n" | /tmp/be_mount.6Hlw/usr/bin/sed "/^\s*$/d" test
Note that while the bug is actual, there is an error in your reproducer, `sed "s/^\s*$/d"` is wrong, there should not be an 's' command at the beginning.
(In reply to freebsd from comment #0) Regardless of how 13 ends up, you should patch this to replace \s with [[:space:]]. This expression is not doing what the authors intended it to do on any other version of FreeBSD, and the new logic is only pointing that out.
A simpler test case is just: printf "\n\n\n\n" | sed "/^\s*$/d" Expected behaviour is that the empty lines will be filtered out.
I believe the error is raised in p_simp_re() in /usr/src/lib/libc/regex/regcomp.c.
(In reply to David Schlachter from comment #4) Indeed, the error is quite intentional and to catch unexpected behavior like this.
(In reply to Kyle Evans from comment #5) Do you mean this is expected behaviour now? Because from a user's point of view, this error message is rather unhelpful: what trailing backslash? I can report that changing \s to [[:space:]] is working. So how to proceed? Should I report this to virtualenvwrapper's upstream? Should the port maintainer add a patch?
(In reply to freebsd from comment #6) Yes, the message leaves a lot to be desired, unfortunately; I've not yet been able to study the impact of changing the message since this reused a preexisting "escape error" condition. Indeed, ideally this would both go upstream and to the local port maintainer as a POSIX compliance issue.
`\s` is very much a de-facto standard and should remain supported in 13.x – even if it's not written down in POSIX. GNU sed also has this, and the more their feature sets drift apart, the more annoying any cross-platform scripting becomes.
(In reply to Andreas Sommer from comment #8) \s isn't supported in FreeBSD versions < 13.x, either, so this isn't a "remain supported" but "let's support this." As I noted previously, it doesn't do what you think it does in earlier versions.
\s has worked for me pre FreeBSD 13.0. You claim that it did not actually do what I thought it would to but it *did* in fact match whitespace characters. It surely looks like a regression to me.
GNU sed: $ echo 'a b' | gsed 's/\s/-space-/g' a-space-b FreeBSD 11: $ echo 'a b' | sed 's/\s/-space-/g' a b FreeBSD 12: $ echo 'a b' | sed 's/\s/-space-/g' a b FreeBSD 13: $ echo 'a b' | sed 's/\s/-space-/g' sed: 1: "s/\s/-space-/g": RE error: trailing backslash (\)
(In reply to David Schlachter from comment #3) > printf "\n\n\n\n" | sed "/^\s*$/d" Note that this is not actually testing \s since we're matching 0 or more. For example, try (on 12.x or earlier): $ printf "\n\n\n\n" | sed "/^x*$/d" This also deletes the blank lines. Or, try: $ printf " \n \n \n \n" | sed "/^\s*$/d" and observe that the except-for-whitespace empty lines are not stripped.
So far, neither the port maintainer nor upstream have reacted to this issue. Effectively, the port is broken as-is, but the fix is a simple one line change in virtualenvwrapper.sh (replace \s with [[:space:]] in sed). Could someone with commit rights to the ports do that?
(In reply to Kyle Evans from comment #5) What is the change required to resolve this issue as reported? Works as Intended / Not A Bug, or potentially improve messaging, or something else, and track that here?
(In reply to Kubilay Kocak from comment #14) Ideally someone should fix virtualenvwrapper to use a POSIX-conformant expression like I described back in comment 2.
By the way, has that been taken into account for the handling of backslashes on sh side? echo a\ b | sed s/\\s/-space-/g
Reading this thread it seems that \s (and I note also \S) isn't supported by our sed whereas they are supported as whitespace and not whitespace in GNU sed. While one could work around this, as has been suggested preciously, would it be a good idea to support this? Are there other escape sequences that GNU sed supports that we don't? If patches that support these GNUisms were submitted and were not otherwise objectionable would they be likely to be committed?
(In reply to crb from comment #17) It's a one-liner to link sed against libregex instead of libc regex, bringing it up to parity with grep (i.e., \w, \W, \s, \S, \b, \B) -- the problem is that it might break a lot of things, it adds repetition operators to BREs that weren't previously there. If someone bogusly escaped one in the past that suddenly gets special behavior, we have to fix that.
This also broke the OpenLDAP test suite when running on FreeBSD 13: Testing account lockout... sed: 1: "s/.*seconds_before_unlo ...": RE error: trailing backslash (\) ldapsearch failed (49)! Affected line is: DELAY=`echo "$DELAYATTR" | sed -n -e 's/.*seconds_before_unlock=\(\d*\)/\1/p'`
(In reply to Quanah Gibson-Mount from comment #19) Note: fixed by using [[:digit:]] in OpenLDAP instead following the standard.
(In reply to Quanah Gibson-Mount from comment #20) Thanks; [[:digit:]] is indeed preferred, though I didn't know about \d when I implemented these extensions in libregex so I'll need to add that one to the list to support.
(In reply to Kyle Evans from comment #21) \d doesn't actually seem to be supported in GNU sed, either? % echo "n9ne" | sed -e 's/\d/x/' n9ne % sed --version sed (GNU sed) 4.8 It's just that they're not as strict as we now are about bogus escapes, so this was a hidden bug.
MARKED AS SPAM
The comment from Sara Adam spam, don't follow the link