Bug 232201 - sh(1): Using unset variables in here-doc with set -u does not cause the script to exit
Summary: sh(1): Using unset variables in here-doc with set -u does not cause the scrip...
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
Keywords: patch
Depends on:
Reported: 2018-10-12 12:49 UTC by Mateusz Piotrowski
Modified: 2020-05-25 22:43 UTC (History)
1 user (show)

See Also:

experimental patch to stop ignoring here-document expansion errors (1.75 KB, patch)
2019-03-30 21:46 UTC, Jilles Tjoelker
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mateusz Piotrowski freebsd_committer 2018-10-12 12:49:12 UTC
The output of sh(1) from the base and of dash from the ports (shells/dash) is different.

Here's the script to reproduce this issue:

#! /bin/sh -

set -u

# Unset variable.
cat << EOF > /dev/null && echo ok || echo error

Output of `sh test.sh`:

> testcase-2.sh: unsetvar: parameter not set
> ok

Output of `dash test.sh`:

> testcase-2.sh: 6: testcase-2.sh: unsetvar: parameter not set
> error

I'm running sh(1) from 12.0-ALPHA6 r338797 and dash from ports.
Comment 1 Jilles Tjoelker freebsd_committer 2018-10-26 11:07:47 UTC
Although dash is not necessarily a good reference (since it has various bugs and shortcuts of itself), I checked this against bash --posix and ksh93 and the common behaviour seems to be that an expansion error in a here-document is treated as a redirection error (so the redirected command is not executed and for some types of command the shell aborts).

When I last changed here-document expansion in SVN r246288, I left the error and side effect behaviour unchanged from what it was to minimize the risk of breaking existing scripts.

A related test case:
sh -c 'd=/dev/null; : <"${d#$((a=1))}"; true <"${d#$((b=2))}"; /usr/bin/true <"${d#$((c=3))}"; echo "a=$a b=$b c=$c"'

Most shells seem to agree that this should output a=1 b=2 c=3.

Bash (POSIX mode as well as default mode) has different behaviour between true (keeps side effects) and /usr/bin/true (discards side effects). This seems incorrect.
Comment 2 Jilles Tjoelker freebsd_committer 2018-12-07 22:30:53 UTC
I think the behaviour here can be improved, but how it should work is not immediately clear. This should take into account our previous behaviour, the behaviour of other shells and POSIX (with the most recent interpretations). The effect of expansions and errors in redirections varies depending on what kind of command the redirection is applied to (special builtin, other simple command, subshell, compound command).

Different from what I wrote in my previous comment, bash also has a different error behaviour between true (shell aborts) and /usr/bin/true (shell continues with non-zero exit status).
Comment 3 Jilles Tjoelker freebsd_committer 2019-03-30 21:46:59 UTC
Created attachment 203269 [details]
experimental patch to stop ignoring here-document expansion errors

The simplest thing that could possibly work is to treat an expansion error in a here-document the same way as an expansion error in a redirection filename or file descriptor number. Such an error will cause the script or subshell to be aborted ('command .' and 'command eval' can also "catch" such an error).

Trying to build a few ports using that does not appear to cause breakage.