Bug 266791 - bc does not flush output before exit when input is a file argument
Summary: bc does not flush output before exit when input is a file argument
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 13.1-RELEASE
Hardware: Any Any
: --- Affects Some People
Assignee: Stefan Eßer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-10-03 14:12 UTC by Jason W. Bacon
Modified: 2022-11-02 15:24 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jason W. Bacon freebsd_committer freebsd_triage 2022-10-03 14:12:35 UTC
> cat test.bc 
sqrt(2)
quit

> bc -l test.bc

> bc -l < test.bc
1.41421356237309504880

On NetBSD, both commands produce the same output.
Comment 1 Stefan Eßer freebsd_committer freebsd_triage 2022-10-05 17:52:28 UTC
(In reply to Jason W. Bacon from comment #0)

I'll forward this bug report to Gavin D. Howard, the author of this bc implementation.
Comment 2 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-11 16:30:16 UTC
Thanks.

FYI, the issue does not occur on 12.3-RELEASE.
Comment 3 Gavin D. Howard 2022-10-12 01:18:18 UTC
I'm the original author.

I have pushed two branches to my Gitea instance that fix the issue, but they do so in different ways because of a possible ambiguity in how to interpret the bc standard.

One way (the `line_by_line` branch) matches what GNU and NetBSD bc implementations do, but in my opinion, they are not following the standard. The other (the `stmt_by_stmt` branch) does. In my opinion, the difference between NetBSD/GNU and this branch is not worrying about because it will only happen if `quit` is after other statements and *on the same line*. I don't think that will happen often in practice.

> FYI, the issue does not occur on 12.3-RELEASE.

@jwb, I'm confused why this issue does not occur in 12.3-RELEASE. As far as I know, that should only be the case if WITHOUT_GH_BC is used to build because my bc has had this problem since the beginning (due to an interpretation of the standard I held until recently).

Could you give me more information?

Also, could you test both of those branches?

Which of those branches is chosen is mostly up to Stefan and any other FreeBSD committers that care. I'll merge either one and put out a release for Stefan to update the base system.
Comment 4 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-12 11:33:55 UTC
On 12.3, the man page lists different authors, so I'm assuming it's not using this implementation.  Correct me if I'm wrong...

I'll check out the patches when I get a chance.  Thanks.
Comment 5 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-12 12:09:33 UTC
For me, the stmt_by_stmt branch works with quit on the same line or separate.

The line_by_line branch only works with quit on a separate line.  The following produces no output:

sqrt(2); quit

Thanks for the quick response...
Comment 6 Gavin D. Howard 2022-10-12 14:26:40 UTC
> On 12.3, the man page lists different authors, so I'm assuming
> it's not using this implementation.  Correct me if I'm wrong...

You are absolutely correct.

If different authors are listed, then it's a different bc. My guess is that your 12.3-RELEASE was built using WITHOUT_GH_BC, which would use a different one. But that's only a guess, and I wouldn't know how to tell. (I'm not a regular FreeBSD user.)

> For me, the stmt_by_stmt branch works with quit on the same line or separate.

Great!

> The line_by_line branch only works with quit on a separate line.
> The following produces no output:

> sqrt(2); quit

This is intentional; that is why there are two branches.

I personally prefer the behavior of stmt_by_stmt because I believe it matches the bc standard. However, the line_by_line behavior matches the behavior of the NetBSD and GNU bc implementations.

Since this bug originated with a behavior discrepancy between NetBSD and mine, I wasn't sure which behavior you or Stefan would prefer; in fact, I thought you might prefer line_by_line. Thus, I implemented both, so you could choose.

I'm happy that you seem to prefer stmt_by_stmt, though!
Comment 7 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-12 15:26:09 UTC
I can confirm that on NetBSD:

NetBSD netbsd9.acadix  bacon ~ 1012: (pkgsrc): cat same.bc 
sqrt(2); quit
NetBSD netbsd9.acadix  bacon ~ 1013: (pkgsrc): bc -l same.bc 
NetBSD netbsd9.acadix  bacon ~ 1014: (pkgsrc): cat separate.bc 
sqrt(2)
quit
NetBSD netbsd9.acadix  bacon ~ 1015: (pkgsrc): bc -l separate.bc 
1.41421356237309504880

I think the line_by_line behavior is a bug in NetBSD and GNU that people just don't run into.
Comment 8 Gavin D. Howard 2022-10-12 15:43:52 UTC
> I think the line_by_line behavior is a bug in NetBSD and GNU
> that people just don't run into.

I agree.

@se, do you agree with @jwb to go with stmt_by_stmt?
Comment 9 Gavin D. Howard 2022-10-12 22:38:57 UTC
After reading the standard *yet again*, I'm now not sure which interpretation is correct.

The relevant portions say:

> The quit statement (quit) shall stop execution of a bc program at the
> point where the statement occurs in the input, even if it occurs in a
> function definition, or in an if, for, or while statement.

and

> Statements separated by <semicolon> or <newline> characters shall be
> executed sequentially. In an interactive invocation of bc, each time a
> <newline> is read that satisfies the grammatical production:
> 
> ```
> input_item : semicolon_list NEWLINE
> ```
> 
> the sequential list of statements making up the semicolon_list shall
> be executed immediately and any output produced by that execution
> shall be written without any delay due to buffering.

The phrase "where the [quit] statement occurs in the input" (in the first quote) seems to imply statement-by-statement behavior.

However, the phrase "the sequential list of statements making up the semicolon_list shall be executed immediately" (in the second quote, last paragraph) seems to imply that the series of statements on the same line should be executed as a unit, which would imply that the quit statement would be executed *at the same time* and thus, take precedence, preventing any previous statements on the line from executing.

To my brain, those quotes contradict each other.

I don't know what the correct behavior is, which would mean that the correct behavior will be up to FreeBSD.

Another hang-up is an example @se came up with:

```
for (i = 0; i < 3; ++i) i; \
quit
```

The backslash+newline combo causes the two lines to be treated the same. I think this should have the same behavior as:

```
for (i = 0; i < 3; ++i) i; quit
```

and both branches do just that.

Anyway, this is one of those few times with bc that I may not have much of an opinion and will do whatever I am asked.
Comment 10 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-13 12:58:47 UTC
I think the clarifying statement in the specs is

"Statements separated by <semicolon> or <newline> characters shall be
executed sequentially."

I would not interpret "immediately" as "at the same time".  It sounds to me like they are saying "[each statement in] the sequential list of statements making  up the semicolon_list shall be executed immediately and any output produced by that execution shall be written without any delay due to buffering".  So they were clarifying that a semicolon should have exactly the same effect as a newline: The statement is executed immediately and output is flushed every time a ; is encountered, just like a newline.  With this interpretation, everything is in agreement.

It also would not make sense to drop a statement on a line before a quit.

Thanks for being so thorough.  You're attention to detail will undoubtedly prevent a lot of headaches.
Comment 11 Gavin D. Howard 2022-10-14 01:09:59 UTC
In an series of emails with @se, he and I have agreed that the stmt_by_stmt branch still has the preferred behavior, so I will start preparing that branch for merge, testing, and release, at which point, it should be merged into the FreeBSD 13 series and may also make it in time for the 12.4 merge window.
Comment 12 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-14 14:02:45 UTC
Excellent, thanks.

And heads up on another bug:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267051
Comment 13 Gavin D. Howard 2022-10-14 14:43:29 UTC
Thank you for letting me know! I've replied to the other bug now.

Also, thank you for the compliments; I do hope to keep FreeBSD users satisfied.
Comment 14 Gavin D. Howard 2022-10-21 19:22:07 UTC
I apologize that this is still open.

The fuzzer found a crash that required a redesign. [1]

I have managed to fix it, though, and I am restarting the release process. I hope to have a release out in a week or so.

[1]: https://git.yzena.com/gavin/bc/commit/9573cc5c4cc7a7efbee08f6a2046ef0bc1d4ac01
Comment 15 Jason W. Bacon freebsd_committer freebsd_triage 2022-10-22 14:34:03 UTC
SE@: Might this affect the 12.4 release, or will 12.x continue to to use the older implementation?
Comment 16 Gavin D. Howard 2022-11-02 12:53:54 UTC
I have released 6.1.0 with the changes.
Comment 17 Jason W. Bacon freebsd_committer freebsd_triage 2022-11-02 15:24:01 UTC
(In reply to Gavin D. Howard from comment #16)
Awesome, thanks!