Bug 204156 - fmemopen tests failing after r289863
Summary: fmemopen tests failing after r289863
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Many People
Assignee: Andrey A. Chernov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-10-30 17:27 UTC by Ed Maste
Modified: 2015-11-08 13:53 UTC (History)
3 users (show)

See Also:
ache: mfc-stable10+


Attachments
Source file (stripped of "ATF-ness") (3.25 KB, text/x-csrc)
2015-11-01 05:14 UTC, Enji Cooper
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer 2015-10-30 17:27:29 UTC
https://people.freebsd.org/~andrew/arm64/kyua/

Test case: lib/libc/stdio/fmemopen2_test:test_data_length

    Test program: /usr/tests/lib/libc/stdio/fmemopen2_test
    Result: failed: /usr/src/lib/libc/tests/stdio/fmemopen2_test.c:203: pos == 0 not met
    Duration: 0.052s
    Execution context

Metadata

    allowed_architectures =
    allowed_platforms =
    description =
    has_cleanup = false
    required_configs =
    required_disk_space = 0
    required_files =
    required_memory = 0
    required_programs =
    required_user =
    timeout = 300

Standard output
Test case did not write anything to stdout.
Standard error
Test case did not write anything to stderr. 


n_test:test09

    Test program: /usr/tests/lib/libc/stdio/fmemopen_test
    Result: failed: 3956 checks failed; see output for more details
    Duration: 0.197s
    Execution context

Metadata

    allowed_architectures =
    allowed_platforms =
    description = test09
    has_cleanup = false
    required_configs =
    required_disk_space = 0
    required_files =
    required_memory = 0
    required_programs =
    required_user =
    timeout = 300

Standard output
Test case did not write anything to stdout.
Standard error

*** Check failed: /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_fmemopen.c:437: ftello(fp) == i not met
*** Check failed: /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_fmemopen.c:437: ftello(fp) == i not met
... many failures ...


Test case: lib/libc/stdio/fmemopen_test:test11

    Test program: /usr/tests/lib/libc/stdio/fmemopen_test
    Result: failed: 6020 checks failed; see output for more details
    Duration: 0.327s
    Execution context

Metadata

    allowed_architectures =
    allowed_platforms =
    description = test11
    has_cleanup = false
    required_configs =
    required_disk_space = 0
    required_files =
    required_memory = 0
    required_programs =
    required_user =
    timeout = 300

Standard output
Test case did not write anything to stdout.
Standard error

*** Check failed: /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_fmemopen.c:541: ftello(fp) == len + i not met
*** Check failed: /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_fmemopen.c:541: ftello(fp) == len + i not met
... many failures ...



Test case: lib/libc/stdio/fmemopen_test:test14

    Test program: /usr/tests/lib/libc/stdio/fmemopen_test
    Result: failed: 1806 checks failed; see output for more details
    Duration: 0.126s
    Execution context

Metadata

    allowed_architectures =
    allowed_platforms =
    description = test14
    has_cleanup = false
    required_configs =
    required_disk_space = 0
    required_files =
    required_memory = 0
    required_programs =
    required_user =
    timeout = 300

Standard output
Test case did not write anything to stdout.
Standard error

*** Check failed: /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_fmemopen.c:731: ftello(fp) == len - i not met
*** Check failed: /usr/src/contrib/netbsd-tests/lib/libc/stdio/t_fmemopen.c:731: ftello(fp) == len - i not met
Comment 1 Andrew Turner freebsd_committer 2015-10-31 15:24:08 UTC
I've looked at test09 and think it's failure is a result of https://svnweb.freebsd.org/changeset/base/289863. The test calls fmemopen with the append flag set, then tries to seek on it. I'm not sure what the correct behaviour is, other than it seems to have changed.

This is not arm64 specific.
Comment 2 Andrey A. Chernov freebsd_committer 2015-11-01 04:11:57 UTC
r289863 may affect position only when write happens, but all test09, test11 and test14 never write anything.

I don't have ATF compiled and attempt to write plain C program which does the same as most failing cases from test09,11,14 and it not fails on -current. 

Moreover, something is fishing here, I can't even find fmemopen2_test.c file in my -current for lib/libc/stdio/fmemopen2_test:test_data_length mentioned. Is your ATF copy not damaged/modified somehow? 

BTW, I already meet ATF-specific bugs before, so prefer plain C test cases.
Comment 3 Andrey A. Chernov freebsd_committer 2015-11-01 04:33:02 UTC
(In reply to Andrey A. Chernov from comment #2)
Forget about fmemopen2_test.c, just found (another dir). Theoretically it is the only case when fwrite() happens so r289863 may affect it. I'll write plain C test a bit later and report.
Comment 4 Andrey A. Chernov freebsd_committer 2015-11-01 05:02:40 UTC
(In reply to Andrey A. Chernov from comment #2)
The last case:
failed: /usr/src/lib/libc/tests/stdio/fmemopen2_test.c:203: pos == 0 not met
not failed in -current when written in plain C:

fp = fmemopen(buf, sizeof(buf), "a+");
fwrite(str2, 1, sizeof(str2), fp);
fseek(fp, 0, SEEK_SET);
printf("pos %d\n", ftell(fp) == 0);

So I think all of this is ATF-specific bugs, probably with wrong type cast in their macros, and I don't want to debug ATF. If you think otherwise, please post plain C test which fails after r289863. 

IMHO, ATF is evil and can't be used to prove anything, only to extract from there and convert to plain C.
Comment 5 Enji Cooper freebsd_committer 2015-11-01 05:14:31 UTC
Created attachment 162664 [details]
Source file (stripped of "ATF-ness")
Comment 6 Enji Cooper freebsd_committer 2015-11-01 05:17:00 UTC
Don't blame ATF.

I run into the same issues on amd64 once I install new system headers and a new libc, not before then.

I've added a source file and I can point you to a core if you need one, but compiling fmemopen2_test.c like the following should work as well:

cc -O0 -g -Wall -o fmemopen2_test fmemopen2_test.c
Comment 7 Enji Cooper freebsd_committer 2015-11-01 05:19:47 UTC
$ gdb710 ~/fmemopen2_test fmemopen2_test.core 
GNU gdb (GDB) 7.10 [GDB v7.10 for FreeBSD]
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd11.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/ngie/fmemopen2_test...done.
[New process 100105]
Core was generated by `fmemopen2_test'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00000008008dd2ca in thr_kill () from /lib/libc.so.7
(gdb) bt
#0  0x00000008008dd2ca in thr_kill () from /lib/libc.so.7
#1  0x00000008008dd29b in raise () from /lib/libc.so.7
#2  0x00000008008dd209 in abort () from /lib/libc.so.7
#3  0x00000008009293d1 in __assert () from /lib/libc.so.7
#4  0x0000000000400d79 in main () at /home/ngie/fmemopen2_test.c:94
(gdb) list 94
89              rc = fseek(fp, 0, SEEK_SET);
90              ATF_REQUIRE(rc == 0);
91
92              /* Make sure we're at the beginning. */
93              pos = ftell(fp);
94              ATF_REQUIRE(pos == 0);
95
96              /* Read the whole buffer. */
97              nofr = fread(str3, 1, sizeof(buf), fp);
98              ATF_REQUIRE(nofr == sizeof(str3));
(gdb) frame 4
(gdb) p *fp                                                                                                                                                                                                 
$1 = {_p = 0x800bb60b7 "", _r = 0, _w = 0, _flags = 282, _file = -1, _bf = {_base = 0x800bb60b7 "", _size = 1}, _lbfsize = 0, _cookie = 0x800e15000, _close = 0x80086dd50, _read = 0x80086dc10, 
  _seek = 0x80086dce0, _write = 0x80086dc60, _ub = {_base = 0x0, _size = 0}, _up = 0x0, _ur = 0, _ubuf = "\000\000", _nbuf = "", _lb = {_base = 0x0, _size = 0}, _blksize = 0, _offset = 0, 
  _fl_mutex = 0x0, _fl_owner = 0x0, _fl_count = 0, _orientation = -1, _mbstate = {__mbstate8 = '\000' <repeats 127 times>, _mbstateL = 0}, _flags2 = 0}
(gdb) p pos
$2 = 44
Comment 8 Enji Cooper freebsd_committer 2015-11-01 05:20:19 UTC
So yeah... you broke rewinding with fseek somehow.
Comment 9 commit-hook freebsd_committer 2015-11-01 06:15:54 UTC
A commit references this bug:

Author: ache
Date: Sun Nov  1 06:15:14 UTC 2015
New revision: 290230
URL: https://svnweb.freebsd.org/changeset/base/290230

Log:
  Don't seek to the end if write buffer is empty (in append modes).

  PR:             204156
  MFC after:      1 week

Changes:
  head/lib/libc/stdio/ftell.c
Comment 10 commit-hook freebsd_committer 2015-11-01 06:47:57 UTC
A commit references this bug:

Author: ache
Date: Sun Nov  1 06:47:06 UTC 2015
New revision: 290231
URL: https://svnweb.freebsd.org/changeset/base/290231

Log:
  Addition to prev. commit.
  In some edge cases fp->_p can be changed in _sseek(), recalculate.

  PR:     204156
  MFC after:      1 week

Changes:
  head/lib/libc/stdio/ftell.c
Comment 11 Andrey A. Chernov freebsd_committer 2015-11-01 07:48:32 UTC
(In reply to NGie Cooper from comment #5)
Thanx for stripped fmemopen2_test.c, I found the bug.

It is not ATF nor recent fmemopen() change, it is 1 year old ftell() bug with all append mode files (after attempt to integrate Apple patch).
Should be fixed in just committed r290230-r290231.

Other tests using append mode and ftell() should be fixed too, tell me if not.
Comment 12 Enji Cooper freebsd_committer 2015-11-01 07:55:22 UTC
LGTM now -- thank you for the quick fix:

# (cd /usr/tests/lib/libc/stdio; sudo kyua test)
...
51/51 passed (0 failed)
#

Please close the bug unless you plan on MFCing the r289863.
Comment 13 commit-hook freebsd_committer 2015-11-08 13:37:35 UTC
A commit references this bug:

Author: ache
Date: Sun Nov  8 13:37:17 UTC 2015
New revision: 290544
URL: https://svnweb.freebsd.org/changeset/base/290544

Log:
  MFC: 	r289863,r289931,r290110,r290230,r290231,r290232

  r290232:

  Microoptimize.

  r290231:

  Addition to prev. commit.
  In some edge cases fp->_p can be changed in _sseek(), recalculate.

  r290230:

  Don't seek to the end if write buffer is empty (in append modes).
  PR:     204156

  r290110:

  Add _flags2 per jhb@ suggestion since no room left in _flags.
  Rewrite O_APPEND flag checking using new __S2OAP flag.

  r289931:

  According to POSIX, a write operation shall start at the current size of
  the stream (if mode had 'a' as the first character).

  r289863:

  Since no room left in the _flags, reuse __SALC for O_APPEND.
  It helps to remove _fcntl() call from _ftello() and optimize seek position
  calculation in _swrite().

Changes:
_U  stable/10/
  stable/10/include/stdio.h
  stable/10/lib/libc/stdio/fdopen.c
  stable/10/lib/libc/stdio/findfp.c
  stable/10/lib/libc/stdio/fmemopen.c
  stable/10/lib/libc/stdio/fopen.c
  stable/10/lib/libc/stdio/freopen.c
  stable/10/lib/libc/stdio/ftell.c
  stable/10/lib/libc/stdio/stdio.c