Bug 191354 - [tests] tools/regression/lib/libc/regex fails to link after xlocale changes done in 10.x+
Summary: [tests] tools/regression/lib/libc/regex fails to link after xlocale changes d...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.0-STABLE
Hardware: Any Any
: --- Affects Some People
Assignee: Enji Cooper
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-25 04:31 UTC by Enji Cooper
Modified: 2015-12-20 00:48 UTC (History)
1 user (show)

See Also:
ngie: mfc-stable9+
ngie: mfc-stable10+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2014-06-25 04:31:54 UTC
The lib/libc/regex/grot code is used by tools/regression/lib/libc/regex. Unfortunately the changes made for xlocale in 10.x broke them so they won't link anymore.

I tried to make it work, but it required a bit more than 30 minutes to work through the issues.

% uname -a
FreeBSD freebsd-10-x64.localdomain 10.0-STABLE FreeBSD 10.0-STABLE #0 7905925(stable/10): Wed Jun 11 14:10:17 PDT 2014     root@freebsd-x86.localdomain:/usr/obj/usr/src/sys/GENERIC-WITHOUT-WITNESS  amd64
% cd tools/regression/lib/libc/regex
% make
cd ../../../../../lib/libc/regex/grot; make && echo "PASS regex"
sh mkh  -i _REGEX_H_ ../regex2.h ../reg*.c >regex.tmp
cmp -s regex.tmp regex.h 2>/dev/null || cp regex.tmp regex.h
rm -f regex.tmp
sh mkh  -p /usr/src/lib/libc/regex/grot/../regcomp.c >regcomp.ih
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c /usr/src/lib/libc/regex/grot/../regcomp.c
In file included from /usr/src/lib/libc/regex/grot/../regcomp.c:58:
In file included from /usr/src/lib/libc/regex/grot/../../locale/collate.h:41:
/usr/src/lib/libc/regex/grot/../../locale/xlocale_private.h:147:18: warning: passing 'long *' to parameter of type 'volatile u_long *' (aka 'volatile unsigned long *') converts between pointers to integer types with different sign [-Wpointer-sign]
        atomic_add_long(&(obj->retain_count), 1);
                        ^~~~~~~~~~~~~~~~~~~~
/usr/include/machine/atomic.h:315:1: note: passing argument to parameter 'p' here
ATOMIC_ASM(add,      long,  "addq %1,%0",  "ir",  v);
^
/usr/include/machine/atomic.h:112:43: note: expanded from macro 'ATOMIC_ASM'
atomic_##NAME##_##TYPE(volatile u_##TYPE *p, u_##TYPE v)\
                                          ^
In file included from /usr/src/lib/libc/regex/grot/../regcomp.c:58:
In file included from /usr/src/lib/libc/regex/grot/../../locale/collate.h:41:
/usr/src/lib/libc/regex/grot/../../locale/xlocale_private.h:158:36: warning: passing 'long *' to parameter of type 'volatile u_long *' (aka 'volatile unsigned long *') converts between pointers to integer types with different sign [-Wpointer-sign]
        long count = atomic_fetchadd_long(&(obj->retain_count), -1) - 1;
                                          ^~~~~~~~~~~~~~~~~~~~
/usr/include/machine/atomic.h:197:39: note: passing argument to parameter 'p' here
atomic_fetchadd_long(volatile u_long *p, u_long v)
                                      ^
2 warnings generated.
sh mkh  -p /usr/src/lib/libc/regex/grot/../engine.c >engine.ih
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c /usr/src/lib/libc/regex/grot/../regexec.c
sh mkh  -p /usr/src/lib/libc/regex/grot/../regerror.c >regerror.ih
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c /usr/src/lib/libc/regex/grot/../regerror.c
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c /usr/src/lib/libc/regex/grot/../regfree.c
sh mkh  -p debug.c >debug.ih
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c debug.c
debug.c:189:27: warning: format specifies type 'int' but the argument has type 'unsigned long' [-Wformat]
                        fprintf(d, "!%d(%d)!", OP(*s), opnd);
                                     ~~        ^~~~~~
                                     %lu
/usr/src/lib/libc/regex/grot/../regex2.h:80:15: note: expanded from macro 'OP'
#define OP(n)   ((n)&OPRMASK)
                ^~~~~~~~~~~~~
debug.c:189:35: warning: format specifies type 'int' but the argument has type 'sop' (aka 'unsigned long') [-Wformat]
                        fprintf(d, "!%d(%d)!", OP(*s), opnd);
                                        ~~             ^~~~
                                        %lu
2 warnings generated.
sh mkh  -p main.c >main.ih
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c main.c
main.c:29:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main(argc, argv)
^~~~
main.c:47:14: warning: implicit declaration of function 'getopt' is invalid in C99 [-Wimplicit-function-declaration]
        while ((c = getopt(argc, argv, "c:e:S:E:x")) != -1)
                    ^
main.c:56:25: warning: implicit declaration of function 'atoi' is invalid in C99 [-Wimplicit-function-declaration]
                        startoff = (regoff_t)atoi(optarg);
                                             ^
main.c:72:3: warning: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))'
                exit(2);
                ^
main.c:72:3: note: please include the header <stdlib.h> or explicitly provide a declaration for 'exit'
main.c:84:17: warning: format specifies type 'int' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
                        eprint(err), len, sizeof(erbuf), erbuf);
                                     ^~~
main.c:84:22: warning: format specifies type 'int' but the argument has type 'unsigned long' [-Wformat]
                        eprint(err), len, sizeof(erbuf), erbuf);
                                          ^~~~~~~~~~~~~
main.c:102:17: warning: format specifies type 'int' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
                        eprint(err), len, sizeof(erbuf), erbuf);
                                     ^~~
main.c:102:22: warning: format specifies type 'int' but the argument has type 'unsigned long' [-Wformat]
                        eprint(err), len, sizeof(erbuf), erbuf);
                                          ^~~~~~~~~~~~~
main.c:109:22: warning: field precision should have type 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat]
                                printf("match `%.*s'\n", len,
                                               ~~^~      ~~~
main.c:238:6: warning: format specifies type 'int' but the argument has type 'unsigned long' [-Wformat]
                                        sizeof(erbuf), erbuf);
                                        ^~~~~~~~~~~~~
main.c:269:6: warning: format specifies type 'int' but the argument has type 'unsigned long' [-Wformat]
                                        sizeof(erbuf), erbuf);
                                        ^~~~~~~~~~~~~
11 warnings generated.
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include -c split.c
cc -O2 -pipe   -DPOSIX_MISTAKE -DREDEBUG  -I/usr/src/lib/libc/regex/grot/.. -I/usr/src/lib/libc/regex/grot/../../locale -I/usr/src/lib/libc/regex/grot/../../../../include  regcomp.o regexec.o regerror.o regfree.o debug.o main.o split.o -o re
regcomp.o: In function `p_bracket':
/usr/src/lib/libc/regex/grot/../regcomp.c:(.text+0x3752): undefined reference to `__has_thread_locale'
/usr/src/lib/libc/regex/grot/../regcomp.c:(.text+0x3758): undefined reference to `__xlocale_global_locale'
/usr/src/lib/libc/regex/grot/../regcomp.c:(.text+0x376a): undefined reference to `__thread_locale'
/usr/src/lib/libc/regex/grot/../regcomp.c:(.text+0x3779): undefined reference to `__xlocale_global_locale'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Stop.
make[1]: stopped in /usr/src/lib/libc/regex/grot
*** Error code 1

Stop.
make: stopped in /usr/src/tools/regression/lib/libc/regex
Comment 1 David Chisnall freebsd_committer 2014-06-25 07:48:55 UTC
It looks like it's including some private libc headers, but relying on private libc symbols.  Is it meant to run without linking to libc, or to test internal libc functionality?  If it's the former, it can probably be fixed by just adding the relevant locale files to the build.
Comment 2 Enji Cooper freebsd_committer 2014-06-25 23:13:34 UTC
(In reply to David Chisnall from comment #1)
> It looks like it's including some private libc headers, but relying on
> private libc symbols.  Is it meant to run without linking to libc, or to
> test internal libc functionality?  If it's the former, it can probably be
> fixed by just adding the relevant locale files to the build.

I'm not entirely sure to be honest. I would think based on the sourcecode and test inputs that it's largely blackbox testcases, but like you suggest it seems to be accessing whitebox/graybox features/source.

I have some testcases from NetBSD which supplement this. I'll check and see whether or not the NetBSD testcases are a super set of the FreeBSD regex/grot testcases. If they are, then the code doesn't need to be fixed, and lib/libc/regex/grot should probably be deorbited from the tree as it's bitrotted code.
Comment 3 Kurt Jaeger freebsd_committer 2015-11-02 10:25:24 UTC
See

https://lists.freebsd.org/pipermail/freebsd-current/2015-October/057781.html

for further discussions on that topic.
Comment 4 commit-hook freebsd_committer 2015-12-13 06:34:47 UTC
A commit references this bug:

Author: ngie
Date: Sun Dec 13 06:33:52 UTC 2015
New revision: 292153
URL: https://svnweb.freebsd.org/changeset/base/292153

Log:
  Add -static to CFLAGS to unbreak the tests by using a libc.a with
  the xlocale private symbols exposed which aren't exposed publicly
  via the DSO

  PR: 191354
  MFC after: 1 week
  Sponsored by: EMC / Isilon Storage Division

Changes:
  head/lib/libc/regex/grot/Makefile
Comment 5 commit-hook freebsd_committer 2015-12-20 00:41:31 UTC
A commit references this bug:

Author: ngie
Date: Sun Dec 20 00:40:34 UTC 2015
New revision: 292481
URL: https://svnweb.freebsd.org/changeset/base/292481

Log:
  MFC r292153:

  Add -static to CFLAGS to unbreak the tests by using a libc.a with
  the xlocale private symbols exposed which aren't exposed publicly
  via the DSO

  PR: 191354
  Sponsored by: EMC / Isilon Storage Division

Changes:
_U  stable/10/
  stable/10/lib/libc/regex/grot/Makefile
Comment 6 commit-hook freebsd_committer 2015-12-20 00:46:33 UTC
A commit references this bug:

Author: ngie
Date: Sun Dec 20 00:45:35 UTC 2015
New revision: 292482
URL: https://svnweb.freebsd.org/changeset/base/292482

Log:
  MFstable/10 r292481:

  MFC r292153:

  Add -static to CFLAGS to unbreak the tests by using a libc.a with
  the xlocale private symbols exposed which aren't exposed publicly
  via the DSO

  PR: 191354
  Sponsored by: EMC / Isilon Storage Division

Changes:
_U  stable/9/
_U  stable/9/lib/
_U  stable/9/lib/libc/
  stable/9/lib/libc/regex/grot/Makefile