Bug 255290 - _POSIX_C_SOURCE=200809 hides static_assert
Summary: _POSIX_C_SOURCE=200809 hides static_assert
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: standards (show other bugs)
Version: Unspecified
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-standards (Nobody)
URL: https://github.com/gbdev/rgbds/issues...
Keywords:
: 260251 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-04-21 06:22 UTC by Tobias Kortkamp
Modified: 2021-12-06 15:57 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias Kortkamp freebsd_committer freebsd_triage 2021-04-21 06:22:38 UTC
$ cat test.c <<EOF
#include <assert.h>
int main() { static_assert(1 == 1, ""); }
EOF
$ cc -D_POSIX_C_SOURCE=200809L -std=gnu11 test.c
test.c:2:14: warning: implicit declaration of function 'static_assert' is invalid in C99 [-Wimplicit-function-declaration]
int main() { static_assert(1 == 1, ""); }
             ^
1 warning generated.
ld: error: undefined symbol: static_assert
>>> referenced by test.c
>>>               /tmp/test-8e234d.o:(main)
cc: error: linker command failed with exit code 1 (use -v to see invocation)

This affects devel/rgbds where I had to remove _POSIX_C_SOURCE.

In assert.h we have:
#if __ISO_C_VISIBLE >= 2011 && !defined(__cplusplus)
#define static_assert   _Static_assert
#endif

In sys/cdefs.h we have:
#if _POSIX_C_SOURCE >= 200809
#define __POSIX_VISIBLE         200809
#define __ISO_C_VISIBLE         1999

so static_assert is hidden when _POSIX_C_SOURCE=200809L because
__ISO_C_VISIBLE is clamped to 1999.  I've reported this upstream
but we have found no evidence that POSIX 2008 mandates C99 and
should hide static_assert.

On Linux with musl and glibc and OpenBSD this works fine.

OpenBSD uses __STDC_VERSION__ instead of __ISO_C_VISIBLE
in assert.h which might be the more correct choice.
Comment 1 Warner Losh freebsd_committer freebsd_triage 2021-04-21 14:09:45 UTC
(In reply to Tobias Kortkamp from comment #0)
> I've reported this upstream
> but we have found no evidence that POSIX 2008 mandates C99 and
> should hide static_assert.

If POSIX 2008 and/or C99 do not mandate static_assert, it must be hidden for them. A general rule is that system headers cannot pollute name spaces of the user when directed to do so by things like POSIX_C_SOURCE.

static_assert was added, I believe, in C11 and C++11. As such, it should be hidden from the 2008 posix environment.

I've not checked to see if there's a newer value of POSIX_C_SOURCE that should define a higher value of either __POSIX_VISIBLE or __ISO_C_VISIBLE, but since you're using 200809, that's not relevant.
Comment 2 Warner Losh freebsd_committer freebsd_triage 2021-04-21 14:10:54 UTC
(In reply to Warner Losh from comment #1)
I should add that in C99, static_assert was in the user namespace. I meant to include that in my last reply.
Comment 3 Warner Losh freebsd_committer freebsd_triage 2021-04-21 14:13:41 UTC
Further, the 2004 edition of the Open Group standard has the following in its Base Specification Issue 6, section 2.2.1. I imagine that similar text is in all the standards:

---- Quoted from https://pubs.opengroup.org/onlinepubs/007904975/functions/xsh_chap02_02.html by fair use:

The _POSIX_C_SOURCE Feature Test Macro
A POSIX-conforming application should ensure that the feature test macro _POSIX_C_SOURCE is defined before inclusion of any header.

When an application includes a header described by IEEE Std 1003.1-2001, and when this feature test macro is defined to have the value 200112L:

All symbols required by IEEE Std 1003.1-2001 to appear when the header is included shall be made visible.

Symbols that are explicitly permitted, but not required, by IEEE Std 1003.1-2001 to appear in that header (including those in reserved name spaces) may be made visible.

Additional symbols not required or explicitly permitted by IEEE Std 1003.1-2001 to be in that header shall not be made visible, except when enabled by another feature test macro.

Identifiers in IEEE Std 1003.1-2001 may only be undefined using the #undef directive as described in Use and Implementation of Functions or The Name Space. These #undef directives shall follow all #include directives of any header in IEEE Std 1003.1-2001.
-------
Comment 4 Tobias Kortkamp freebsd_committer freebsd_triage 2021-04-21 14:33:22 UTC
> static_assert was added, I believe, in C11 and C++11. As such, it should
> be hidden from the 2008 posix environment.

This was my initial reaction too when I tried to defend FreeBSD's behavior.
But what are you basing this on?

> If POSIX 2008 and/or C99 do not mandate static_assert, it must be hidden for
> them. A general rule is that system headers cannot pollute name spaces of the
> user when directed to do so by things like POSIX_C_SOURCE.

That would be fine if we were asking for C99 but we are asking for
C11 with POSIX 2008 specifically, so should static_assert then not
be unhidden since it is allowed by C11?

> Additional symbols not required or explicitly permitted by IEEE Std 1003.1-2001
> to be in that header shall not be made visible, except when enabled by another
> feature test macro.

Would you interpret -std={c,gnu}11 as "another feature test macro" (or
something that sets some, specifically __STDC_VERSION__=201112)?
Comment 5 Garrett Wollman freebsd_committer freebsd_triage 2021-04-21 14:45:20 UTC
There is no such thing as "C11 with POSIX 2008 specifically". POSIX.1-2008 explicitly specifies C99, and *only* C99.  So you're asking for undefined behavior here.
Comment 6 Warner Losh freebsd_committer freebsd_triage 2021-04-21 14:46:44 UTC
The POSIX-2018 version (issue 7) also defines that it's based on C99:
"ISO/IEC 9899:1999, Programming Languages - C, including ISO/IEC 9899:1999/Cor.1:2001(E), ISO/IEC 9899:1999/Cor.2:2004(E), and ISO/IEC 9899:1999/Cor.3" from section 1.1. It goes on to say:

Emphasis has been placed on standardizing existing practice for existing users, with changes and additions limited to correcting deficiencies in the following areas:
...
o Alignment with the ISO/IEC 9899:1999 standard, including ISO/IEC 9899:1999/Cor.2:2004(E)
...

I could find no newer editions.
Comment 7 Warner Losh freebsd_committer freebsd_triage 2021-04-21 14:48:39 UTC
(In reply to Tobias Kortkamp from comment #4)
>Would you interpret -std={c,gnu}11 as "another feature test macro" (or
>something that sets some, specifically __STDC_VERSION__=201112)?

I would interpret that as requesting undefined behavior. POSIX is C99 and C99 only. You requested POSIX, and then requested something else also. That behavior is not defined. Pick one or the other, but you cannot have both as there's no well-defined way to provide both.
Comment 8 Garrett Wollman freebsd_committer freebsd_triage 2021-04-21 14:51:02 UTC
(In reply to Warner Losh from comment #6)
The next POSIX revision is currently in preparation and will be aligned with C17.
Comment 9 Warner Losh freebsd_committer freebsd_triage 2021-04-21 15:10:33 UTC
(In reply to Garrett Wollman from comment #8)
> The next POSIX revision is currently in preparation and will be aligned with C17.

Excellent.
Comment 10 Eldred Habert 2021-04-21 19:38:06 UTC
(In reply to Warner Losh from comment #7)

Hey, upstream here. Thank you very much Tobik both for reporting the issue to us and here ^^

I interpret the following:

> Additional symbols not required or explicitly permitted by IEEE Std 1003.1-2001 to be in that header shall not be made visible, except when enabled by another feature test macro.

... as permitting the use of C11 functionality, since our codebase specifies the `-std=gnu11` flag (which should be c11, but we had to switch to fix the build with DJGPP), itself defining __STDC_VERSION__ to request `static_assert`.

That point has been made already, and I disagree with the following reply to it:

> I would interpret that as requesting undefined behavior. POSIX is C99 and C99 only. You requested POSIX, and then requested something else also. That behavior is not defined. Pick one or the other, but you cannot have both as there's no well-defined way to provide both.

I don't read the snippet copied in Comment #6 as "POSIX requires C99 functionality, neither more nor less", but rather "POSIX does not aim to be https://xkcd.com/927/, thus definitions were aligned on C99 whenever possible". As far as I'm aware, C11 does not make any modifications to the standard that are incompatible with POSIX 2008, and thus there are no conflicts, and hence, no undefined behavior.

I back this up by POSIX allowing other feature test macros to define symbols outside of POSIX's scope; if defining any other such macros would result in UB, why would the standard explicitly allow it?
Comment 11 Warner Losh freebsd_committer freebsd_triage 2021-04-21 20:06:07 UTC
> As far as I'm aware, C11 does not make any modifications to the standard that are incompatible with POSIX 2008, and thus there are no conflicts, and hence, no undefined behavior.

C11 adds things. Those additions intrude into the namespace that POSIX allows the user to use. That's where the conflict arises.

The following should be a strictly conforming, even if compiled with C11 compiler, as there's no exception in POSIX for newer versions of C introducing new things. The requested change would make it not be.

#define __POSIX_C_SOURCE 200809

#include <assert.h>
#include <stdio.h>

void static_assert(int a, int b, int c)
{ printf("A is %d, b is %d and c is %d\n", a, b, c);}
int main(int argc, char *argv[]) { static_assert(1, 2, 3); exit(0); }

Further, __STDC_VERSION__ is not a feature test macro that the user can define. It is a pre-defined macro defined by the compiler, and can't be redefined by the programmer.
Comment 12 Olivier Certner freebsd_committer freebsd_triage 2021-04-21 21:49:19 UTC
(In reply to Garrett Wollman from comment #5)
(In reply to Warner Losh from comments #1, #7 and #11)

I think that what others are suggesting for static_assert is perfectly admissible, and that you are over-interpreting the standard.

To be precise, I'm quoting text from IEEE Std 1003.1-2017, which is the Open Group Base Specifications Issue 7, in the 2018 edition (there is no such thing as POSIX-2018, rather, we are talking about POSIX.1-2017), document also referenced as SUSv4 2018 edition, the latest one downloadable from the Open Group site.

Some quotes from Base definitions (XBD), chapter 2: "Conformance"

> 2.1 Implementation Conformance
> 2.1.1 Requirements
> 2.1.1.4
> The system may provide non-standard extensions. These are features not
> required by POSIX.1-2017 and may include, but are not limited to:
>  * Additional functions
>  * Additional headers
>  * Additional symbols in standard headers
> (snip)
> Non-standard extensions of the utilities, functions, or facilities
> specified in POSIX.1-2017 should be identified as such in the system
> documentation. Non-standard extensions, when used, may change the
> behavior of utilities, functions, or facilities defined by POSIX.1-2017.
> The conformance document shall define an environment in which an
> application can be run with the behavior specified by POSIX.1-2017.
> In no case shall such an environment require modification of a Strictly
> Conforming POSIX Application (see Strictly Conforming POSIX Application).

From here, it seems clear that adding functionality in standard headers is allowed. Doing so is defining non-standard extensions. I concede that macros and keywords are not explicitly mentioned in the list above, but I think the spirit is that they are included as well.

It would be possible for FreeBSD to state that the environment supporting strictly conforming C applications is one where no other standard than C99, if any, is specified to 'cc', and that the application must define __POSIX_C_SOURCE, but must not modify any symbol indicating the ISO C standard used.

Conversely, requiring C11 at compile-time is an extension whose behavior can be specified by the system. And the minimal specification is to require POSIX behavior except for the differences introduced by C11.

> The following should be a strictly conforming, even if compiled
> with C11 compiler, as there's no exception in POSIX for newer versions
> of C introducing new things. The requested change would make it not be.

I don't think there is anything in the standard saying that a strictly conforming application can be compiled with a C11 compiler. And although defining __POSIX_C_SOURCE is required of strictly conforming applications, doing so alone does not force an application to be strictly conforming (again, nothing in the standard says or implies so, correct me if I'm wrong). On the contrary, as quoted above, the system environment in which a conforming application can be run with strict POSIX behavior has to be defined in the conformance document of the system (FreeBSD).

So this is not a matter of standards, this is a matter of policy.

And because:
1. Other widely-used systems are doing so,
2. And in accordance with the motto "Be liberal in what you accept",
I'll argue that the most reasonable path, from the application support point of view, would be for FreeBSD to follow suit.
Comment 13 Warner Losh freebsd_committer freebsd_triage 2021-04-21 22:34:49 UTC
This isn't a case of being liberal about what's accepted. It's a case of doing exactly what the standard specifies, and nothing more. There's no feature test macro defined for sources to say they want a C11 extensions in addition to the strict posix environment requested by __POSIX_C_SOURCE__ = 200809.

If there is a feature test macro that's well defined that includes this, that could be implemented, but __STDC_VERSION__ is not such a macro because it cannot be defined by the programmer, only the compiler.
Comment 14 Warner Losh freebsd_committer freebsd_triage 2021-04-21 23:20:43 UTC
_ISOC11_SOURCE appears to be a newer extension in glibc that could solve the issue about what to do when compiling in C11 mode. glibc choses to enable this by default, but it's unclear that's standard conformant. I'll see about adding this feature test macro to FreeBSD to specify you also want isoc11 features. That would be compatible with glibc and other systems, as well as being something to fill in the gap before the open group updates posix to align it to a newer version of C.

This would also preserve purely standards conforming programs that just happen to be compiled with a c11 because the system default has changed, and be minimally intrusive a change for code ported from other systems.
Comment 15 Warner Losh freebsd_committer freebsd_triage 2021-04-21 23:31:28 UTC
https://reviews.freebsd.org/D29902
Comment 16 Olivier Certner freebsd_committer freebsd_triage 2021-04-22 07:58:27 UTC
(In reply to Warner Losh from comment #13)

> It's a case of doing exactly what the standard specifies, and nothing more.

The standard doesn't specify that. In particular, as said above, it never says that defining _POSIX_C_SOURCE is enough to obtain a strictly POSIX environment.

And more precisely, it says this, which directly answers our concern:

> 2.2 The Compilation Environment
> 2.2.1 POSIX.1 Symbols
> Certain symbols in this volume of POSIX.1-2017 are defined in
> headers (see XBD Headers). Some of those headers could also define
> symbols other than those defined by POSIX.1-2017, potentially
> conflicting with symbols used by the application. Also, POSIX.1-2017
> defines symbols that are not permitted by other standards to appear
> in those headers without some control on the visibility of those symbols.
> Symbols called "feature test macros" are used to control the visibility
> of symbols that might be included in a header. Implementations, future
> versions of this standard, and other standards may define additional
> feature test macros.
> (snip)
> The _POSIX_C_SOURCE Feature Test Macro
> (snip)
> When an application includes a header described by POSIX.1-2017,
> and when this feature test macro is defined to have the value 200809L:
> 1 All symbols required by POSIX.1-2017 to appear when the header is
> included shall be made visible.
> 2 Symbols that are explicitly permitted, but not required, by
> POSIX.1-2017 to appear in that header (including those in reserved name
> spaces) may be made visible.
> 3 Additional symbols not required or explicitly permitted by
> POSIX.1-2017 to be in that header shall not be made visible, except when
> enabled by another feature test macro.

So yes, it's possible to define additional macros under a feature test macro.

> There's no feature test macro defined for sources to say they want a C11
> extensions in addition to the strict posix environment requested by
> __POSIX_C_SOURCE__ = 200809.

As quoted above, an implementation is free to define any feature test macro they wish. So, compiling with C11 with appropriate flags on the command-line should automatically define some feature test macro, and then we'll be standard compliant.

> glibc choses to enable this by default, but it's unclear that's standard
> conformant.

It is standard conformant. The standard says that conformant programs should be built with c99, not cc (I could not find references to cc in the standard), so it seems clear (to me at least) that the default C standard supported by cc is irrelevant to this discussion.

> https://reviews.freebsd.org/D29902

Thanks. Commenting there.
Comment 17 commit-hook freebsd_committer freebsd_triage 2021-05-06 22:22:57 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=a709a4f0d4e0303eb40ea1b39b4bb35837800805

commit a709a4f0d4e0303eb40ea1b39b4bb35837800805
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2021-05-06 22:20:19 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2021-05-06 22:20:36 +0000

    headers: Implement _ISOC11_SOURCES macro when __POSIX_C_SOURCE defined

    When _ISOC11_SOURCES is defined for glibc at the same time
    __POSIX_C_SOURCE is defined, it extends the __POSIX_C_SOURCE definition
    by exaclty what C11 adds to the spec for each system header.  We follow
    both OpenBSD's and glibc's convention by also C11 or higher compliation
    mode is selected.

    The Open Group is working on issuing a new version of the POSIX standard
    that will realign the standard from C99 to a newer version of C. This
    commit is a stop-gap measure for greater compatibility until that
    environment has been standardized.

    Reviewed by:            brooks@, arichards@, Olivier Certne
                            (comments tweaked before commit)
    PR:                     255290
    Sponsored by:           Netflix
    Differential Revision:  https://reviews.freebsd.org/D29902

 sys/sys/cdefs.h | 11 +++++++++++
 1 file changed, 11 insertions(+)
Comment 18 Tobias Kortkamp freebsd_committer freebsd_triage 2021-09-01 19:18:18 UTC
Thanks.
Comment 19 Jan Beich freebsd_committer freebsd_triage 2021-12-06 11:38:29 UTC
*** Bug 260251 has been marked as a duplicate of this bug. ***
Comment 20 Jan Beich freebsd_committer freebsd_triage 2021-12-06 12:22:05 UTC
Do you plan to MFC (backport) this into /stable/13 (for 13.1) and /stable/12 (for 12.4) ? wlroots upstream (FreeBSD CI via builds.sr.ht) is curious how long they have to keep a workaround. Without any MFC it'd be until 2026-01-31 (or about 4 years) when /stable/13 reaches EOL.
Comment 21 commit-hook freebsd_committer freebsd_triage 2021-12-06 15:56:26 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=77353824700794eac987ae11e82d3ec5e4a99228

commit 77353824700794eac987ae11e82d3ec5e4a99228
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2021-05-06 22:20:19 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2021-12-06 15:54:11 +0000

    headers: Implement _ISOC11_SOURCES macro when __POSIX_C_SOURCE defined

    When _ISOC11_SOURCES is defined for glibc at the same time
    __POSIX_C_SOURCE is defined, it extends the __POSIX_C_SOURCE definition
    by exaclty what C11 adds to the spec for each system header.  We follow
    both OpenBSD's and glibc's convention by also C11 or higher compliation
    mode is selected.

    The Open Group is working on issuing a new version of the POSIX standard
    that will realign the standard from C99 to a newer version of C. This
    commit is a stop-gap measure for greater compatibility until that
    environment has been standardized.

    Reviewed by:            brooks@, arichards@, Olivier Certne
                            (comments tweaked before commit)
    PR:                     255290
    Sponsored by:           Netflix
    Differential Revision:  https://reviews.freebsd.org/D29902

    (cherry picked from commit a709a4f0d4e0303eb40ea1b39b4bb35837800805)

 sys/sys/cdefs.h | 11 +++++++++++
 1 file changed, 11 insertions(+)
Comment 22 commit-hook freebsd_committer freebsd_triage 2021-12-06 15:57:29 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3f82687cdf02983d8f3294df4d97b09cf211141b

commit 3f82687cdf02983d8f3294df4d97b09cf211141b
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2021-05-06 22:20:19 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2021-12-06 15:56:00 +0000

    headers: Implement _ISOC11_SOURCES macro when __POSIX_C_SOURCE defined

    When _ISOC11_SOURCES is defined for glibc at the same time
    __POSIX_C_SOURCE is defined, it extends the __POSIX_C_SOURCE definition
    by exaclty what C11 adds to the spec for each system header.  We follow
    both OpenBSD's and glibc's convention by also C11 or higher compliation
    mode is selected.

    The Open Group is working on issuing a new version of the POSIX standard
    that will realign the standard from C99 to a newer version of C. This
    commit is a stop-gap measure for greater compatibility until that
    environment has been standardized.

    Reviewed by:            brooks@, arichards@, Olivier Certne
                            (comments tweaked before commit)
    PR:                     255290
    Sponsored by:           Netflix
    Differential Revision:  https://reviews.freebsd.org/D29902

    (cherry picked from commit a709a4f0d4e0303eb40ea1b39b4bb35837800805)

 sys/sys/cdefs.h | 11 +++++++++++
 1 file changed, 11 insertions(+)