Bug 255910 - lang/ruby26 lang/ruby27: Fix clang 12 -Wcompound-token-split-by-macro warning in ruby.h
Summary: lang/ruby26 lang/ruby27: Fix clang 12 -Wcompound-token-split-by-macro warning...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-ruby (Nobody)
URL:
Keywords:
Depends on:
Blocks: 255570
  Show dependency treegraph
 
Reported: 2021-05-15 19:20 UTC by Dimitry Andric
Modified: 2021-05-29 14:15 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (ruby)


Attachments
Fix -Wcompound-token-split-by-macro warning in ruby26's ruby.h (1.33 KB, patch)
2021-05-15 19:20 UTC, Dimitry Andric
no flags Details | Diff
Fix -Wcompound-token-split-by-macro warning in ruby27's ruby.h (1.33 KB, patch)
2021-05-15 19:21 UTC, Dimitry Andric
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dimitry Andric freebsd_committer 2021-05-15 19:20:31 UTC
Created attachment 224976 [details]
Fix -Wcompound-token-split-by-macro warning in ruby26's ruby.h

During an exp-run for llvm 12 (see bug 255570), it turned out that several ruby gem extensions do not build with clang 12.0.0, for example devel/rubygem-thrift [1]:

compiling binary_protocol_accelerated.c
binary_protocol_accelerated.c:404:68: error: '(' and '{' tokens introducing statement expression appear in different macro expansion contexts [-Werror,-Wcompound-token-split-by-macro]
  VALUE thrift_binary_protocol_class = rb_const_get(thrift_module, rb_intern("BinaryProtocol"));
                                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/ruby-2.7/ruby/ruby.h:1847:23: note: expanded from macro 'rb_intern'
        __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                      ^
binary_protocol_accelerated.c:404:68: note: '{' token is here
  VALUE thrift_binary_protocol_class = rb_const_get(thrift_module, rb_intern("BinaryProtocol"));
                                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/ruby-2.7/ruby/ruby.h:1847:24: note: expanded from macro 'rb_intern'
        __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/ruby-2.7/ruby/ruby.h:1832:5: note: expanded from macro 'RUBY_CONST_ID_CACHE'
    {                                                   \
    ^

and similarly www/unit-ruby [2]:

src/ruby/nxt_ruby.c:242:21: error: '(' and '{' tokens introducing statement expression appear in different macro expansion contexts [-Werror,-Wcompound-token-split-by-macro]
    nxt_ruby_call = rb_intern("call");
                    ^~~~~~~~~~~~~~~~~
/usr/local/include/ruby-2.7/ruby/ruby.h:1847:23: note: expanded from macro 'rb_intern'
        __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                      ^
src/ruby/nxt_ruby.c:242:21: note: '{' token is here
    nxt_ruby_call = rb_intern("call");
                    ^~~~~~~~~~~~~~~~~
/usr/local/include/ruby-2.7/ruby/ruby.h:1847:24: note: expanded from macro 'rb_intern'
        __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/ruby-2.7/ruby/ruby.h:1832:5: note: expanded from macro 'RUBY_CONST_ID_CACHE'
    {                                                   \
    ^

The gem extensions appear to purposefully compile using -Werror, and this new -Wcompound-token-split-by-macro is enabled by default in clang 12 and later (see [3]). Of course we could go over all these extensions, and either remove -Werror or add -Wno-compound-token-split-by-macro, but that seems quite a lot of effort.

Instead, I have submitted a pull request to ruby's GitHub, which fixes this by modifying the main ruby.h header:
* Add RUBY_CONST_ID_CACHE_NB() (i.e. no-brace) which contains the code itself, without any braces
* RUBY_CONST_ID_CACHE() which uses RUBY_CONST_ID_CACHE_NB(), but puts braces around it (so no existing code using this macro breaks)
* Finally, change rb_intern() so the __extension__ directly creates a gcc statement expression, using the RUBY_CONST_ID_CACHE_NB() macro

I am attaching a patch that adds these changes to the lang/ruby26 and lang/ruby27 ports.

Also noted in the pull request is that ruby 3.0 does not need these changes, as they have refactored ruby.h, so that it avoids emitting this warning.

[1] http://package22.nyi.freebsd.org/data/mainamd64PR255570-default/2021-05-08_16h02m24s/logs/errors/rubygem-thrift-0.14.0,1.log
[2] http://package22.nyi.freebsd.org/data/mainamd64PR255570-default/2021-05-08_16h02m24s/logs/errors/unit-ruby2.7-1.23.0.log
[3] https://github.com/llvm/llvm-project/commit/0e00a95b4fad5e72851de012d3a0b2c2d01f8685
[4] https://github.com/ruby/ruby/pull/4504
Comment 1 Dimitry Andric freebsd_committer 2021-05-15 19:21:27 UTC
Created attachment 224977 [details]
Fix -Wcompound-token-split-by-macro warning in ruby27's ruby.h
Comment 2 Dimitry Andric freebsd_committer 2021-05-16 08:12:08 UTC
Unfortunately it looks like the ruby people only accept pull requests against their master branch, but the code there does not have the problem! Hence, upstream cannot (or will not) solve this issue. :)

So that means we will have to either carry these patches for the life of the ruby 2.6 and 2.7 branches, or disable -Wcompound-token-split-by-macro for all ruby gems, somehow. I would just apply this patch, since it seems unlikely to me that upstream will ever change this header again.
Comment 3 Dimitry Andric freebsd_committer 2021-05-16 10:11:20 UTC
(In reply to Dimitry Andric from comment #2)
Upstream reopened the pull request after reading it again, and asked me to also submit a ruby PR, which I added in https://bugs.ruby-lang.org/issues/17865.

Hopefully this will be committed by upstream at some point, an in the mean time we can use the patch in this PR.
Comment 4 commit-hook freebsd_committer 2021-05-29 14:09:40 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=6f95cc52457d9c074ce91cdbd652a782424e41ee

commit 6f95cc52457d9c074ce91cdbd652a782424e41ee
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2021-05-15 17:12:06 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2021-05-29 14:08:19 +0000

    lang/ruby{26,27}: work around clang 12 -Wcompound-token-split-by-macro warning

    During an exp-run for llvm 12 (see bug 255570), it turned out that
    several ruby gem extensions do not build with clang 12.0.0, for example
    devel/rubygem-thrift:

    compiling binary_protocol_accelerated.c
    binary_protocol_accelerated.c:404:68: error: '(' and '{' tokens introducing statement expression appear in different macro expansion contexts [-Werror,-Wcompound-token-split-by-macro]
      VALUE thrift_binary_protocol_class = rb_const_get(thrift_module, rb_intern("BinaryProtocol"));
                                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
    /usr/local/include/ruby-2.7/ruby/ruby.h:1847:23: note: expanded from macro 'rb_intern'
            __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                          ^
    binary_protocol_accelerated.c:404:68: note: '{' token is here
      VALUE thrift_binary_protocol_class = rb_const_get(thrift_module, rb_intern("BinaryProtocol"));
                                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
    /usr/local/include/ruby-2.7/ruby/ruby.h:1847:24: note: expanded from macro 'rb_intern'
            __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /usr/local/include/ruby-2.7/ruby/ruby.h:1832:5: note: expanded from macro 'RUBY_CONST_ID_CACHE'
        {                                                   \
        ^

    and similarly www/unit-ruby:

    src/ruby/nxt_ruby.c:242:21: error: '(' and '{' tokens introducing statement expression appear in different macro expansion contexts [-Werror,-Wcompound-token-split-by-macro]
        nxt_ruby_call = rb_intern("call");
                        ^~~~~~~~~~~~~~~~~
    /usr/local/include/ruby-2.7/ruby/ruby.h:1847:23: note: expanded from macro 'rb_intern'
            __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                          ^
    src/ruby/nxt_ruby.c:242:21: note: '{' token is here
        nxt_ruby_call = rb_intern("call");
                        ^~~~~~~~~~~~~~~~~
    /usr/local/include/ruby-2.7/ruby/ruby.h:1847:24: note: expanded from macro 'rb_intern'
            __extension__ (RUBY_CONST_ID_CACHE((ID), (str))) : \
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /usr/local/include/ruby-2.7/ruby/ruby.h:1832:5: note: expanded from macro 'RUBY_CONST_ID_CACHE'
        {                                                   \
        ^

    The gem extensions appear to purposefully compile using -Werror, and
    this new -Wcompound-token-split-by-macro is enabled by default in clang
    12 and later. Of course we could go over all these extensions, and
    either remove -Werror or add -Wno-compound-token-split-by-macro, but
    that seems quite a lot of effort.

    Instead, I have submitted a pull request to ruby's GitHub, which fixes
    this by modifying the main ruby.h header:
    * Add RUBY_CONST_ID_CACHE_NB() (i.e. no-brace) which contains the code
      itself, without any braces
    * RUBY_CONST_ID_CACHE() which uses RUBY_CONST_ID_CACHE_NB(), but puts
      braces around it (so no existing code using this macro breaks)
    * Finally, change rb_intern() so the __extension__ directly creates a
      gcc statement expression, using the RUBY_CONST_ID_CACHE_NB() macro

    Patch this locally in our lang/ruby26 and lang/ruby27 ports for now,
    until upstream manages to get this in.

    Approved by:    maintainer timeout (2 weeks)
    PR:             255910
    MFH:            2021Q2

 lang/ruby26/files/patch-include_ruby_ruby.h (new) | 29 +++++++++++++++++++++++
 lang/ruby27/files/patch-include_ruby_ruby.h (new) | 29 +++++++++++++++++++++++
 2 files changed, 58 insertions(+)