Bug 216770 - lang/julia: fails to build with clang 4.0
Summary: lang/julia: fails to build with clang 4.0
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Dimitry Andric
Keywords: needs-patch
Depends on:
Blocks: 216008
  Show dependency treegraph
Reported: 2017-02-04 04:18 UTC by Jan Beich
Modified: 2017-03-09 02:18 UTC (History)
5 users (show)

See Also:
iblis: maintainer-feedback+

workaround (317 bytes, patch)
2017-02-09 07:26 UTC, Jan Beich
no flags Details | Diff
workaround (x86_64 asm diff) (2.32 KB, patch)
2017-02-09 07:28 UTC, Jan Beich
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer 2017-02-04 04:18:59 UTC
LoadError("sysimg.jl",309,LoadError("irrationals.jl",112,AssertionError("Float64(φ) == Float64(big(φ))")))
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
gmake[2]: *** [Makefile:229: /wrkdirs/usr/ports/lang/julia/work/julia-0.5.0/usr/lib/julia/sys.o] Error 1

Comment 1 Jan Beich freebsd_committer 2017-02-04 05:14:33 UTC
devel/llvm40 or CC=clang40 is unaffected, so you may need full environment to reproduce:

  $ poudriere jail -cj clang40 -v projects/clang400-import -m svn
  $ poudriere bulk -Ctj clang40 lang/julia
Comment 2 Iblis Lin 2017-02-05 08:38:10 UTC
(In reply to Jan Beich (mail not working) from comment #1)

reported to upstream: https://github.com/JuliaLang/julia/issues/20454
Comment 3 Iblis Lin 2017-02-08 07:20:28 UTC
(In reply to Jan Beich (mail not working) from comment #1)

we got weird behavior of `strtod`:
Quote from https://github.com/JuliaLang/julia/issues/20454#issuecomment-278240419

julia@clang40-dev:~ % cat test.c
#include <stdio.h>      /* printf, NULL */
#include <stdlib.h>     /* strtod */

int main ()
  char str[] = "1.0633823966279327e37";
  char* end;
  double d;
  d = strtod (str, &end);
  printf("%f\n", d);
  return 0;
julia@clang40-dev:~ % cc test.c
julia@clang40-dev:~ % ./a.out 

"1.0633823966279327e37" and
 1 0633823966279321080272352895186239488
Comment 4 Jan Beich freebsd_committer 2017-02-08 10:56:38 UTC
-O2 optimizes out (k > 0) check in ratio() from contrib/gdtoa/smisc.c. Dimitry, can you confirm?
Comment 5 Dimitry Andric freebsd_committer 2017-02-08 20:34:14 UTC
(In reply to Jan Beich (mail not working) from comment #4)
> -O2 optimizes out (k > 0) check in ratio() from contrib/gdtoa/smisc.c.
> Dimitry, can you confirm?

I don't see it being optimized out. On i386, the assembly becomes:

        .globl  __ratio_D2A
        .p2align        4, 0x90
        .type   __ratio_D2A,@function
__ratio_D2A:                            # @__ratio_D2A
# BB#0:                                 # %entry
        pushl   %ebp
        movl    %esp, %ebp
        pushl   %edi
        pushl   %esi
        andl    $-8, %esp
        subl    $48, %esp
        movl    12(%ebp), %esi
        movl    8(%ebp), %edi
        movl    __stack_chk_guard, %eax
        movl    %eax, 44(%esp)
        leal    20(%esp), %eax
        movl    %eax, 4(%esp)
        movl    %edi, (%esp)
        calll   __b2d_D2A
        fstl    8(%esp)                 # 8-byte Folded Spill
        fstpl   32(%esp)
        leal    16(%esp), %eax
        movl    %eax, 4(%esp)
        movl    %esi, (%esp)
        calll   __b2d_D2A
        fstl    24(%esp)
        movl    20(%esp), %eax
        subl    16(%esp), %eax
        movl    16(%edi), %ecx
        subl    16(%esi), %ecx
        shll    $5, %ecx
        addl    %eax, %ecx
        movl    %ecx, %edx
        sarl    $31, %edx
        leal    (%ecx,%edx), %eax
        xorl    %edx, %eax
        testl   %ecx, %ecx
        jg      .LBB1_1
[... this is the k > 0 comparison ...]
# BB#2:                                 # %entry
        leal    24(%esp), %ecx
        jmp     .LBB1_3
        leal    32(%esp), %ecx
.LBB1_3:                                # %entry
        shll    $20, %eax
        orl     $4, %ecx
        addl    %eax, (%ecx)
        fldl    8(%esp)                 # 8-byte Folded Reload
        fdivp   %st(1)
        movl    __stack_chk_guard, %eax
        cmpl    44(%esp), %eax
        jne     .LBB1_5
# BB#4:                                 # %entry
        leal    -8(%ebp), %esp
        popl    %esi
        popl    %edi
        popl    %ebp
.LBB1_5:                                # %entry
        fstp    %st(0)
        calll   __stack_chk_fail
        .size   __ratio_D2A, .Lfunc_end1-__ratio_D2A

On amd64, it doesn't use jmps, but conditional movs:

        .globl  __ratio_D2A
        .p2align        4, 0x90
        .type   __ratio_D2A,@function
__ratio_D2A:                            # @__ratio_D2A
# BB#0:                                 # %entry
        pushq   %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset %rbp, -16
        movq    %rsp, %rbp
        .cfi_def_cfa_register %rbp
        pushq   %r14
        pushq   %rbx
        subq    $48, %rsp
        .cfi_offset %rbx, -32
        .cfi_offset %r14, -24
        movq    %rsi, %r14
        movq    %rdi, %rbx
        movq    __stack_chk_guard(%rip), %rax
        movq    %rax, -24(%rbp)
        leaq    -44(%rbp), %rsi
        callq   __b2d_D2A
        movsd   %xmm0, -56(%rbp)        # 8-byte Spill
        movsd   %xmm0, -32(%rbp)
        leaq    -48(%rbp), %rsi
        movq    %r14, %rdi
        callq   __b2d_D2A
        movsd   %xmm0, -40(%rbp)
        movl    -44(%rbp), %eax
        subl    -48(%rbp), %eax
        movl    20(%rbx), %ecx
        subl    20(%r14), %ecx
        shll    $5, %ecx
        addl    %eax, %ecx
        movl    %ecx, %eax
        negl    %eax
        cmovll  %ecx, %eax
        testl   %ecx, %ecx
[... here the value of k is tested ...]
        leaq    -32(%rbp), %rcx
        leaq    -40(%rbp), %rdx
        cmovgq  %rcx, %rdx
[... and here is the conditional move-if-greater ...]
        shll    $20, %eax
        orq     $4, %rdx
        addl    %eax, (%rdx)
        movq    __stack_chk_guard(%rip), %rax
        cmpq    -24(%rbp), %rax
        jne     .LBB1_2
# BB#1:                                 # %entry
        movsd   -56(%rbp), %xmm1        # 8-byte Reload
                                        # xmm1 = mem[0],zero
        divsd   %xmm0, %xmm1
        movapd  %xmm1, %xmm0
        addq    $48, %rsp
        popq    %rbx
        popq    %r14
        popq    %rbp
.LBB1_2:                                # %entry
        callq   __stack_chk_fail
        .size   __ratio_D2A, .Lfunc_end1-__ratio_D2A

That said, I *do* see a difference in the values produced by that strtod test, e.g. libc compiled with 3.9.1 or 4.0.0 gives:

3.9.1: 10633823966279326983230456482242756608.000000
4.0.0: 10633823966279321080272352895186239488.000000

This is consistent on i386 and amd64.  Certainly needs a bit more investigation.
Comment 6 Jan Beich freebsd_committer 2017-02-09 07:26:23 UTC
Created attachment 179784 [details]
Comment 7 Jan Beich freebsd_committer 2017-02-09 07:28:21 UTC
Created attachment 179785 [details]
workaround (x86_64 asm diff)

Of course, the diff is empty with Clang 3.9.
Comment 8 tavert 2017-02-09 08:55:42 UTC
Is that working around a compiler bug in clang 4.0, or undefined behavior in the source where other compiler versions behaved differently?
Comment 9 Dimitry Andric freebsd_committer 2017-02-09 15:36:51 UTC
(In reply to tavert from comment #8)
> Is that working around a compiler bug in clang 4.0, or undefined behavior in
> the source where other compiler versions behaved differently?

It's not entirely clear yet.  It seems that adding that line, or putting some printf's in the ratio() function "fixes" it, either due to some sort of undefined behavior or a compiler bug.  So as a quick stopgap it would probably be OK.

That said, I did some bisecting of upstream llvm/clang, and this revision causes the trouble: http://llvm.org/viewvc/llvm-project?view=revision&revision=280351

r280351 | jamesm | 2016-09-01 12:44:35 +0200 (Thu, 01 Sep 2016) | 22 lines

[SimplifyCFG] Change the algorithm in SinkThenElseCodeToEnd

It shuffles around some code in smisc.c's ratio() function, apparently with interesting effects.  I still need to figure out some way of transforming this into a test case for upstream, or if it is actually caused by undefined behavior, some workaround that can be applied permanently.
Comment 10 Dimitry Andric freebsd_committer 2017-02-10 14:36:24 UTC
Submitted upstream bug at https://llvm.org/bugs/show_bug.cgi?id=31928 .
Comment 11 commit-hook freebsd_committer 2017-02-13 20:14:24 UTC
A commit references this bug:

Author: dim
Date: Mon Feb 13 20:13:29 UTC 2017
New revision: 313706
URL: https://svnweb.freebsd.org/changeset/base/313706

  For now, compile all of contrib/gdtoa without requiring strict aliasing,
  since it is using type punning of union members, and clang does not yet
  support gcc's extensions which allow this (refer to
  for more information).

  This should fix strtod(3) return values for the lang/julia port, so it
  does not fail on an assertion during its build.

  PR:		216770