Bug 221127

Summary: math/R: cannot install packages from CRAN
Product: Ports & Packages Reporter: rsos+freebsd
Component: Individual Port(s)Assignee: Joseph Mingrone <jrm>
Status: Closed FIXED    
Severity: Affects Only Me CC: emaste, gerald, rhurlin, tijl
Priority: --- Flags: bugzilla: maintainer-feedback? (jrm)
Version: Latest   
Hardware: Any   
OS: Any   

Description rsos+freebsd 2017-07-31 16:38:07 UTC
Hello,

recently I discovered an issue by installing a new package from CRAN:
`install.packages("roxygen2")` results in a segfault:

```
...
trying URL 'https://cran.r-project.org/src/contrib/brew_1.0-6.tar.gz'

 *** caught segfault ***
address 0x7fffdfdfe000, cause 'invalid permissions'

Traceback:
 1: download.file(url, destfile, method, mode = "wb", ...)
 2: doTryCatch(return(expr), name, parentenv, handler)
 3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 4: tryCatchList(expr, classes, parentenv, handlers)
 5: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        msg <- conditionMessage(e)        sm <- strsplit(msg, "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && identical(getOption("show.error.messages"),         TRUE)) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
 6: try(download.file(url, destfile, method, mode = "wb", ...))
 7: download.packages(pkgs, destdir = tmpd, available = available,     contriburl = contriburl, method = method, type = "source",     quiet = quiet, ...)
 8: install.packages("roxygen2")
```

I tried to install various packages from CRAN. Segfault occurs after downloading a bunch of dependencies (there is not a certain dependency that throws this error). I tried the R package from the quarterly branch as well as the package from the latest branch without any luck. Does this affect more people? Are there any workarounds?

Best
Comment 1 Joseph Mingrone freebsd_committer freebsd_triage 2017-07-31 22:47:45 UTC
An upstream bug report has been submitted.

https://stat.ethz.ch/pipermail/r-devel/2017-August/074726.html
Comment 2 Joseph Mingrone freebsd_committer freebsd_triage 2017-07-31 22:52:33 UTC
CC some people that have shown recent interest in the math/R port.
Comment 3 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-01 02:01:50 UTC
I just installed "roxygen2" without a problem.

R-3.4.1_3 with RBLAS=yes FLANG=yes
Comment 4 Joseph Mingrone freebsd_committer freebsd_triage 2017-08-01 02:04:32 UTC
Could you confirm you're running 11.1-RELEASE or later?
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-01 02:07:53 UTC
Segfault on download through https might result from the mixup of base and port SSL libraries. You need to see if 'ldd /usr/local/bin/R' shows such mix.
Comment 6 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-01 02:08:23 UTC
running FreeBSD 11.1-STABLE r321380M
Comment 7 rsos+freebsd 2017-08-01 06:24:21 UTC
I'm running FreeBSD 11.1, too. I had no problems before upgrading to 11.1 but I haven't used R for one month. So it may as well possible that the problems occured with current quarterly branch.

For me, ldd shows nothing strange:
```
$ ldd /usr/local/lib/R/bin/exec/R 
/usr/local/lib/R/bin/exec/R:
	libR.so => /usr/local/lib/R/lib/libR.so (0x800a00000)
	libRblas.so => /usr/local/lib/R/lib/libRblas.so (0x800fe9000)
	libc.so.7 => /lib/libc.so.7 (0x801212000)
	libgfortran.so.3 => /usr/local/lib/gcc5/libgfortran.so.3 (0x8015ca000)
	libm.so.5 => /lib/libm.so.5 (0x8018ec000)
	libquadmath.so.0 => /usr/local/lib/gcc5/libquadmath.so.0 (0x801b17000)
	libintl.so.8 => /usr/local/lib/libintl.so.8 (0x801d56000)
	libreadline.so.7 => /usr/local/lib/libreadline.so.7 (0x801f60000)
	libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x8021b0000)
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x802428000)
	libbz2.so.4 => /usr/lib/libbz2.so.4 (0x802651000)
	libz.so.6 => /lib/libz.so.6 (0x802865000)
	librt.so.1 => /usr/lib/librt.so.1 (0x802a7e000)
	libicuuc.so.58 => /usr/local/lib/libicuuc.so.58 (0x802c83000)
	libicui18n.so.58 => /usr/local/lib/libicui18n.so.58 (0x803200000)
	libgcc_s.so.1 => /usr/local/lib/gcc5/libgcc_s.so.1 (0x8036ac000)
	libncursesw.so.8 => /lib/libncursesw.so.8 (0x8038c2000)
	libthr.so.3 => /lib/libthr.so.3 (0x803b20000)
	libicudata.so.58 => /usr/local/lib/libicudata.so.58 (0x803d48000)
	libc++.so.1 => /usr/lib/libc++.so.1 (0x803f49000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x804210000)

```

`/usr/local/bin/R` is only a shell script.
Comment 8 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-01 06:32:51 UTC
(In reply to rs.os from comment #7)

Add these lines to /etc/make.conf:
WITH_DEBUG=Y
DEBUG_FLAGS=-g
FLAGS=-g
CFLAGS=-g
CPPFLAGS=-g
STRIP=

Rebuild:
# portupgrade -f R

Run under debugger and find the line where it crashes.
Comment 9 Rainer Hurling freebsd_committer freebsd_triage 2017-08-01 08:15:32 UTC
(In reply to rs.os from comment #0)

This happend also on my recent HEAD amd64 boxes. I think, it has something to do with newer changes to libcurl (FreeBSD related)?

At least, if I change the download method for packages within R, all went fine:


update.packages(checkBuilt=TRUE, dependencies=TRUE, ask='graphics', method='wget')

This also works with install.packages(..., method='wget')
Comment 10 Joseph Mingrone freebsd_committer freebsd_triage 2017-08-01 17:14:04 UTC
The third call to

CURLcode ret = curl_easy_perform(hnd);

in in_do_curlGetHeaders() from src/modules/internet/libcurl.c triggers
the problem _unless_ R is built with flang instead of gfortran.

For FreeBSD 11.1 R users, there are two workarounds [1].

1) Build the FreeBSD R package with the FLANG option instead of
   GFORTRAN.  FLANG may be the default soon.

2) Add

   options(download.file.method="wget")

   to ~/.Rprofile with a newline after it.  Ensure ftp/wget is
   installed, because it is not pulled in by math/R.

[1] Both these workarounds prevent a predictable crash, but have only
been lightly tested.
Comment 11 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-01 17:25:30 UTC
(In reply to Joseph Mingrone from comment #10)

Then it might be crashing because of mixing of libgcc_s.so from gfortran and from the base.
Comment 12 Joseph Mingrone freebsd_committer freebsd_triage 2017-08-02 19:37:37 UTC
I see the same result with USE_GCC=yes and using gfortran.  A backtrace indicates the problem may be with lang/gcc5, so I'm copying gerald.

Reading symbols from /usr/local/lib/R/bin/exec/R...done.
[New LWP 100936]
[New LWP 100805]
Core was generated by `/usr/local/lib/R/bin/exec/R'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  x86_64_freebsd_fallback_frame_state (context=0x7fffdfffde20, context=0x7fffdfffde20, fs=0x7fffdfffdb70) at ./md-unwind-support.h:60
60        if (!(   *(unsigned int *)(context->ra)      == 0x247c8d48
[Current thread is 1 (LWP 100936)]
(gdb) l
55           <pc + 14>:  syscall
56
57           If we can't find this pattern, we're at the end of the stack.
58        */
59
60        if (!(   *(unsigned int *)(context->ra)      == 0x247c8d48
61              && *(unsigned int *)(context->ra +  4) == 0x48006a10
62              && *(unsigned int *)(context->ra +  8) == 0x01a1c0c7
63              && *(unsigned int *)(context->ra + 12) == 0x050f0000 ))
64          return _URC_END_OF_STACK;
(gdb) bt
#0  x86_64_freebsd_fallback_frame_state (context=0x7fffdfffde20, context=0x7fffdfffde20, fs=0x7fffdfffdb70) at ./md-unwind-support.h:60
#1  uw_frame_state_for (context=context@entry=0x7fffdfffde20, fs=fs@entry=0x7fffdfffdb70) at /usr/home/jrm/scm/freebsd/freebsd-ports.svn/lang/gcc5/work/gcc-5.4.0/libgcc/unwind-dw2.c:1249
#2  0x00000008034bbfdb in _Unwind_ForcedUnwind_Phase2 (exc=exc@entry=0x8051d5830, context=context@entry=0x7fffdfffde20)
    at /usr/home/jrm/scm/freebsd/freebsd-ports.svn/lang/gcc5/work/gcc-5.4.0/libgcc/unwind.inc:155
#3  0x00000008034bc314 in _Unwind_ForcedUnwind (exc=0x8051d5830, stop=0x8039367b0 <thread_unwind_stop>, stop_argument=<optimized out>)
    at /usr/home/jrm/scm/freebsd/freebsd-ports.svn/lang/gcc5/work/gcc-5.4.0/libgcc/unwind.inc:207
#4  0x0000000803936613 in _Unwind_ForcedUnwind (ex=<optimized out>, stop_func=0x7fffdfffd948, stop_arg=0x8051d5600) at /usr/src/lib/libthr/thread/thr_exit.c:106
#5  thread_unwind () at /usr/src/lib/libthr/thread/thr_exit.c:172
#6  _pthread_exit_mask (status=<optimized out>, mask=<optimized out>) at /usr/src/lib/libthr/thread/thr_exit.c:257
#7  0x000000080393642b in _pthread_exit (status=0x8051d5600) at /usr/src/lib/libthr/thread/thr_exit.c:206
#8  0x0000000803928bcd in thread_start (curthread=0x8051d5600) at /usr/src/lib/libthr/thread/thr_create.c:289
#9  0x00007fffdfdfe000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fffdfffe000
Comment 13 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-02 19:50:35 UTC
(In reply to Joseph Mingrone from comment #12)

> I see the same result with USE_GCC=yes and using gfortran.  A backtrace indicates the problem may be with lang/gcc5, so I'm copying gerald.

C++ from gcc and clang can't be mixed. They use different versions of libgcc_s.so in the same process, and they are conflict with each other. This is a known problem, and one of the reasons to have flang as a fortran compiler. As far as I know, there is no simple solution that doesn't involve removing gcc.
Comment 14 Joseph Mingrone freebsd_committer freebsd_triage 2017-08-02 19:59:15 UTC
It would be helpful to figure out what changed in 11.1 to determine if there is a solution.  This wasn't a problem prior to 11.1, even when clang was used to compile the base system.  We could make flang the default, but this only helps for amd64.
Comment 15 Joseph Mingrone freebsd_committer freebsd_triage 2017-08-02 20:38:49 UTC
By the way, I don't see any c++ code in R.  It's mostly mostly C and Fortran source under src/.
Comment 16 commit-hook freebsd_committer freebsd_triage 2017-08-02 21:28:37 UTC
A commit references this bug:

Author: jrm
Date: Wed Aug  2 21:27:43 UTC 2017
New revision: 447178
URL: https://svnweb.freebsd.org/changeset/ports/447178

Log:
  math/R: Work around bug 221127.

  PR:		221127
  Reported by:	rs.os@t-online.de

Changes:
  head/math/R/Makefile
  head/math/R/pkg-message
Comment 17 Yuri Victorovich freebsd_committer freebsd_triage 2017-08-02 21:32:47 UTC
This bug motivated me to add the check for bad library mix: https://reviews.freebsd.org/D11812

It will probably fail on some of the involved packages when gfortran is used.
Comment 18 Tijl Coosemans freebsd_committer freebsd_triage 2017-10-28 10:49:44 UTC
This looks like the same problem as https://lists.freebsd.org/pipermail/freebsd-current/2017-August/066855.html and https://lists.freebsd.org/pipermail/freebsd-current/2017-October/067254.html triggered by the curl port enabling THREADED_RESOLVER by default.  It's not because of mixing the wrong libraries.
Comment 19 commit-hook freebsd_committer freebsd_triage 2017-11-18 13:19:45 UTC
A commit references this bug:

Author: tijl
Date: Sat Nov 18 13:19:22 UTC 2017
New revision: 454429
URL: https://svnweb.freebsd.org/changeset/ports/454429

Log:
  Remove pkg-message about problems with gfortran.  The problem has been
  addressed in r454161.

  PR:		221127

Changes:
  head/math/R/pkg-message