Bug 244785 - sysutils/terraform: Crashes at runtime on 12-STABLE: netpoll: break fd ready for -2
Summary: sysutils/terraform: Crashes at runtime on 12-STABLE: netpoll: break fd ready ...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Mikael Urankar
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2020-03-13 06:38 UTC by pete
Modified: 2020-10-05 06:05 UTC (History)
8 users (show)

See Also:
bugzilla: maintainer-feedback? (lifanov)
mikael: merge-quarterly+


Attachments
patch with fix (3.05 KB, patch)
2020-07-25 06:06 UTC, Pavel Timofeev
no flags Details | Diff
v1 (1.22 KB, patch)
2020-10-03 13:31 UTC, Mikael Urankar
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description pete 2020-03-13 06:38:29 UTC
running 0.12.18 on amd64 12-STABLE and getting this error on startup:

$ terraform
runtime: netpoll: break fd ready for -2
fatal error: runtime: netpoll: break fd ready for something unexpected

runtime stack:
runtime.throw(0x1d9d0fa, 0x39)
	/usr/local/go/src/runtime/panic.go:1112 +0x72
runtime.netpoll(0x253748e59, 0x10e10b4ff201)
	/usr/local/go/src/runtime/netpoll_kqueue.go:140 +0x30a
runtime.findrunnable(0xc00004f000, 0x0)
	/usr/local/go/src/runtime/proc.go:2323 +0x72b
runtime.schedule()
	/usr/local/go/src/runtime/proc.go:2520 +0x2fc
runtime.park_m(0xc000001800)
	/usr/local/go/src/runtime/proc.go:2690 +0x9d
runtime.mcall(0x200000)
	/usr/local/go/src/runtime/asm_amd64.s:318 +0x5b

goroutine 1 [runnable, locked to thread]:
github.com/chzyer/readline.init()
	/wrkdirs/usr/ports/sysutils/terraform/work/terraform-0.12.18/vendor/github.com/chzyer/readline/operation.go:10

goroutine 6 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000094e60)
	/wrkdirs/usr/ports/sysutils/terraform/work/terraform-0.12.18/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100
created by go.opencensus.io/stats/view.init.0
	/wrkdirs/usr/ports/sysutils/terraform/work/terraform-0.12.18/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2020-03-13 07:56:41 UTC
Thank you for the report Pete

Could you provide additional information including:

- Exact FreeBSD version (uname -a)
- /etc/make.conf contents (as an attachment) if using ports
- Is this a recent change in behaviour? If so, when was the last working port version?
Comment 2 pete 2020-03-13 16:50:18 UTC
(In reply to Kubilay Kocak from comment #1)
here ya go:
1)
$ uname -ar
FreeBSD duke 12.1-STABLE FreeBSD 12.1-STABLE b26ce9ae44a(stable/12) GENERIC-NODEBUG  amd64

this is the commit i'm at (from Thurs):
commit b26ce9ae44a544746cf2c1682b85b90a48ef3544 (HEAD -> stable/12, origin/stable/12)
Author: mav <mav@FreeBSD.org>
Date:   Thu Mar 12 03:59:51 2020 +0000

    MFC r358683: Remove vfs.zfs.top_maxinflight tunable/sysctl.
    
    It is dead since sorted scrub import at r334844.

2) this is installed from the upstream FreeBSD pkg server: "pkg+http://pkg.FreeBSD.org/${ABI}/latest"

3) I have the same version of terraform working on my system running CURRENT.  I also verified that it wasn't an issue with the terraform templates I'm using (it fails even in an empty directory, and the same templates work Ok on my CURRENT system).

Let me know if you'd like me to try to get more debug info for you, or would like me to test any patches out.
Comment 3 Luca Pizzamiglio freebsd_committer freebsd_triage 2020-03-25 10:45:29 UTC
I installed terraform from packages and I've recently updated CURRENT.

I've the same error with netpoll.

My src is revision 359195.

I downloaded the tarball provided by HashiCorp (https://www.terraform.io/downloads.html) and it works (but it's a slightly different version).
Comment 4 a.milevsky 2020-04-09 16:31:40 UTC
I have the same problem.

1. uname -srm
FreeBSD 12.1-RELEASE-p3 amd64

2. go version go1.14.2 freebsd/amd64

3. If terraform compiled by go version go1.13.9 freebsd/amd64 or go version go1.13.10 freebsd/amd64 the error is disappear.

4. If terraform v0.11.14 compiled by any go version (1.13.9, 1.13.10, 1.14.2) the error is not appear at all.

5. I had see the same kind of problem with security/vault, sysutils/consul, net-mgmt/thanos, net-mgmt/prometheus2, www/grafana6 when wheh go version in ports changed from 13.7 to 14.0.
Comment 5 Antonio Huete Jimenez 2020-05-29 18:39:23 UTC
Relevant CL: https://go-review.googlesource.com/c/go/+/207078/
Comment 6 Lundin Cahill 2020-06-04 08:00:28 UTC
I just hit this on a FreeNAS 11.3 FreeBSD jail with terraform installed via pkg.

FreeNAS-11.3-U3.1
FreeBSD 11.3-RELEASE-p9
terraform-0.12.18

I merely fell back on an old pkg build to continue working for now.
Comment 7 Pavel Timofeev 2020-07-25 06:06:18 UTC
Created attachment 216755 [details]
patch with fix

Backport patch from terraform 0.13 branch.

issues that helped me to find this patch are
https://github.com/hashicorp/packer/issues/9032
https://github.com/hashicorp/terraform/pull/24562

Should should be committed to quarterly branch also I suppose!
Comment 8 Mikael Urankar freebsd_committer freebsd_triage 2020-07-25 13:18:02 UTC
(In reply to timp87 from comment #7)
you can use this construct to fetch upstream patch:

--- Makefile    (revision 543245)
+++ Makefile    (working copy)
@@ -5,6 +5,9 @@
 DISTVERSIONPREFIX=     v
 CATEGORIES=    sysutils
 
+PATCH_SITES=    https://github.com/${GH_ACCOUNT}/${GH_PROJECT}/commit/
+PATCHFILES=    cd65b28da051174a13ac76e54b7bb95d3051255c.patch:-p1
+
 MAINTAINER=    lifanov@FreeBSD.org
 COMMENT=       Provides a common configuration to launch infrastructure

and "make makesum"
Comment 9 a.milevsky 2020-07-25 15:34:42 UTC
(In reply to Mikael Urankar from comment #8)
I confirm: patch applied, terraform have been tested on two simple IaC - not panicked :)
Thank you!
Comment 10 Pavel Timofeev 2020-08-07 09:25:23 UTC
lifanov@ could you please review and approve the patch?
Comment 11 Mikael Urankar freebsd_committer freebsd_triage 2020-10-03 09:47:59 UTC
Can someone provides a patch for v0.13.4 and tests it?
Comment 12 a.milevsky 2020-10-03 12:10:16 UTC
(In reply to Mikael Urankar from comment #11)

Compiled from sources 0.13.4 works fine.
e.g. https://github.com/behemoth3663/ports_local/tree/master/terraform-0.13
Comment 13 Mikael Urankar freebsd_committer freebsd_triage 2020-10-03 13:31:47 UTC
Created attachment 218491 [details]
v1
Comment 14 commit-hook freebsd_committer freebsd_triage 2020-10-03 13:40:01 UTC
A commit references this bug:

Author: mikael
Date: Sat Oct  3 13:39:08 UTC 2020
New revision: 551315
URL: https://svnweb.freebsd.org/changeset/ports/551315

Log:
  sysutils/terraform: update to 0.13.4

  Update to 0.13.4 which fixes a crash at runtime:
  netpoll: break fd ready for -2

  PR:		244785
  Submitted by:	a.milevsky@gmail.com
  Reported by:	many
  Approved by:	lifanov (maintainer timeout, 6 months)
  MFH:		2020Q4 (runtime fix)

Changes:
  head/sysutils/terraform/Makefile
  head/sysutils/terraform/distinfo
Comment 15 commit-hook freebsd_committer freebsd_triage 2020-10-05 06:05:22 UTC
A commit references this bug:

Author: mikael
Date: Mon Oct  5 06:05:18 UTC 2020
New revision: 551476
URL: https://svnweb.freebsd.org/changeset/ports/551476

Log:
  MFH: r551315

  sysutils/terraform: update to 0.13.4

  Update to 0.13.4 which fixes a crash at runtime:
  netpoll: break fd ready for -2

  PR:		244785
  Submitted by:	a.milevsky@gmail.com
  Reported by:	many
  Approved by:	lifanov (maintainer timeout, 6 months)

  Approved by:	ports-secteam (fluffy)

Changes:
_U  branches/2020Q4/
  branches/2020Q4/sysutils/terraform/Makefile
  branches/2020Q4/sysutils/terraform/distinfo