Bug 177785 - security/ipsec-tools: ipsec-tools 0.8.0 racoon tends to segfault when multiple Phase1's aren't establishing
Summary: security/ipsec-tools: ipsec-tools 0.8.0 racoon tends to segfault when multipl...
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: VANHULLEBUS Yvan
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-04-11 20:20 UTC by Todd Blum
Modified: 2015-06-01 06:39 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Todd Blum 2013-04-11 20:20:00 UTC
I had two Phase1's that were not coming up, and it seemed like racoon
was segfaulting once every 4 days or so.  Then when I had three Phase1's
down, it seems like racoon started to segfault once every few days.

I've been noticing this behavior for several months.

I've opened this ticket at ipsec-tools with more detailed information:

https://sourceforge.net/tracker/?func=detail&aid=3603844&group_id=74601&atid=541482

Fix: 

My current workaround is to reboot if racoon crashes (falling back on
CARP slave while rebooting):

#!/bin/sh
#
# Find the pid of the process (PPID will be the shell that started it)
#  remember no spaces allowed between varnames, just equals sign, and the value
# Script name cannot contain the word racoon in order to to avoid self-triggering

sleep 30

FIND_PROC=`pgrep racoon`
# if FIND_PROC is empty, the process has died; restart it

if [ -z "${FIND_PROC}" ]; then
      echo racoon failed at `date`
       nohup  shutdown -r now &
fi

exit

With cron entry:

*     *     *     *     *     root     /root/ipsec-watchdog.sh >> /root/ipsec-watchdog.log
How-To-Repeat: Try configuring 3 or more Ipsec tunnels that will not come up, then wait
7 days.
Comment 1 Mark Linimon 2013-04-13 00:59:56 UTC
Does this apply to a port, or to the base system?
Comment 2 Todd Blum 2013-04-17 23:44:45 UTC
Hi Mark,

   This is for ports.  I should reference previous problem report:

http://www.freebsd.org/cgi/query-pr.cgi?pr=168104

Thanks,

Todd
Comment 3 Mark Linimon freebsd_committer freebsd_triage 2013-04-18 02:08:46 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-ports-bugs

ports PR.
Comment 4 Edwin Groothuis freebsd_committer freebsd_triage 2013-04-18 02:09:10 UTC
Responsible Changed
From-To: freebsd-ports-bugs->sumikawa

Over to maintainer (via the GNATS Auto Assign Tool)
Comment 5 Todd Blum 2013-04-22 18:37:01 UTC
The error message 'failed to get sainfo' is usually appearing in the logs
prior to the segfaults, then not at all afterwards:

Apr 16 09:59:51 192.168.116.250 racoon: [xx.xx.xxx.xx] ERROR: unknown
Informational exchange received.
Apr 16 09:59:55 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 09:59:57 192.168.116.250 racoon: [yy.yy.yy.yyy] ERROR: unknown
Informational exchange received.
Apr 16 10:00:02 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:02 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:16 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:23 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:23 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:41 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:44 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:00:44 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:02 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:06 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:06 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:14 192.168.116.250 racoon: INFO: respond new phase 1
negotiation: zz.zz.zz.zz[500]<=>hh.hh.hh.hh[500]
Apr 16 10:01:14 192.168.116.250 racoon: INFO: begin Identity Protection
mode.
Apr 16 10:01:17 192.168.116.250 racoon: INFO: ISAKMP-SA established
zz.zz.zz.zz[500]-hh.hh.hh.hh[500] spi:baa4c93e8c16198c:482ba6110eeabc0
c
Apr 16 10:01:17 192.168.116.250 racoon: INFO: purged IPsec-SA proto_id=ESP
spi=2201026904.
Apr 16 10:01:17 192.168.116.250 racoon: INFO: purged IPsec-SA proto_id=ESP
spi=3679806084.
Apr 16 10:01:18 192.168.116.250 racoon: INFO: respond new phase 2
negotiation: zz.zz.zz.zz[500]<=>hh.hh.hh.hh[500]
Apr 16 10:01:18 192.168.116.250 racoon: INFO: IPsec-SA established: ESP
zz.zz.zz.zz[500]->hh.hh.hh.hh[500] spi=119993144(0x726f338)
Apr 16 10:01:18 192.168.116.250 racoon: INFO: IPsec-SA established: ESP
zz.zz.zz.zz[500]->hh.hh.hh.hh[500] spi=2718404122(0xa2078e1a)
Apr 16 10:01:19 192.168.116.250 racoon: INFO: ISAKMP-SA expired
zz.zz.zz.zz[500]-hh.hh.hh.hh[500] spi:baa4c93e8c16198c:482ba6110eeabc0c
Apr 16 10:01:19 192.168.116.250 racoon: INFO: ISAKMP-SA deleted
zz.zz.zz.zz[500]-hh.hh.hh.hh[500] spi:baa4c93e8c16198c:482ba6110eeabc0c
Apr 16 10:01:27 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:27 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:30 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:51 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:51 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:01:55 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:13 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:13 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:16 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:37 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:37 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:40 192.168.116.250 racoon: ERROR: failed to get sainfo.
Apr 16 10:02:41 192.168.116.250 racoon: INFO: respond new phase 1
negotiation: zz.zz.zz.zz[500]<=>hh.hh.hh.hh[500]
Apr 16 10:02:41 192.168.116.250 racoon: INFO: begin Identity Protection
mode.
Apr 16 10:02:42 192.168.116.250 kernel: pid 45397 (racoon), uid 0: exited
on signal 11 (core dumped)

Is there any relation to this error report?

https://bugs.launchpad.net/ubuntu/+source/ipsec-tools/+bug/913935
Comment 6 Todd Blum 2013-04-23 15:21:55 UTC
Today I've found that I had duplicate IPSec tunnels configured in pfSense,
one disabled and the other enabled.

I've moved this tunnel elsewhere, and I've removed both from the pfSense
config to see if this improves my racoon stability.
Comment 7 Munechika Sumikawa freebsd_committer freebsd_triage 2013-04-24 08:19:59 UTC
Responsible Changed
From-To: sumikawa->ports

This is a bug report for security/ipsec-tools, not security/racoon2
Comment 8 Mark Linimon freebsd_committer freebsd_triage 2013-04-24 10:07:15 UTC
Responsible Changed
From-To: ports->freebsd-ports-bugs

Canonicalize assignment.
Comment 9 Todd Blum 2013-05-01 18:04:45 UTC
racoon segfaulted again, but this time without any sainfo messages.

The crash coincided with an ISP outage that affected at least 6 remote
endpoints.  DPD was enabled on these tunnels:

...
May  1 01:18:27 192.168.116.250 racoon: INFO: ISAKMP-SA deleted
my.end.poi.nt[500]-x.x.x.x [500] spi:48131b4e56ac24b8:32ef67f65454935e
May  1 01:18:28 192.168.116.250 racoon: [y.y.y.y ] INFO: DPD: remote
(ISAKMP-SA spi=622012ee7f51261d:7e39cc0f5ee916a0) seems to be dead.
May  1 01:18:28 192.168.116.250 racoon: INFO: purging ISAKMP-SA
spi=622012ee7f51261d:7e39cc0f5ee916a0.
May  1 01:18:28 192.168.116.250 racoon: INFO: purged IPsec-SA
spi=2284023606.
May  1 01:18:28 192.168.116.250 racoon: INFO: purged IPsec-SA spi=187964617.
May  1 01:18:28 192.168.116.250 racoon: INFO: purged ISAKMP-SA
spi=622012ee7f51261d:7e39cc0f5ee916a0.
May  1 01:18:28 192.168.116.250 racoon: INFO: ISAKMP-SA deleted
my.end.poi.nt[500]-y.y.y.y [500] spi:622012ee7f51261d:7e39cc0f5ee916a0
May  1 01:18:29 192.168.116.250 racoon: [z.z.z.z ] INFO: DPD: remote
(ISAKMP-SA spi=3c837090349206bf:1086e896dce5e982) seems to be dead.
May  1 01:18:29 192.168.116.250 racoon: INFO: purging ISAKMP-SA
spi=3c837090349206bf:1086e896dce5e982.
May  1 01:18:29 192.168.116.250 racoon: INFO: purged IPsec-SA
spi=3531119898.
May  1 01:18:29 192.168.116.250 racoon: INFO: purged IPsec-SA spi=124488619.
May  1 01:18:29 192.168.116.250 racoon: INFO: purged ISAKMP-SA
spi=3c837090349206bf:1086e896dce5e982.
...
Comment 10 VANHULLEBUS Yvan freebsd_committer freebsd_triage 2013-05-21 16:53:32 UTC
Responsible Changed
From-To: freebsd-ports-bugs->vanhu

Hi. 

Can you provide us a backtrace of the crash ?
Comment 11 Todd Blum 2013-05-29 22:11:18 UTC
E-mailed a core dump privately.

The problems seemed to have correlated with DSL outages of a specific ISP.  The
ISP replaced/repaired a DSL DSLAM and possibly some core routers as well.

racoon has been up stable now for several weeks since this change.
Comment 12 Xin LI freebsd_committer freebsd_triage 2015-06-01 06:39:39 UTC
Mark this as closed as there is no further complains (the port is now 0.8.2), and it haven't been touched on upstream for quite some time.

However, the problem does seem like that there is a remote DoS possibility which, without a backtrace it's highly unlikely to be traced down.  I'd suggest ping'ing the developers and see if they would have some discoveries.