Bug 201143

Summary: devel/ice: New release 3.6.0
Product: Ports & Packages Reporter: Roger Leigh <rleigh>
Component: Individual Port(s)Assignee: Michael Gmelin <grembo>
Status: Closed FIXED    
Severity: Affects Only Me CC: freebsd, grembo
Priority: --- Keywords: patch
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
poudriere build of devel/ice
none
poudriere build of devel/py-ice
none
poudriere build of devel/php5-ice
none
Poudriere log for failing test IceGrid/simple with 10.1-RELEASE / D2930 patch
none
poudriere build log of devel/ice on 10.1 after the latest changes
none
poudriere build log of devel/php5-ice on 10.1 after the latest changes
none
poudriere build log of devel/py-ice on 10.1 after the latest changes
none
poudriere build log of devel/ice on 10.1 as of 2015-07-04
none
poudriere build log of devel/php5-ice on 10.1 as of 2015-07-04
none
poudriere build log of devel/py-ice on 10.1 as of 2015-07-04 none

Description Roger Leigh 2015-06-27 08:51:58 UTC
If this could be updated at some point, it would be greatly appreciated.

The upstream source releases have been moved to github; the 3.6.0 
release is here:
https://github.com/zeroc-ice/ice/archive/v3.6.0.tar.gz

The main change of note seems to be the dropping of python (you can install with pip).  That might need packaging as separate ports for python 2 and 3?

The other thing is the switching of the java build to use gradle.  That 
might need some checking to be sure it's not downloading stuff during 
the build that should be satisfied through stuff available in the ports collection.  They also seem to have embedded some sort of wrapper around gradle which downloads it; it should probably use devel/gradle (is it too old)?

If there's anything particular I can do to assist, or you want any help 
testing, please just let me know.


Kind regards,
Roger
Comment 1 Michael Gmelin freebsd_committer 2015-06-27 09:14:22 UTC
I started working on the port this morning. We never had a java port, so that makes this part easier (I might create one in the process though).

I did a port for 3.6 beta quite some time ago (you can find it in the Patches section of the zeroc forum), but that was before the move to github. Github download is quite easy thanks to GH_ support in Mk, let's see what other changes in the build system might complicate that.

As soon as I have something that builds, I'll share the patches with you for testing.

One thing the port never got right was building and running with OpenSSL from ports and LibreSSL - if you have any experience there, any help would be appreciated.
Comment 2 Roger Leigh 2015-06-27 17:52:54 UTC
I haven't got much experience with the open/libressl from ports, but I can have a look at it once you've got something that builds.  My previous experience was with the Ice packaging for Debian and MacOS X homebrew, but I'll be happy to look at the port build.  [For both Debian and homebrew we historically had to patch it fairly extensively, but it looks a bit cleaner in 3.6.0]
Comment 3 Michael Gmelin freebsd_committer 2015-06-28 12:10:07 UTC
Created attachment 158125 [details]
poudriere build of devel/ice
Comment 4 Michael Gmelin freebsd_committer 2015-06-28 12:10:46 UTC
Created attachment 158126 [details]
poudriere build of devel/py-ice
Comment 5 Michael Gmelin freebsd_committer 2015-06-28 12:11:16 UTC
Created attachment 158127 [details]
poudriere build of devel/php5-ice
Comment 6 Michael Gmelin freebsd_committer 2015-06-28 12:18:53 UTC
This was more effort than expected, all three ports build ok now (see poudrier logs), I don't use the PHP port for anything serious, an in-depth test of C++ and Python looks good.

You can find the code review request for this change here:
https://reviews.freebsd.org/D2930

Direct link to raw diff:
https://reviews.freebsd.org/D2930?download=true
Comment 7 Roger Leigh 2015-06-29 14:57:15 UTC
Thanks.  I tried to give these a test, applying your patch to SVN ports/head, but failed running the tests:

*** running tests 52/93 in /usr/ports/devel/ice/work/ice-3.6.0/cpp/test/Ice/networkProxy
*** configuration: Default 
*** test started: 06/29/15 15:52:21
*** using Ice source dist (64bit) 
starting SOCKS proxy... ok
starting server... ok
starting client... ok
ConnectionI.cpp:2044: Ice::ConnectTimeoutException:
timeout while establishing a connection
unexpected exit status: expected: 0, got 1
('test in /usr/ports/devel/ice/work/ice-3.6.0/cpp/test/Ice/networkProxy failed with exit status', 1)
*** Error code 1

This is as root in the ports tree inside a VirtualBox VM; networking should be fully functional and all the prior tests passed.


Regards,
Roger
Comment 8 Michael Gmelin freebsd_committer 2015-06-29 15:18:57 UTC
Well, I built on bare metal as well as jails and it builds.

What is the network configuration of your VM (and should we care)?
Comment 9 Michael Gmelin freebsd_committer 2015-06-29 15:20:00 UTC
Could you maybe try building it using poudriere?

This will give all details (like OS version etc.), to narrow it down a bit more.

Does the previous version (Ice 3.5.1) build on your host? Might also be IPv6 related.
Comment 10 Michael Gmelin freebsd_committer 2015-06-29 15:32:56 UTC
If you can't test poudriere, you need to give exact details about your setup (OS version, access to the internet etc.).

I tested the ports out of ports on 10.0, using poudriere on 10.0 and 10.1 and also within a VM running 10.1 out of ports.

If you have a specific application you want to check, just build without TEST and go ahead and test your app.
Comment 11 Michael Gmelin freebsd_committer 2015-06-29 17:11:18 UTC
One more idea: As you're running this in VirtualBox, the timeout exception might be due to emulated system clock issues. I've seen the system clock move backwards and similar issues when running FreeBSD on VirtualBox before, especially under heavy load.

Do you see anything special in /var/log/messages or dmesg?
Comment 12 Roger Leigh 2015-06-29 20:23:19 UTC
I'll have to wait until I'm back at work tomorrow to retest the virtualbox failure.  I'm trying on my home PC (bare metal 8-core AMD FX8350 with 16GB RAM with FreeBSD 10.1).  This gets much further but fails here:

*** running tests 71/93 in /usr/ports/devel/ice/work/ice-3.6.0/cpp/test/IceGrid/simple
*** configuration: Default 
*** test started: 06/29/15 21:07:28
*** using Ice source dist (64bit) 
starting icegrid registry... ok
starting icegrid replica-1... ok
starting icegrid replica-2... ok
starting server... ok
starting client... ok
testing stringToProxy... ok
testing IceGrid.Locator is present... ok
testing checked cast... ok
pinging server... ok
testing locator finder... ok
testing discovery... ok
shutting down server... ok
shutting down icegrid replica-2... ok
shutting down icegrid replica-1... ok
shutting down icegrid registry... ok
unexpected exit status: expected: 0, got -6
('test in /usr/ports/devel/ice/work/ice-3.6.0/cpp/test/IceGrid/simple failed with exit status', 1)
*** Error code 1

dmesg for two separate "make install" runs:
pid 96158 (icegridregistry), uid 0: exited on signal 6 (core dumped)
pid 96157 (icegridregistry), uid 0: exited on signal 6 (core dumped)
pid 96156 (icegridregistry), uid 0: exited on signal 6 (core dumped)
pid 97369 (icegridregistry), uid 0: exited on signal 6 (core dumped)
pid 97368 (icegridregistry), uid 0: exited on signal 6 (core dumped)
pid 97367 (icegridregistry), uid 0: exited on signal 6 (core dumped)

Nothing more informative in /var/log/messages.  Interestingly, I did get this from work/ice-3.6.0/cpp/test/IceGrid/simple/icegridregistry.core:

#0  0x00000008035c56ca in thr_kill () from /lib/libc.so.7
#1  0x000000080369a149 in abort () from /lib/libc.so.7
#2  0x0000000802cf342d in __cxa_rethrow () from /lib/libcxxrt.so.1
#3  0x000000000046a99e in IceUtil::Mutex::lock ()
#4  0x0000000802804fa9 in IceUtil::Shared::__decRef () from /usr/ports/devel/ice/work/ice-3.6.0/cpp/lib/libIceUtil.so.36
#5  0x0000000000587d87 in (anonymous namespace)::Init::~Init ()
#6  0x0000000803674280 in __cxa_finalize () from /lib/libc.so.7
#7  0x0000000803615d4c in exit () from /lib/libc.so.7
#8  0x000000000043e956 in _start ()
#9  0x00000008008c8000 in ?? ()
#10 0x0000000000000000 in ?? ()

Some locking issue during exit cleanup?


Networking is simple IPv4 NAT + v6:
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 60:a4:4c:5f:12:57
        inet6 fe80::62a4:4cff:fe5f:1257%re0 prefixlen 64 scopeid 0x1 
        inet6 2001:8b0:860:ddbd:62a4:4cff:fe5f:1257 prefixlen 64 autoconf 
        inet 192.168.1.134 netmask 0xffffff00 broadcast 192.168.1.255 
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

I'll retry with poudriere once I've set it up.
Comment 13 Michael Gmelin freebsd_committer 2015-06-29 20:41:29 UTC
Regarding VirtualBox:

After a failed build (the current state?) try doing the following a couple of times:

  cd /usr/ports/devel/ice/work/ice-3.6.0/cpp/test/Ice/networkProxy
  ./run.py

See if it keeps failing or only failes somtimes, observer /var/log/messages and the wall clock.

Regarding your bare metal build failure:
This is very strange. I know this problem, it is caused by a dynamic linker problem in 10.0-RELEASE and I added a workaround for it in a patch that depends on the FreeBSD version in use. The fact that you're experiencing it makes we wonder if you're running an inconsistent installation (maybe your world is not complete). Would be interesting to see the output of the following commands:

  - uname -a
  - freebsd-version
  - grep FreeBSD_version /usr/include/sys/param.h

Regarding poudriere:
That's surprisingly easy, run:

  pkg install poudriere
  poudriere jail -c -j 101amd64 -v 10.1-RELEASE
  poudriere ports -c -p local

# Patch the port skeleton in /usr/local/poudriere/ports/local (devel/ice etc.)

  poudriere testport -j 101amd64 -o devel/ice -v -p local
Comment 14 Roger Leigh 2015-06-29 22:25:11 UTC
For the bare metal system:

% uname -a
FreeBSD sorilea 10.1-RELEASE-p10 FreeBSD 10.1-RELEASE-p10 #0: Wed May 13 06:54:13 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

% freebsd-version
10.1-RELEASE-p13

% grep FreeBSD_version /usr/include/sys/param.h 
 * __FreeBSD_version numbers are documented in the Porter's Handbook.
#undef __FreeBSD_version
#define __FreeBSD_version 1001000       /* Master, propagated to newvers */

I've tested on the base system and with poudriere and I get the same segfault consistently whenever I run the IceGrid/simple test.

The system was a clean 10.1 install in Nov 2014; it's kept up-to-date with freebsd-update and pkg, and has no custom alterations or customisations other than a handful of files in /etc like rc.conf.  I just updated from p10 to p13; rebooted in case the kernel needed updating (but it didn't) and this didn't change the failure.
Comment 15 Michael Gmelin freebsd_committer 2015-06-29 22:51:16 UTC
That's pretty strange, as I cannot reproduce the issue on my 10.1 systems.

If you modify devel/ice/files/patch-cpp-src-IceGrid-PluginFacadeI.cpp so that line 7 says

+#if 0

instead of

+#if !defined(ICE_BROKEN_ATEXIT)

it will probably work.

This is likely to be a crash on exit due to invalid order of static destruction, I originally thought this was 10.0 only (as there is another problem in Ice that's definitely legal code and broke on 10.0). This one probably needs some investigation to see if it's FreeBSD specific or a problem upstream.
Comment 16 Michael Gmelin freebsd_committer 2015-06-29 22:58:56 UTC
Maybe you could attach your poudriere build log of the failed build/test.
Comment 17 Michael Gmelin freebsd_committer 2015-06-29 23:17:06 UTC
My last spam on the issue for tonight:

When you're back at work, could you also run that test inside of the VM?

 cd /usr/ports/devel/ice/work/ice-3.6.0/cpp/test/IceGrid/simple
 ./run.py

I also tried running icegridregistry and that unit test on a host I just installed a few days ago (FreeBSD 10.1-RELEASE-p10, older Xeon X3430) and it works fine.
Comment 18 Roger Leigh 2015-06-29 23:41:17 UTC
Created attachment 158172 [details]
Poudriere log for failing test IceGrid/simple with 10.1-RELEASE / D2930 patch
Comment 19 Roger Leigh 2015-06-30 10:26:03 UTC
I have retested in the VirtualBox VM in the base system ports tree and with poudriere.  

Ice/networkProxy - times out same as before in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=201143#c7 for both ports and poudriere
IceGrid/simple - segfaults for both ports and poudriere as for the bare metal system


Networking on the VM does have two interfaces, not that it should really break anything.  I'll retry with just one to see if it changes anything.

vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=6c03bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 08:00:27:d1:68:c8
	inet6 fe80::a00:27ff:fed1:68c8%vtnet0 prefixlen 64 scopeid 0x1
	inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=6c03bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 08:00:27:90:6e:63
	inet6 fe80::a00:27ff:fe90:6e63%vtnet1 prefixlen 64 scopeid 0x2
	inet 192.168.56.102 netmask 0xffffff00 broadcast 192.168.56.255
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
	inet 127.0.0.1 netmask 0xff000000
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Comment 20 Michael Gmelin freebsd_committer 2015-06-30 11:38:34 UTC
I was able to reproduce and track down the problem in icegridregistry which makes IceGrid/simple fail (it's only exposed here when building with DEBUG=on, as an assertion in IceUtil::Handle is triggered. If it segfaults or not seems to depend on additional factors).

This is what happens:

IceGrid/PluginFacadeI.cpp makes assumptions about the order of static initialization and (what is causing the problem in this case) static deinitialization in another translation unit (libIceGrid). IMHO this is not covered by the standard - the order is only guaranteed within one translation unit.

This is in IceGrid/PluginFacadeI.cpp:

 namespace
 {

 class Init
 {
 public:

     Init()
     {
         IceGrid::setRegistryPluginFacade(new RegistryPluginFacadeI);
     }

     ~Init()
     {
         IceGrid::setRegistryPluginFacade(0);
     }
 };

 Init init;
 ...

And this is in IceGridLib/PluginFacadeI.cpp [same filename, different path, this goes into libIceGrid.so, which icegridnode and icegridregistry link against]:

 namespace
 {

 RegistryPluginFacadePtr pluginFacade;

 };

 ...

 void
 IceGrid::setRegistryPluginFacade(const RegistryPluginFacadePtr& facade)
 {
     pluginFacade = facade;
 }

So if pluginFacade is deinitialized before init in the other translation unit on exit, bad things happen (also if it's not initialized just yet, which works out in this case but is also not guaranteed).

I removed the static initialization class Init in IceGrid/PluginFacadeI.cpp and moved the calls to setRegistryPluginFacade to the ctors/dtors of RegistryI (which, looking at the code, is a reasonable place for these calls anyway).

I created a new iteration of the patch: https://reviews.freebsd.org/D2930?download=true, would be great if you could test that.

Regarding the networkProxy unit test:
Maybe you could play a but with its run.py, e.g. those lines:

TestUtil.clientServerTest(additionalClientOptions="--Ice.SOCKSProxyHost=localhost --Ice.SOCKSProxyPort=12030")

TestUtil.clientServerTest(additionalClientOptions="--Ice.HTTPProxyHost=localhost --Ice.HTTPProxyPort=12031")

It seems like it's using a mix of "default" and localhost within the tests, also specifying an udp endpoint. A might have to setup VirtualBox myself to understand better what's going on there.
Comment 21 Roger Leigh 2015-06-30 13:04:30 UTC
With VirtualBox and only a single network interface, I still see the failure, so it's not related to having two interfaces.

WRT the icegridnode segfault, shall I report this back to ZeroC or do you want to do that?


Thanks,
Roger
Comment 22 Michael Gmelin freebsd_committer 2015-06-30 14:03:02 UTC
Can you confirm that the change to icegridregistry is working for you as well?

I'll report all bugs and updates to ZeroC once this works (this patch also includes missing functionality, so there's a lot to be merged). We need to sort out a few legal issues (contributor agreement) before I can send pull requests to them though. I don't think that's really time critical though, as it doesn't seem to happen on their supported platforms.

Did you have to chance to experiment a bit more with VirtualBox?
Comment 23 Michael Gmelin freebsd_committer 2015-06-30 17:41:01 UTC
I also tried building in VirtualBox and it worked just fine (That's on VirtualBox 4.3.22, dual CPU, 4GB RAM). So not much I can do about this right now. Can you maybe paste your /etc/rc.conf /etc/pf.conf etc.?

Here are some system details:

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 08:00:27:d5:b0:ce
        inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.1.1.1 netmask 0xffffff00 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

[root@ /usr/ports/lang/python]# freebsd-version
10.1-RELEASE-p13
[root@ /usr/ports/lang/python]# uname -a
FreeBSD  10.1-RELEASE-p10 FreeBSD 10.1-RELEASE-p10 #0: Wed May 13 06:54:13 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
[root@ /usr/ports/lang/python]# 

  
root@:/usr/ports/devel/ice/work/ice-3.6.0/cpp/test/Ice/networkProxy # ./run.py
*** using Ice source dist (64bit) 
starting SOCKS proxy... ok
starting server... ok
starting client... ok
testing connection... ok
testing connection information... ok
shutting down server... ok
testing connection failure... x2ok
terminating SOCKS proxy... ok
starting HTTP proxy... ok
starting server... ok
starting client... ok
testing connection... ok
testing connection information... ok
shutting down server... ok
testing connection failure... ok     
terminating HTTP proxy... ok
root@:/usr/ports/devel/ice/work/ice-3.6.0/cpp/test/Ice/networkProxy #
Comment 24 Roger Leigh 2015-06-30 21:02:28 UTC
OK, I've spent a few hours doing some more testing with your latest patch.

- bare metal, poudriere: all tests pass and build completes
- bare metal, /usr/ports: all tests pass and build completes
- VirtualBox VM (Windows, different machine): poudriere: Ice/networkProxy still failing.  I'll play some more with run.py tomorrow to see if I can figure this out.  Seems VM-specific though; I'll see if I can tweak the network adapter settings and I'll post the configuration details you wanted (it will have to wait until work tomorrow though).
Comment 25 Michael Gmelin freebsd_committer 2015-07-01 10:14:01 UTC
Created attachment 158212 [details]
poudriere build log of devel/ice on 10.1 after the latest changes
Comment 26 Michael Gmelin freebsd_committer 2015-07-01 10:14:30 UTC
Created attachment 158213 [details]
poudriere build log of devel/php5-ice on 10.1 after the latest changes
Comment 27 Michael Gmelin freebsd_committer 2015-07-01 10:15:40 UTC
Created attachment 158214 [details]
poudriere build log of devel/py-ice on 10.1 after the latest changes
Comment 28 Michael Gmelin freebsd_committer 2015-07-01 10:21:51 UTC
Output of sysctl -a could be interesting as well.

If you can't figure it out and your VM doesn't contain confidential information you could tar up / and give me a download link to it.
Comment 29 Roger Leigh 2015-07-01 23:47:18 UTC
Just FYI, I've got it successfully building in VirtualBox.  The changes I made were
- changed the adapter type from virtual to intel pro/1000 MT desktop (82549EM); corresponding change from vtnet0 to em0 in rc.conf.  This had no effect.
- changed the adapter from bridged network with promiscuous mode enabled to NAT
- changed the number of CPUs from 1 to 4

So it doesn't look like it's anything to do with the FreeBSD system configuration, which is pretty much a stock install.  I've not tested every adapter type/adapter/cpu combination, but it's certainly not a problem with this port.


4 vCPUs:
em0 bridged:    OK
em0 NAT:        OK
vtnet0 NAT:     OK
vtnet0 bridged: OK

1 vCPU:
em0 bridged:    FAIL
em0 NAT:        FAIL
vtnet0 NAT:     FAIL
vtnet0 bridged: FAIL

So looks like this is down to the number of vCPUs configured.  Not sure why that would be though.  So should hopefully be fairly simple to reproduce if you just set the number of CPUs to 1 unless there are additional complicating factors.
Comment 30 Roger Leigh 2015-07-02 10:34:05 UTC
Just a thought, wasn't there a situation with the Java code with Ice 3.5 that the socket options behaved differently on BSD under some conditions and their code wasn't handling the special case properly?  Could it be a similar situation here--just thinking of why the behaviour might change with SMP.
Comment 31 Michael Gmelin freebsd_committer 2015-07-04 13:04:28 UTC
Thanks for testing, it really helps to make this port better.

I found the problem and your suspicion was right, it is indeed similar to the Java problem in that it evolves around handling POSIX compliant synchronous calls to connect(2) incorrectly. The difference here is that it only applies to the NetworkProxy code and is only a bug and not a design error.

When a connection is done synchronously, the code detects this correctly and sets the status to StateConnected. In case of proxies this is not correct, as they should go to StateProxyWrite, StateProxyConnected and then to StateConnected.

Tracking it down was a bit nasty, fixing it quite easy.

I also refined the patch for the static deinitialization problem (basically an incarnation of [1]), as it turns on custom load balancing plugins depend on static initialization of the facade by design [2] and breaking a documented API isn't a good idea. The unit test for this is only run when the port is not built by root, that's why I missed that one at first. I patched unit tests which can't be run by root, so they drop to uid 65534 when run as root to make sure they provide maximum coverage.

A new version of the patch is available on phabricator:
https://reviews.freebsd.org/D2930?download=true

[1] https://isocpp.org/wiki/faq/ctors#construct-on-first-use-v2
[2] https://doc.zeroc.com/display/Ice36/Load+Balancing#LoadBalancing-custom
Comment 32 Michael Gmelin freebsd_committer 2015-07-04 13:05:19 UTC
Created attachment 158330 [details]
poudriere build log of devel/ice on 10.1 as of 2015-07-04
Comment 33 Michael Gmelin freebsd_committer 2015-07-04 13:08:23 UTC
Created attachment 158331 [details]
poudriere build log of devel/php5-ice on 10.1 as of 2015-07-04
Comment 34 Michael Gmelin freebsd_committer 2015-07-04 13:08:54 UTC
Created attachment 158332 [details]
poudriere build log of devel/py-ice on 10.1 as of 2015-07-04
Comment 35 Michael Gmelin freebsd_committer 2015-07-13 13:54:08 UTC
@Roger: Did you have a chance to test the latest patch? I'm very confident that it will be ok, but after you spent so much effort testing I want to make sure it really works on your setup (and also add you as a tester on commit).
Comment 36 Roger Leigh 2015-07-13 14:48:02 UTC
Sorry for the delay.  I retested with the latest patch and it all works for me.

One minor discrepancy: the default build target for ice runs the unit tests, but the default target for py-ice does not (make test runs them though).  Not sure if that's intentional or by design.  In both cases TESTS was enabled with "make config".


Regards,
Roger
Comment 37 Michael Gmelin freebsd_committer 2015-07-13 19:33:25 UTC
Are you certain this is the case?

Both ports (ice and py-ice) should run unit tests. During the build stage "TEST" should only build the unit tests for Ice, but they should be run on make stage (which is run by "make").

I just tested myself (removed all options from /var/db/ports to build with default settings) and both ports ran their unit tests like expected.

I discovered one more issue running the unit test IceGrid/activation. This hangs quite often on a machine with many cores. This seems to be a timing issue in IceGrid/Activator.cpp, but I couldn't really track it down. This unit test never ran before and I could reproduce the issue on the current Ice 3.5.1 port as well (when building as an unprivileged user). As I could only reproduce it on 10.0 and not on 10.1 I'll ignore the issue for now (10.0 is EoL anyway).
Comment 38 commit-hook freebsd_committer 2015-07-13 19:48:47 UTC
A commit references this bug:

Author: grembo
Date: Mon Jul 13 19:48:41 UTC 2015
New revision: 391942
URL: https://svnweb.freebsd.org/changeset/ports/391942

Log:
  Update devel/ice, devel/py-ice and devel/php5-ice to 3.6.0

  PR:		201143
  Differential Revision:	https://reviews.freebsd.org/D2930
  Reviewed by:	bapt
  Approved by:	bapt
  Tested by:	Roger Leigh <rleigh@codelibre.net>

Changes:
  head/devel/ice/Makefile
  head/devel/ice/distinfo
  head/devel/ice/files/Make.rules.FreeBSD
  head/devel/ice/files/patch-config-Make.common.rules
  head/devel/ice/files/patch-cpp-Makefile
  head/devel/ice/files/patch-cpp-allTests.py
  head/devel/ice/files/patch-cpp-config-Make.rules
  head/devel/ice/files/patch-cpp-include-Ice-Basicstream.h
  head/devel/ice/files/patch-cpp-include-Ice-FactoryTableInit.h
  head/devel/ice/files/patch-cpp-include-Ice-IconvStringConverter.h
  head/devel/ice/files/patch-cpp-include-IceUtil-Config.h
  head/devel/ice/files/patch-cpp-include-IceUtil-ScannerConfig.h
  head/devel/ice/files/patch-cpp-src-Glacier2CryptPermissionsVerifier-CryptPermissionsVerifierI.cpp
  head/devel/ice/files/patch-cpp-src-Ice-.depend
  head/devel/ice/files/patch-cpp-src-Ice-StreamSocket.cpp
  head/devel/ice/files/patch-cpp-src-IceGrid-AdapterCache.cpp
  head/devel/ice/files/patch-cpp-src-IceGrid-AdapterCache.h
  head/devel/ice/files/patch-cpp-src-IceGrid-Database.cpp
  head/devel/ice/files/patch-cpp-src-IceGrid-PluginFacadeI.cpp
  head/devel/ice/files/patch-cpp-src-IceGrid-RegistryI.cpp
  head/devel/ice/files/patch-cpp-src-slice2cpp-Gen.cpp
  head/devel/ice/files/patch-cpp-test-Glacier2-staticFiltering-run.py
  head/devel/ice/files/patch-cpp-test-Ice-info-AllTests.cpp
  head/devel/ice/files/patch-cpp-test-Ice-metrics-AllTests.cpp
  head/devel/ice/files/patch-cpp-test-IceGrid-admin-run.py
  head/devel/ice/files/patch-cpp-test-IceGrid-deployer-AllTests.cpp
  head/devel/ice/files/patch-cpp-test-IceGrid-deployer-Makefile
  head/devel/ice/files/patch-cpp-test-IceGrid-deployer-application.xml
  head/devel/ice/files/patch-cpp-test-IceGrid-distribution-AllTests.cpp
  head/devel/ice/files/patch-cpp-test-IceGrid-distribution-application.xml
  head/devel/ice/files/patch-cpp-test-IceGrid-distribution-run.py
  head/devel/ice/files/patch-cpp-test-IceGrid-session-run.py
  head/devel/ice/files/patch-cpp-test-IceGrid-simple-AllTests.cpp
  head/devel/ice/files/patch-cpp-test-IceSSL-configuration-AllTests.cpp
  head/devel/ice/files/patch-cpp-test-IceSSL-configuration-run.py
  head/devel/ice/files/patch-cpp-test-Slice-headers-run.py
  head/devel/ice/files/patch-php-config-Make.rules.php
  head/devel/ice/files/patch-py-Makefile
  head/devel/ice/files/patch-py-config-Make.rules
  head/devel/ice/files/patch-py-modules-IcePy-Types.cpp
  head/devel/ice/files/patch-py-modules-IcePy-Types.h
  head/devel/ice/files/patch-py-python-Makefile
  head/devel/ice/files/patch-py-test-Ice-info-AllTests.py
  head/devel/ice/files/patch-python-Makefile
  head/devel/ice/files/patch-python-config-Make.rules
  head/devel/ice/files/patch-python-modules-IcePy-Types.cpp
  head/devel/ice/files/patch-python-modules-IcePy-Types.h
  head/devel/ice/files/patch-python-python-Makefile
  head/devel/ice/files/patch-python-test-Ice-info-AllTests.py
  head/devel/ice/files/patch-scripts-Expect.py
  head/devel/ice/files/patch-scripts-IceGridAdmin.py
  head/devel/ice/files/patch-scripts-TestUtil.py
  head/devel/ice/files/patch-scripts-icehashpassword.py
  head/devel/ice/pkg-plist
  head/devel/php5-ice/Makefile
  head/devel/php5-ice/pkg-plist
  head/devel/py-ice/Makefile
  head/devel/py-ice/pkg-plist
Comment 39 Michael Gmelin freebsd_committer 2015-07-13 19:51:42 UTC
I committed the changes, please reopen the bug in case there are any pending issues I didn't address.
Comment 40 Roger Leigh 2015-07-16 19:16:52 UTC
My mistake about the py-ice tests; they must have scrolled past before I saw it!  I retested with the current ports and it's as you described.  So many thanks for all your hard work!