Bug 220827 - net/mosquitto: Fails to build with WS option (websockets support) enabled
Summary: net/mosquitto: Fails to build with WS option (websockets support) enabled
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2017-07-18 12:35 UTC by stl
Modified: 2017-08-18 17:04 UTC (History)
2 users (show)

See Also:
joe: maintainer-feedback+
iblis.dif01: maintainer-feedback+
koobs: merge-quarterly?


Attachments
mosquitto patch (1.94 KB, patch)
2017-07-18 16:33 UTC, Iblis Lin
joe: maintainer-approval+
Details | Diff
Stack trace, debug info (20.17 KB, text/plain)
2017-07-24 16:11 UTC, stl
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description stl 2017-07-18 12:35:36 UTC
Building net/mosquitto with websockets support enabled now fails with the below error.  The port is also a few versions behind the latest mosquitto release (1.4.10 vs 1.4.14).  At first glance, this could be an issue in net/libwebsockets...

FAILED: src/CMakeFiles/mosquitto.dir/__/lib/read_handle_shared.c.o 
/usr/bin/cc -DCMAKE -DTIMESTAMP="\"2017-07-18 14:28:16+0200\"" -DVERSION=\"1.4.10\" -DWITH_BRIDGE -DWITH_BROKER -DWITH_EC -DWITH_MEMORY_TRACKING -DWITH_PERSISTENCE -DWITH_SOCKS -DWITH_SYS_TREE -DWITH_TLS -DWITH_TLS_PSK -DWITH_UUID -DWITH_WEBSOCKETS -I/usr/local/include -I. -Isrc -Ilib -O2 -pipe  -fstack-protector -fno-strict-aliasing -O2 -pipe  -fstack-protector -fno-strict-aliasing -MD -MT src/CMakeFiles/mosquitto.dir/__/lib/read_handle_shared.c.o -MF src/CMakeFiles/mosquitto.dir/__/lib/read_handle_shared.c.o.d -o src/CMakeFiles/mosquitto.dir/__/lib/read_handle_shared.c.o   -c lib/read_handle_shared.c
In file included from lib/read_handle_shared.c:29:
In file included from lib/util_mosq.h:25:
In file included from src/mosquitto_broker.h:24:
In file included from /usr/local/include/libwebsockets.h:209:
In file included from /usr/local/include/uv.h:62:
In file included from /usr/local/include/uv-unix.h:42:
/usr/include/pthread.h:238:38: error: use of undeclared identifier '__mutex'
                        __nonnull(1) __requires_unlocked(*__mutex);
                                                          ^
/usr/include/pthread.h:241:38: error: use of undeclared identifier '__mutex'
                        __nonnull(1) __requires_unlocked(*__mutex);
                                                          ^
/usr/include/pthread.h:243:36: error: use of undeclared identifier '__mutex'
                        __nonnull(1) __locks_exclusive(*__mutex);
                                                        ^
/usr/include/pthread.h:250:28: error: use of undeclared identifier '__mutex'
                        __nonnull(1) __unlocks(*__mutex);
                                                ^
4 errors generated.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-18 13:20:20 UTC
Version updates should be separated if possible, from bugfixes so that the latter can be merged to quarterly (if affected)

CC net/libwebsockets MAINTAINER
Comment 2 Iblis Lin 2017-07-18 13:57:29 UTC
The error message show something wrong in pthread? :/

which version of FreeBSD? I can setup a VM and try to reproduce it...
Comment 3 stl 2017-07-18 14:27:45 UTC
(In reply to Iblis Lin from comment #2)
> which version of FreeBSD? I can setup a VM and try to reproduce it...

This is on 11.0-RELEASE.

libwebsockets was built without the LIBUV option explicitly being set, but LWS_USE_LIBUV ended up being defined in lws_config.h.  I guess its presence was autodetected rather than respecting the option.  But none of that explains the error.. including pthread.h on its own of course works fine.
Comment 4 stl 2017-07-18 14:32:50 UTC
Installing net/libwebsockets from pkg (which was built without the LIBUV option, and LWS_USE_LIBUV didn't get implicitly set) results in being able to build net/mosquitto with WS enabled.  However, it doesn't actually work:

Starting mosquitto.
Error: Websockets support not available.
Comment 5 stl 2017-07-18 14:46:46 UTC
(In reply to stl from comment #4)
> However, it doesn't actually work:
> 
> Starting mosquitto.
> Error: Websockets support not available.

Scratch that, my apologies, I was running the binary I kept aside to use during testing ;-)

So, the original problem still remains when building net/mosquitto when net/libwebsockets was built with LIBUV either detected or explicitly enabled.
Comment 6 Iblis Lin 2017-07-18 14:49:46 UTC
(In reply to stl from comment #4)

Does `poudriere bulk` work for you? 

I just ran poudriere testport with a 11.0 jail (using -c to enable WS support). It passed.
Comment 7 stl 2017-07-18 15:02:28 UTC
(In reply to Iblis Lin from comment #6)
> Does `poudriere bulk` work for you? 
> 
> I just ran poudriere testport with a 11.0 jail (using -c to enable WS support). > It passed.

I assume that works because libuv is not installed and has not implicitly been pulled in by libwebsockets...
Comment 8 Iblis Lin 2017-07-18 16:33:41 UTC
Created attachment 184476 [details]
mosquitto patch

Please try this patch.

(I do not strictly test mosquitto
Comment 9 stl 2017-07-19 09:29:06 UTC
(In reply to Iblis Lin from comment #8)

With this, mosquitto segfaults when I connect via HTTPS:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 802a16000 (LWP 101257/mosquitto)]
0x00000008014bed39 in lws_SHA1 () from /usr/local/lib/libwebsockets.so.9
(gdb) bt
#0  0x00000008014bed39 in lws_SHA1 () from /usr/local/lib/libwebsockets.so.9
#1  0x00000008014ae40b in lws_read () from /usr/local/lib/libwebsockets.so.9
#2  0x00000008014b286e in lws_service_fd_tsi () from /usr/local/lib/libwebsockets.so.9
#3  0x00000008014c0ea2 in lws_plat_service_tsi () from /usr/local/lib/libwebsockets.so.9
#4  0x000000000040c927 in mosquitto_log_printf ()
#5  0x000000000040d6d2 in mosquitto_log_printf ()

I will try to rebuild with debug symbols and dig further later...
Comment 10 Iblis Lin 2017-07-22 08:37:00 UTC
(In reply to stl from comment #9)
Seems the secured websocket is still an isssue from upstream: https://github.com/eclipse/mosquitto/issues/497

Good to get this patch merged?
Comment 11 Iblis Lin 2017-07-22 08:49:43 UTC
I also upgraded mosquitto to 1.4.14 already, here is my personal repo:
https://github.com/iblis17/ports/tree/master/net/mosquitto

I will send it once patch in this PR merged.
Comment 12 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-23 12:11:20 UTC
Appears to be pending confirmation that the patch fixes the issue (contrary to comment 9) and maintainer approval (subsequent to that)
Comment 13 stl 2017-07-24 09:18:23 UTC
(In reply to Kubilay Kocak from comment #12)

The patch does indeed fix the original issue.  Thanks!

The upstream issue referred to in comment 10 is unrelated (I am not using client certificates).

However, the new problem (crashes on HTTPS requests) remains.  I can file a separate PR for that if necessary once I have time to investigate.  It only seems to happen when serving static files and not for the actual MQTT websockets TLS connections, so there's a workaround.
Comment 14 Iblis Lin 2017-07-24 09:51:19 UTC
(In reply to stl from comment #13)

Mind sharing your mosquitto config?
Comment 15 stl 2017-07-24 11:41:17 UTC
(In reply to Iblis Lin from comment #14)
> Mind sharing your mosquitto config?

Sure:

pid_file /var/run/mosquitto.pid
user nobody

port 8883
cafile /usr/local/etc/letsencrypt/live/hostname.example.com/chain.pem
certfile /usr/local/etc/letsencrypt/live/hostname.example.com/cert.pem
keyfile /usr/local/etc/letsencrypt/live/hostname.example.com/privkey.pem

listener 1883
protocol mqtt

listener 8000
protocol websockets
http_dir /home/stl/Web/brewery

listener 8443
protocol websockets
http_dir /home/stl/Web/brewery
cafile /usr/local/etc/letsencrypt/live/hostname.example.com/chain.pem
certfile /usr/local/etc/letsencrypt/live/hostname.example.com/cert.pem
keyfile /usr/local/etc/letsencrypt/live/hostname.example.com/privkey.pem

password_file /usr/local/etc/mosquitto/pwfile
acl_file /usr/local/etc/mosquitto/aclfile
Comment 16 stl 2017-07-24 16:11:43 UTC
Created attachment 184667 [details]
Stack trace, debug info

I could also trigger the segfault with libwebsockets' included server (lwsws) when built with libuv and HTTP2 support (see log), and fetching a static file with Firefox (wget was OK).  (Building libwebsockets with lwsws requires libuv support and automatically switches it on.)

With libuv support disabled, mosquitto still crashes at the same location...
Comment 17 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-25 04:22:14 UTC
Pending maintainer approval and QA as per comment 12

@all Please create a separate issue (or use the mailing lists) for discussion of distinct problems/issues/symptoms.
Comment 18 commit-hook freebsd_committer freebsd_triage 2017-08-18 17:04:00 UTC
A commit references this bug:

Author: swills
Date: Fri Aug 18 17:03:12 UTC 2017
New revision: 448259
URL: https://svnweb.freebsd.org/changeset/ports/448259

Log:
  net/mosquitto: Fails to build with WS option enabled

  PR:		220827
  Submitted by:	stl@koffein.net
  Approved by:	joe@thrallingpenguin.com (maintainer)

Changes:
  head/net/mosquitto/files/patch-lib_mosquitto__internal.h
  head/net/mosquitto/files/patch-src_CMakeLists.txt
Comment 19 Steve Wills freebsd_committer freebsd_triage 2017-08-18 17:04:29 UTC
Committed, thanks!