bug #231940 after application rtmpdump(1) core dumps and dies The patch for OpenSSL-1.1 is broken. The core dump is vomited just before the start of communication even if build is completed and it dies. > /usr/local/bin/rtmpdump -v -r rtmpe://f-radiko.smartstream.ne.jp --app OBC/_definst_ --playpath simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA --live --flv OBC_2018-11-02-14_04.flv > RTMPDump v2.4 > (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL > WARNING: No application or playpath in URL! > Segmentation fault (core dumped)
Note that the original commit (ports rr482833) of bug 231940, was then reverted in ports r482870 due to breaking librtmp consumers. The last commit (ports r482967) then landed and merged. Please make sure you have updated your tree past r482967 and confirm that the issue is still reproducible. Assign to committer of ports r482967 and CC antoine (who merged the last commit)
I have libressl. With or without openssl patch: The command core dumps. If I quoted parts of the command: rtmpdump -v -r "rtmpe://f-radiko.smartstream.ne.jp" --app "OBC/_definst_" --playpath "simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA" --live --flv "OBC_2018-11-02-14_04.flv" I got c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL WARNING: No application or playpath in URL! Failed to open file! OBC_2018-11-02-14_04.flv and no segfault. So I think it is not related to the patch.
(In reply to w.schwarzenfeld from comment #2) libressl does not dump the core. However, if the patch file exists, the build will fail. By removing the patch file added by r482870, The build succeeds in libressl.
Created attachment 199920 [details] Failure script Japan is a service in the domestic streaming service `radiko.jp', which seems to be streaming at the time of capturing core dumps. Results of a failed script run. $ env LANG=C sh -x /usr/local/bin/rec_radiko MBS + pid=65215 + env 'TZ=JST-09' date +%Y-%m-%d-%H_%M + date=2018-12-07-13_16 + playerurl=http://radiko.jp/apps/js/flash/myplayer-release.swf + tmpdir=/tmp + cookiefile=/tmp/cookie.txt + playerfile=/tmp/player.swf + keyfile=/tmp/authkey.png + AUTH1=/tmp/auth1_fms_65215 + AUTH2=/tmp/auth2_fms_65215 + echo MBS + tr '[[a-z]]' '[[A-Z]]' + channel=MBS + [ 1 -eq 1 ] + output=MBS_2018-12-07-13_16.flv + [ ] + [ ! -f /tmp/player.swf ] + /usr/local/bin/wget -q -O /tmp/player.swf http://radiko.jp/apps/js/flash/myplayer-release.swf + [ 0 -ne 0 ] + [ ! -f /tmp/authkey.png ] + /usr/local/bin/swfextract -b 12 /tmp/player.swf -o /tmp/authkey.png + [ ! -f /tmp/authkey.png ] + rm -f /tmp/player.swf + [ -f /tmp/auth1_fms_65215 ] + /usr/local/bin/wget -q '--header=pragma: no-cache' '--header=X-Radiko-App: pc_ts' '--header=X-Radiko-App-Version: 4.0.0' '--header=X-Radiko-User: test-stream' '--header=X-Radiko-Device: pc' '--post-data=\r\n' --no-check-certificate --load-cookies /tmp/cookie.txt --save-headers -O /tmp/auth1_fms_65215 https://radiko.jp/v2/api/auth1_fms + [ 0 -ne 0 ] + /usr/local/bin/perl -ne 'print $1 if(/x-radiko-authtoken: ([\w-]+)/i)' /tmp/auth1_fms_65215 + authtoken=JJXlJFbFfTn4CWkXcW8iIg + /usr/local/bin/perl -ne 'print $1 if(/x-radiko-keyoffset: (\d+)/i)' /tmp/auth1_fms_65215 + offset=105274 + /usr/local/bin/perl -ne 'print $1 if(/x-radiko-keylength: (\d+)/i)' /tmp/auth1_fms_65215 + length=16 + dd 'if=/tmp/authkey.png' 'bs=1' 'skip=105274' 'count=16' + openssl enc -base64 + partialkey='3kf/AFv1pZHAX7ikg8cY6w==' + printf 'authtoken: JJXlJFbFfTn4CWkXcW8iIg \noffset: 105274 length: 16 \npartialkey: 3kf/AFv1pZHAX7ikg8cY6w==' authtoken: JJXlJFbFfTn4CWkXcW8iIg offset: 105274 length: 16 partialkey: 3kf/AFv1pZHAX7ikg8cY6w==+ rm -f /tmp/authkey.png /tmp/auth1_fms_65215 + [ -f /tmp/auth2_fms_65215 ] + /usr/local/bin/wget -q '--header=pragma: no-cache' '--header=X-Radiko-App: pc_ts' '--header=X-Radiko-App-Version: 4.0.0' '--header=X-Radiko-User: test-stream' '--header=X-Radiko-Device: pc' '--header=X-Radiko-AuthToken: JJXlJFbFfTn4CWkXcW8iIg' '--header=X-Radiko-PartialKey: 3kf/AFv1pZHAX7ikg8cY6w==' '--post-data=\r\n' --load-cookies /tmp/cookie.txt --no-check-certificate -O /tmp/auth2_fms_65215 https://radiko.jp/v2/api/auth2_fms + [ 0 -ne 0 -o ! -f /tmp/auth2_fms_65215 ] + echo 'authentication success' authentication success + /usr/local/bin/perl -ne 'print $1 if(/^([^,]+),/i)' /tmp/auth2_fms_65215 + areaid=JP27 + echo 'areaid: JP27' areaid: JP27 + rm -f /tmp/auth2_fms_65215 + [ -f MBS.xml ] + /usr/local/bin/wget -q http://radiko.jp/v2/station/stream/MBS.xml + echo 'cat /url/item[1]/text()' + /usr/local/bin/xmllint --shell MBS.xml + tail -2 + head -1 + stream_url=rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream + echo rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream + /usr/local/bin/perl -pe 's!^(.*)://(.*?)/(.*)/(.*?)$/!$1://$2!' + url_parts0=rtmpe://f-radiko.smartstream.ne.jp + echo rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream + /usr/local/bin/perl -pe 's!^(.*)://(.*?)/(.*)/(.*?)$/!$3!' + url_parts1=MBS/_definst_ + echo rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream + /usr/local/bin/perl -pe 's!^(.*)://(.*?)/(.*)/(.*?)$/!$4!' + url_parts2=simul-stream.stream + rm -f MBS.xml + /usr/local/bin/rtmpdump -v -r rtmpe://f-radiko.smartstream.ne.jp --app MBS/_definst_ --playpath simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:JJXlJFbFfTn4CWkXcW8iIg --live --flv MBS_2018-12-07-13_16.flv RTMPDump v2.4 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL WARNING: No application or playpath in URL! Segmentation fault (core dumped) + /usr/local/bin/wget -q --load-cookies /tmp/cookie.txt --no-check-certificate -O /dev/null https://radiko.jp/ap/member/webapi/member/logout + rm -f /tmp/cookie.txt + exit 0 Take a look at the core dumps in the debugger. $ gdb -c rtmpdump.core /usr/local/bin/rtmpdump GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Core was generated by `/usr/local/bin/rtmpdump -v -r rtmpe://f-radiko.smartstream.ne.jp --app MBS/_defi'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/librtmp.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/librtmp.so.1 Reading symbols from /usr/lib/libssl.so.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.8 Reading symbols from /lib/libcrypto.so.8...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypto.so.8 Reading symbols from /lib/libz.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.6 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000800f27817 in EVP_MD_CTX_init () from /lib/libcrypto.so.8
same here # uname -v FreeBSD 12.0-STABLE r341839 GENERIC # svnliteversion 487508 rtmpdump w/ its debug option (--debug) shows (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL DEBUG: Parsing... DEBUG: Parsed protocol: 2 DEBUG: Parsed host : f-radiko.smartstream.ne.jp WARNING: No application or playpath in URL! DEBUG: Protocol : RTMPE DEBUG: Hostname : f-radiko.smartstream.ne.jp DEBUG: Port : 1935 DEBUG: Playpath : simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA DEBUG: tcUrl : rtmpe://f-radiko.smartstream.ne.jp:1935/OBC/_definst_ DEBUG: app : OBC/_definst_ DEBUG: live : yes DEBUG: timeout : 30 sec DEBUG: Setting buffer time to: 36000000ms Connecting ... DEBUG: RTMP_Connect1, ... connected, handshaking DEBUG: HandShake: Client type: 06 DEBUG: HandShake: DH pubkey position: 48 Segmentation fault (core dumped) and core dumped WITH_DEBUG(only librtmp, rtmpdump couldn't build WITH_DEBUG) say # lldb rtmpdump -c rtmpdump.core (lldb) target create "rtmpdump" --core "rtmpdump.core" Core file '/tmp/rtmpdump.core' (x86_64) was loaded. (lldb) bt * thread #1, name = 'rtmpdump', stop reason = signal SIGSEGV * frame #0: 0x0000000800568bc4 libcrypto.so.111`BN_num_bits(a=0x0000000000000000) at bn_lib.c:137 frame #1: 0x0000000800521ccd libcrypto.so.111`generate_key(dh=0x0000000800adc000) at dh_key.c:85 frame #2: 0x000000080026330f librtmp.so.1`___lldb_unnamed_symbol12$$librtmp.so.1 + 47 frame #3: 0x000000080025993f librtmp.so.1`RTMP_Connect1 + 719 frame #4: 0x000000080025b309 librtmp.so.1`RTMP_Connect + 137 frame #5: 0x0000000000206bcd rtmpdump`___lldb_unnamed_symbol11$$rtmpdump + 5069 frame #6: 0x000000000020411b rtmpdump`___lldb_unnamed_symbol1$$rtmpdump + 283 (lldb) f frame #0: 0x0000000800568bc4 libcrypto.so.111`BN_num_bits(a=0x0000000000000000) at bn_lib.c:137 134 135 int BN_num_bits(const BIGNUM *a) 136 { -> 137 int i = a->top - 1; 138 bn_check_top(a); 139 140 if (BN_is_zero(a))
Created attachment 202807 [details] librtmp-2.4.20151223_4.patch fix: rtmpe protocol Operation Check OpenSSL coredump when using Rtmpe, so we changed the library to GnuTLS.
Mine is built with security/openssl 1.0.2r,1. /usr/local/bin/rtmpdump: librtmp.so.1 => /usr/local/lib/librtmp.so.1 (0x80064b000) libssl.so.9 => /usr/local/lib/libssl.so.9 (0x80066b000) libcrypto.so.9 => /usr/local/lib/libcrypto.so.9 (0x8006e6000) libz.so.6 => /lib/libz.so.6 (0x8008ff000) libc.so.7 => /lib/libc.so.7 (0x800919000) libthr.so.3 => /lib/libthr.so.3 (0x800d0c000) $ rtmpdump -v -r "rtmpe://f-radiko.smartstream.ne.jp" --app "OBC/_definst_" --playpath "simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA" --live --flv "OBC_2018-11-02-14_04.flv" RTMPDump v2.4 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL WARNING: No application or playpath in URL! Connecting ... Segmentation fault (コアダンプ) After removing the patch, I got no coredump. # rm /usr/ports/multimedia/librtmp/files/patch-openssl-1.1 # portmaster -d multimedia/librtmp (same as above rtmpdump command) RTMPDump v2.4 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL WARNING: No application or playpath in URL! Failed to open file! OBC_2018-11-02-14_04.flv no coredumps!
I am reviewing the committed patch for OpenSSL 1.1.x support and fixing this problem now. Stay tuned.
Created attachment 202872 [details] librtmp openssl 1.1.x patch Can anyone try this patch and let me know if it works on your box and target URL?
(In reply to Hiroki Sato from comment #9) I will.
(In reply to Hiroki Sato from comment #9) Submitter's script now works fine for me. My box is ssl=openssl111 @ 12-STABLE. Save the script as rtmptest.sh. If you're in Tokyo, specify FMT instead. $ ./rtmptest.sh FMFUKUOKA authtoken: NnBCvA1u5jSAv59bkTvUWA offset: 9537 length: 16 partialkey: OBkAH3A6fgacF8wb2IBHRQ==authentication success areaid: JP40 RTMPDump v2.4 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL WARNING: No application or playpath in URL! Connecting ... WARNING: Trying different position for server digest! INFO: Connected... Starting Live Stream INFO: Metadata: 149.226 kB / 24.45 sec^C Caught signal: 2, cleaning up, just a second... 150.464 kB / 24.66 sec Just for the record, without your latest patch, it segfault like this: $ ./fail.sh FMFUKUOKA authtoken: fVNW5Y8NGmJDrhdHv0VZWg offset: 182609 length: 16 partialkey: eMqBwSq8ev8An06UY4IGBg==authentication success areaid: JP40 RTMPDump v2.4 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL WARNING: No application or playpath in URL! Connecting ... セグメンテーション違反 (core dumped)
(In reply to Hiroki Sato from comment #9) with your patch, use openssl in base # ldd /usr/local/lib/librtmp.so.1 /usr/local/lib/librtmp.so.1: libssl.so.111 => /usr/lib/libssl.so.111 (0x800685000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e00000) libz.so.6 => /lib/libz.so.6 (0x80071a000) libc.so.7 => /lib/libc.so.7 (0x800248000) libthr.so.3 => /lib/libthr.so.3 (0x800734000) # ldd /usr/local/bin/rtmpdump /usr/local/bin/rtmpdump: librtmp.so.1 => /usr/local/lib/librtmp.so.1 (0x80024d000) libssl.so.111 => /usr/lib/libssl.so.111 (0x80026d000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x800302000) libz.so.6 => /lib/libz.so.6 (0x8005ef000) libc.so.7 => /lib/libc.so.7 (0x800609000) libthr.so.3 => /lib/libthr.so.3 (0x8009fc000) it works fine for me. log w/ debug are below. RTMPDump v2.4 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL DEBUG: Parsing... DEBUG: Parsed protocol: 2 DEBUG: Parsed host : f-radiko.smartstream.ne.jp WARNING: No application or playpath in URL! DEBUG: Protocol : RTMPE DEBUG: Hostname : f-radiko.smartstream.ne.jp DEBUG: Port : 1935 DEBUG: Playpath : simul-stream.stream DEBUG: tcUrl : rtmpe://f-radiko.smartstream.ne.jp:1935/QRR/_definst_ DEBUG: swfUrl : http://radiko.jp/apps/js/flash/myplayer-release.swf DEBUG: app : QRR/_definst_ DEBUG: StopTime : 300000 msec DEBUG: live : yes DEBUG: timeout : 30 sec DEBUG: SWFSHA256: DEBUG: 0b fc dc 73 8f b5 26 d1 9c 08 ab ee 36 61 e8 8f DEBUG: 5b fb 28 4e 44 b2 dc 7c df 42 15 74 09 04 65 db DEBUG: SWFSize : 1253941 DEBUG: Setting buffer time to: 36000000ms Connecting ... DEBUG: RTMP_Connect1, ... connected, handshaking DEBUG: HandShake: Client type: 06 DEBUG: HandShake: DH pubkey position: 48 DEBUG: HandShake: Client digest offset: 994 DEBUG: HandShake: Initial client digest: DEBUG: 70 f9 77 00 01 63 ff a2 96 d9 f5 de 73 6a 14 91 DEBUG: 60 3d 73 71 2f 01 c5 82 60 23 e1 b2 b4 0e ba de DEBUG: HandShake: Type Answer : 06 DEBUG: HandShake: Server Uptime : 6083647 DEBUG: HandShake: FMS Version : 3.0.2.1 (snip)
A commit references this bug: Author: hrs Date: Sun Mar 17 00:54:17 UTC 2019 New revision: 496010 URL: https://svnweb.freebsd.org/changeset/ports/496010 Log: Add a new OpenSSL 1.1.x patchset and revert broken one imported in r482967 which had SIGSEGV issue. PR: 232901 Changes: head/multimedia/librtmp/Makefile head/multimedia/librtmp/distinfo head/multimedia/librtmp/files/patch-librtmp-Makefile head/multimedia/librtmp/files/patch-librtmp-dh.h head/multimedia/librtmp/files/patch-librtmp-handshake.h head/multimedia/librtmp/files/patch-librtmp-hashswf.c head/multimedia/librtmp/files/patch-librtmp-librtmp.pc.in head/multimedia/librtmp/files/patch-openssl-1.1
librtmp-2.4.20151223_4 should fix this problem. If someone still sees SIGSEGV, please let me know or reopen this PR.