Bug 232901 - multimedia/librtmp: OpenSSL 1.1.x patch (r482967) causes segfault
Summary: multimedia/librtmp: OpenSSL 1.1.x patch (r482967) causes segfault
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Hiroki Sato
URL:
Keywords: needs-qa, regression
Depends on:
Blocks:
 
Reported: 2018-11-02 05:44 UTC by takefu
Modified: 2019-03-17 00:59 UTC (History)
7 users (show)

See Also:
bugzilla: maintainer-feedback? (hrs)
koobs: merge-quarterly?


Attachments
Failure script (4.08 KB, text/plain)
2018-12-07 07:07 UTC, takefu
no flags Details
librtmp-2.4.20151223_4.patch (10.91 KB, patch)
2019-03-12 08:29 UTC, takefu
no flags Details | Diff
librtmp openssl 1.1.x patch (22.36 KB, patch)
2019-03-15 05:50 UTC, Hiroki Sato
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description takefu 2018-11-02 05:44:49 UTC
bug #231940 after application rtmpdump(1) core dumps and dies

The patch for OpenSSL-1.1 is broken.
The core dump is vomited just before the start of communication even if build is completed and it dies.


> /usr/local/bin/rtmpdump -v -r rtmpe://f-radiko.smartstream.ne.jp --app OBC/_definst_ --playpath simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA --live --flv OBC_2018-11-02-14_04.flv
> RTMPDump v2.4
> (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
> WARNING: No application or playpath in URL!
> Segmentation fault (core dumped)
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2018-11-02 05:53:17 UTC
Note that the original commit (ports rr482833) of bug 231940, was then reverted in ports r482870 due to breaking librtmp consumers.

The last commit (ports r482967) then landed and merged.

Please make sure you have updated your tree past r482967 and confirm that the issue is still reproducible.

Assign to committer of ports r482967 and CC antoine (who merged the last commit)
Comment 2 w.schwarzenfeld freebsd_triage 2018-11-02 07:09:09 UTC
I have libressl. With or without openssl patch:

The command core dumps.

If I quoted parts of the command:
rtmpdump -v -r "rtmpe://f-radiko.smartstream.ne.jp" --app "OBC/_definst_" --playpath "simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA" --live --flv "OBC_2018-11-02-14_04.flv"

I got
c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: No application or playpath in URL!
Failed to open file! OBC_2018-11-02-14_04.flv

and no segfault.

So I think it is not related to the patch.
Comment 3 takefu 2018-11-02 08:19:38 UTC
(In reply to w.schwarzenfeld from comment #2)

libressl does not dump the core.
However, if the patch file exists, the build will fail.
By removing the patch file added by r482870, The build succeeds in libressl.
Comment 4 takefu 2018-12-07 07:07:26 UTC
Created attachment 199920 [details]
Failure script

Japan is a service in the domestic streaming service `radiko.jp', which seems to be streaming at the time of capturing core dumps.

Results of a failed script run.

$ env LANG=C sh -x /usr/local/bin/rec_radiko MBS
+ pid=65215
+ env 'TZ=JST-09' date +%Y-%m-%d-%H_%M
+ date=2018-12-07-13_16
+ playerurl=http://radiko.jp/apps/js/flash/myplayer-release.swf
+ tmpdir=/tmp
+ cookiefile=/tmp/cookie.txt
+ playerfile=/tmp/player.swf
+ keyfile=/tmp/authkey.png
+ AUTH1=/tmp/auth1_fms_65215
+ AUTH2=/tmp/auth2_fms_65215
+ echo MBS
+ tr '[[a-z]]' '[[A-Z]]'
+ channel=MBS
+ [ 1 -eq 1 ]
+ output=MBS_2018-12-07-13_16.flv
+ [ ]
+ [ ! -f /tmp/player.swf ]
+ /usr/local/bin/wget -q -O /tmp/player.swf http://radiko.jp/apps/js/flash/myplayer-release.swf
+ [ 0 -ne 0 ]
+ [ ! -f /tmp/authkey.png ]
+ /usr/local/bin/swfextract -b 12 /tmp/player.swf -o /tmp/authkey.png
+ [ ! -f /tmp/authkey.png ]
+ rm -f /tmp/player.swf
+ [ -f /tmp/auth1_fms_65215 ]
+ /usr/local/bin/wget -q '--header=pragma: no-cache' '--header=X-Radiko-App: pc_ts' '--header=X-Radiko-App-Version: 4.0.0' '--header=X-Radiko-User: test-stream' '--header=X-Radiko-Device: pc' '--post-data=\r\n' --no-check-certificate --load-cookies /tmp/cookie.txt --save-headers -O /tmp/auth1_fms_65215 https://radiko.jp/v2/api/auth1_fms
+ [ 0 -ne 0 ]
+ /usr/local/bin/perl -ne 'print $1 if(/x-radiko-authtoken: ([\w-]+)/i)' /tmp/auth1_fms_65215
+ authtoken=JJXlJFbFfTn4CWkXcW8iIg
+ /usr/local/bin/perl -ne 'print $1 if(/x-radiko-keyoffset: (\d+)/i)' /tmp/auth1_fms_65215
+ offset=105274
+ /usr/local/bin/perl -ne 'print $1 if(/x-radiko-keylength: (\d+)/i)' /tmp/auth1_fms_65215
+ length=16
+ dd 'if=/tmp/authkey.png' 'bs=1' 'skip=105274' 'count=16'
+ openssl enc -base64
+ partialkey='3kf/AFv1pZHAX7ikg8cY6w=='
+ printf 'authtoken: JJXlJFbFfTn4CWkXcW8iIg \noffset: 105274 length: 16 \npartialkey: 3kf/AFv1pZHAX7ikg8cY6w=='
authtoken: JJXlJFbFfTn4CWkXcW8iIg
offset: 105274 length: 16
partialkey: 3kf/AFv1pZHAX7ikg8cY6w==+ rm -f /tmp/authkey.png /tmp/auth1_fms_65215
+ [ -f /tmp/auth2_fms_65215 ]
+ /usr/local/bin/wget -q '--header=pragma: no-cache' '--header=X-Radiko-App: pc_ts' '--header=X-Radiko-App-Version: 4.0.0' '--header=X-Radiko-User: test-stream' '--header=X-Radiko-Device: pc' '--header=X-Radiko-AuthToken: JJXlJFbFfTn4CWkXcW8iIg' '--header=X-Radiko-PartialKey: 3kf/AFv1pZHAX7ikg8cY6w==' '--post-data=\r\n' --load-cookies /tmp/cookie.txt --no-check-certificate -O /tmp/auth2_fms_65215 https://radiko.jp/v2/api/auth2_fms
+ [ 0 -ne 0 -o ! -f /tmp/auth2_fms_65215 ]
+ echo 'authentication success'
authentication success
+ /usr/local/bin/perl -ne 'print $1 if(/^([^,]+),/i)' /tmp/auth2_fms_65215
+ areaid=JP27
+ echo 'areaid: JP27'
areaid: JP27
+ rm -f /tmp/auth2_fms_65215
+ [ -f MBS.xml ]
+ /usr/local/bin/wget -q http://radiko.jp/v2/station/stream/MBS.xml
+ echo 'cat /url/item[1]/text()'
+ /usr/local/bin/xmllint --shell MBS.xml
+ tail -2
+ head -1
+ stream_url=rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream
+ echo rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream
+ /usr/local/bin/perl -pe 's!^(.*)://(.*?)/(.*)/(.*?)$/!$1://$2!'
+ url_parts0=rtmpe://f-radiko.smartstream.ne.jp
+ echo rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream
+ /usr/local/bin/perl -pe 's!^(.*)://(.*?)/(.*)/(.*?)$/!$3!'
+ url_parts1=MBS/_definst_
+ echo rtmpe://f-radiko.smartstream.ne.jp/MBS/_definst_/simul-stream.stream
+ /usr/local/bin/perl -pe 's!^(.*)://(.*?)/(.*)/(.*?)$/!$4!'
+ url_parts2=simul-stream.stream
+ rm -f MBS.xml
+ /usr/local/bin/rtmpdump -v -r rtmpe://f-radiko.smartstream.ne.jp --app MBS/_definst_ --playpath simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:JJXlJFbFfTn4CWkXcW8iIg --live --flv MBS_2018-12-07-13_16.flv
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: No application or playpath in URL!
Segmentation fault (core dumped)
+ /usr/local/bin/wget -q --load-cookies /tmp/cookie.txt --no-check-certificate -O /dev/null https://radiko.jp/ap/member/webapi/member/logout
+ rm -f /tmp/cookie.txt
+ exit 0

Take a look at the core dumps in the debugger.

$ gdb -c rtmpdump.core /usr/local/bin/rtmpdump
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)...
Core was generated by `/usr/local/bin/rtmpdump -v -r rtmpe://f-radiko.smartstream.ne.jp --app MBS/_defi'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/local/lib/librtmp.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/librtmp.so.1
Reading symbols from /usr/lib/libssl.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssl.so.8
Reading symbols from /lib/libcrypto.so.8...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.8
Reading symbols from /lib/libz.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.6
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0000000800f27817 in EVP_MD_CTX_init () from /lib/libcrypto.so.8
Comment 5 shellingfield 2018-12-15 16:18:13 UTC
same here

# uname -v
FreeBSD 12.0-STABLE r341839 GENERIC 
# svnliteversion 
487508

rtmpdump w/ its debug option (--debug) shows


(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
DEBUG: Parsing...
DEBUG: Parsed protocol: 2
DEBUG: Parsed host    : f-radiko.smartstream.ne.jp
WARNING: No application or playpath in URL!
DEBUG: Protocol : RTMPE
DEBUG: Hostname : f-radiko.smartstream.ne.jp
DEBUG: Port     : 1935
DEBUG: Playpath : simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA
DEBUG: tcUrl    : rtmpe://f-radiko.smartstream.ne.jp:1935/OBC/_definst_
DEBUG: app      : OBC/_definst_
DEBUG: live     : yes
DEBUG: timeout  : 30 sec
DEBUG: Setting buffer time to: 36000000ms
Connecting ...
DEBUG: RTMP_Connect1, ... connected, handshaking
DEBUG: HandShake: Client type: 06
DEBUG: HandShake: DH pubkey position: 48
Segmentation fault (core dumped)


and core dumped WITH_DEBUG(only librtmp, rtmpdump couldn't build WITH_DEBUG) say

# lldb rtmpdump -c rtmpdump.core
(lldb) target create "rtmpdump" --core "rtmpdump.core"
Core file '/tmp/rtmpdump.core' (x86_64) was loaded.
(lldb) bt
* thread #1, name = 'rtmpdump', stop reason = signal SIGSEGV
  * frame #0: 0x0000000800568bc4 libcrypto.so.111`BN_num_bits(a=0x0000000000000000) at bn_lib.c:137
    frame #1: 0x0000000800521ccd libcrypto.so.111`generate_key(dh=0x0000000800adc000) at dh_key.c:85
    frame #2: 0x000000080026330f librtmp.so.1`___lldb_unnamed_symbol12$$librtmp.so.1 + 47
    frame #3: 0x000000080025993f librtmp.so.1`RTMP_Connect1 + 719
    frame #4: 0x000000080025b309 librtmp.so.1`RTMP_Connect + 137
    frame #5: 0x0000000000206bcd rtmpdump`___lldb_unnamed_symbol11$$rtmpdump + 5069
    frame #6: 0x000000000020411b rtmpdump`___lldb_unnamed_symbol1$$rtmpdump + 283
(lldb) f
frame #0: 0x0000000800568bc4 libcrypto.so.111`BN_num_bits(a=0x0000000000000000) at bn_lib.c:137
   134 
   135  int BN_num_bits(const BIGNUM *a)
   136  {
-> 137      int i = a->top - 1;
   138      bn_check_top(a);
   139 
   140      if (BN_is_zero(a))
Comment 6 takefu 2019-03-12 08:29:32 UTC
Created attachment 202807 [details]
librtmp-2.4.20151223_4.patch

fix:
  rtmpe protocol Operation Check

OpenSSL coredump when using Rtmpe, so we changed the library to GnuTLS.
Comment 7 Koichiro Iwao freebsd_committer 2019-03-13 09:33:53 UTC
Mine is built with security/openssl 1.0.2r,1.

/usr/local/bin/rtmpdump:
        librtmp.so.1 => /usr/local/lib/librtmp.so.1 (0x80064b000)
        libssl.so.9 => /usr/local/lib/libssl.so.9 (0x80066b000)
        libcrypto.so.9 => /usr/local/lib/libcrypto.so.9 (0x8006e6000)
        libz.so.6 => /lib/libz.so.6 (0x8008ff000)
        libc.so.7 => /lib/libc.so.7 (0x800919000)
        libthr.so.3 => /lib/libthr.so.3 (0x800d0c000)

$ rtmpdump -v -r "rtmpe://f-radiko.smartstream.ne.jp" --app "OBC/_definst_" --playpath "simul-stream.stream -W http://radiko.jp/apps/js/flash/myplayer-release.swf -C S: -C S: -C S: -C S:2t0YoVV-4fCMPhwpUT8HDA" --live --flv "OBC_2018-11-02-14_04.flv"
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: No application or playpath in URL!
Connecting ...
Segmentation fault (コアダンプ)

After removing the patch, I got no coredump.

# rm /usr/ports/multimedia/librtmp/files/patch-openssl-1.1
# portmaster -d multimedia/librtmp
(same as above rtmpdump command)
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: No application or playpath in URL!
Failed to open file! OBC_2018-11-02-14_04.flv

no coredumps!
Comment 8 Hiroki Sato freebsd_committer 2019-03-13 09:45:04 UTC
I am reviewing the committed patch for OpenSSL 1.1.x support and fixing this problem now.  Stay tuned.
Comment 9 Hiroki Sato freebsd_committer 2019-03-15 05:50:42 UTC
Created attachment 202872 [details]
librtmp openssl 1.1.x patch

Can anyone try this patch and let me know if it works on your box and target URL?
Comment 10 Koichiro Iwao freebsd_committer 2019-03-15 06:09:32 UTC
(In reply to Hiroki Sato from comment #9)
I will.
Comment 11 Koichiro Iwao freebsd_committer 2019-03-15 07:23:25 UTC
(In reply to Hiroki Sato from comment #9)

Submitter's script now works fine for me. My box is ssl=openssl111 @ 12-STABLE.
Save the script as rtmptest.sh. If you're in Tokyo, specify FMT instead.

$ ./rtmptest.sh FMFUKUOKA
authtoken: NnBCvA1u5jSAv59bkTvUWA
offset: 9537 length: 16
partialkey: OBkAH3A6fgacF8wb2IBHRQ==authentication success
areaid: JP40
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: No application or playpath in URL!
Connecting ...
WARNING: Trying different position for server digest!
INFO: Connected...
Starting Live Stream
INFO: Metadata:
149.226 kB / 24.45 sec^C
Caught signal: 2, cleaning up, just a second...
150.464 kB / 24.66 sec


Just for the record, without your latest patch, it segfault like this:
$ ./fail.sh FMFUKUOKA
authtoken: fVNW5Y8NGmJDrhdHv0VZWg
offset: 182609 length: 16
partialkey: eMqBwSq8ev8An06UY4IGBg==authentication success
areaid: JP40
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: No application or playpath in URL!
Connecting ...
セグメンテーション違反 (core dumped)
Comment 12 shellingfield 2019-03-15 08:26:58 UTC
(In reply to Hiroki Sato from comment #9)

with your patch, use openssl in base 

# ldd /usr/local/lib/librtmp.so.1
/usr/local/lib/librtmp.so.1:
        libssl.so.111 => /usr/lib/libssl.so.111 (0x800685000)
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e00000)
        libz.so.6 => /lib/libz.so.6 (0x80071a000)
        libc.so.7 => /lib/libc.so.7 (0x800248000)
        libthr.so.3 => /lib/libthr.so.3 (0x800734000)
# ldd /usr/local/bin/rtmpdump 
/usr/local/bin/rtmpdump:
        librtmp.so.1 => /usr/local/lib/librtmp.so.1 (0x80024d000)
        libssl.so.111 => /usr/lib/libssl.so.111 (0x80026d000)
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x800302000)
        libz.so.6 => /lib/libz.so.6 (0x8005ef000)
        libc.so.7 => /lib/libc.so.7 (0x800609000)
        libthr.so.3 => /lib/libthr.so.3 (0x8009fc000)

it works fine for me.

log w/ debug are below.

RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
DEBUG: Parsing...
DEBUG: Parsed protocol: 2
DEBUG: Parsed host    : f-radiko.smartstream.ne.jp
WARNING: No application or playpath in URL!
DEBUG: Protocol : RTMPE
DEBUG: Hostname : f-radiko.smartstream.ne.jp
DEBUG: Port     : 1935
DEBUG: Playpath : simul-stream.stream
DEBUG: tcUrl    : rtmpe://f-radiko.smartstream.ne.jp:1935/QRR/_definst_
DEBUG: swfUrl   : http://radiko.jp/apps/js/flash/myplayer-release.swf
DEBUG: app      : QRR/_definst_
DEBUG: StopTime      : 300000 msec
DEBUG: live     : yes
DEBUG: timeout  : 30 sec
DEBUG: SWFSHA256:
DEBUG: 0b fc dc 73 8f b5 26 d1 9c 08 ab ee 36 61 e8 8f
DEBUG: 5b fb 28 4e 44 b2 dc 7c df 42 15 74 09 04 65 db
DEBUG: SWFSize  : 1253941
DEBUG: Setting buffer time to: 36000000ms
Connecting ...
DEBUG: RTMP_Connect1, ... connected, handshaking
DEBUG: HandShake: Client type: 06
DEBUG: HandShake: DH pubkey position: 48
DEBUG: HandShake: Client digest offset: 994
DEBUG: HandShake: Initial client digest: 
DEBUG: 70 f9 77 00 01 63 ff a2 96 d9 f5 de 73 6a 14 91
DEBUG: 60 3d 73 71 2f 01 c5 82 60 23 e1 b2 b4 0e ba de
DEBUG: HandShake: Type Answer   : 06
DEBUG: HandShake: Server Uptime : 6083647
DEBUG: HandShake: FMS Version   : 3.0.2.1

(snip)
Comment 13 commit-hook freebsd_committer 2019-03-17 00:54:40 UTC
A commit references this bug:

Author: hrs
Date: Sun Mar 17 00:54:17 UTC 2019
New revision: 496010
URL: https://svnweb.freebsd.org/changeset/ports/496010

Log:
  Add a new OpenSSL 1.1.x patchset and revert broken one imported
  in r482967 which had SIGSEGV issue.

  PR:	232901

Changes:
  head/multimedia/librtmp/Makefile
  head/multimedia/librtmp/distinfo
  head/multimedia/librtmp/files/patch-librtmp-Makefile
  head/multimedia/librtmp/files/patch-librtmp-dh.h
  head/multimedia/librtmp/files/patch-librtmp-handshake.h
  head/multimedia/librtmp/files/patch-librtmp-hashswf.c
  head/multimedia/librtmp/files/patch-librtmp-librtmp.pc.in
  head/multimedia/librtmp/files/patch-openssl-1.1
Comment 14 Hiroki Sato freebsd_committer 2019-03-17 00:59:13 UTC
librtmp-2.4.20151223_4 should fix this problem.  If someone still sees SIGSEGV, please let me know or reopen this PR.