Bug 196667 - www/firefox crashes often when loading multiple multimedia/gstreamer1 content once again (e.g. HTML 5 YouTube)
Summary: www/firefox crashes often when loading multiple multimedia/gstreamer1 content...
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-gecko (Nobody)
URL:
Keywords:
Depends on: 196051
Blocks:
  Show dependency treegraph
 
Reported: 2015-01-13 03:26 UTC by jakub_lach
Modified: 2016-03-04 18:05 UTC (History)
3 users (show)

See Also:


Attachments
Firefox backtrace (143.07 KB, text/plain)
2015-01-13 03:26 UTC, jakub_lach
no flags Details
additional coredump when exiting DEBUG www/firefox (6.61 KB, text/plain)
2015-01-19 22:29 UTC, jakub_lach
no flags Details
alsa related crash bt (2.22 KB, text/plain)
2015-01-20 00:25 UTC, jakub_lach
no flags Details
full alsa crash with gstreamer (compiled with debug) bits (497.85 KB, text/plain)
2015-01-20 00:42 UTC, jakub_lach
no flags Details
the same gst bit but diffrent dump (155.77 KB, text/plain)
2015-01-20 00:53 UTC, jakub_lach
no flags Details
last one (184.47 KB, text/plain)
2015-01-20 11:00 UTC, jakub_lach
no flags Details
similar crash to the first one, only this time with full debug (144.35 KB, text/plain)
2015-01-20 16:54 UTC, jakub_lach
no flags Details
gst and libav/h264 this time (215.25 KB, text/plain)
2015-01-20 23:06 UTC, jakub_lach
no flags Details
full bt, reproduced the crash with Firefox 35.0.1,1 and gstreamer1 -O0 -g (139.26 KB, text/plain)
2015-02-02 00:43 UTC, jakub_lach
no flags Details
similar to above, but through cubeb_alsa.c (signal 6) (277.51 KB, text/plain)
2015-02-02 10:01 UTC, jakub_lach
no flags Details
additional full bt of coredump on exit if www/firefox is build WITH_DEBUG set (71.16 KB, text/plain)
2015-02-02 10:18 UTC, jakub_lach
no flags Details
another cubeb_alsa.c (signal 6), this time without loaded VAAPI, setup is the same as with 152459 (339.24 KB, text/plain)
2015-02-03 10:06 UTC, jakub_lach
no flags Details
after cubeb alsa patch (314.76 KB, text/plain)
2015-02-06 23:48 UTC, jakub_lach
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description jakub_lach 2015-01-13 03:26:05 UTC
Created attachment 151519 [details]
Firefox backtrace

The crashes are somewhat random, however loading lots of videos at once are surefire way to induce it (especially if restarting a previous session, and you uncheck "Don't load tabs until selected").

I'm 99% sure gstreamer is at fault once again (see attached backtrace).

Console output:

<...>

++DOMWINDOW == 39 (0x8226a0e80) [pid = 69592] [serial = 57] [outer = 0x8227ba600]
++DOCSHELL 0x82421b200 == 17 [pid = 69592] [id = 23]
++DOMWINDOW == 40 (0x823927980) [pid = 69592] [serial = 58] [outer = 0x0]
++DOMWINDOW == 41 (0x823927d00) [pid = 69592] [serial = 59] [outer = 0x823927980]
++DOMWINDOW == 42 (0x8249c8080) [pid = 69592] [serial = 60] [outer = 0x823927980]
libva info: VA-API version 0.37.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/local/lib/va/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_35
libva info: va_openDriver() returns 0
[69592] ###!!! ABORT: This Image should have already allocated data: 'mTextureClient', file /usr/obj/usr/ports/www/firefox/work/mozilla-release/gfx/layers/ipc/SharedPlanarYCbCrIma
std::__1::__tree<std::__1::__value_type<ogg_packet*, long>, std::__1::__map_value_compare<ogg_packet*, std::__1::__value_type<ogg_packet*, long>, std::__1::less<ogg_packet*>, true
std::__1::__tree<std::__1::__value_type<ogg_packet*, long>, std::__1::__map_value_compare<ogg_packet*, std::__1::__value_type<ogg_packet*, long>, std::__1::less<ogg_packet*>, true
__float128+0x002C5056 [/usr/local/lib/firefox/libxul.so +0x026CF7F6]
__float128+0x002D0008 [/usr/local/lib/firefox/libxul.so +0x026DA7A8]
__float128+0x002CE613 [/usr/local/lib/firefox/libxul.so +0x026D8DB3]
__float128+0x002F7907 [/usr/local/lib/firefox/libxul.so +0x027020A7]
__float128+0x002E7E33 [/usr/local/lib/firefox/libxul.so +0x026F25D3]
XRE_AddJarManifestLocation+0x0000F460 [/usr/local/lib/firefox/libxul.so +0x0094C690]
XRE_AddJarManifestLocation+0x0000F54D [/usr/local/lib/firefox/libxul.so +0x0094C77D]
XRE_AddJarManifestLocation+0x0000C877 [/usr/local/lib/firefox/libxul.so +0x00949AA7]
NS_InvokeByIndex+0x0001CDA3 [/usr/local/lib/firefox/libxul.so +0x00975553]
_ZNSt3__16vectorINS_4pairIiiEENS_9allocatorIS2_EEE21__push_back_slow_pathIS2_EEvOT_+0x0000ED1C [/usr/local/lib/firefox/libxul.so +0x00CE039C]
std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<c
XRE_AddJarManifestLocation+0x0000B36E [/usr/local/lib/firefox/libxul.so +0x0094859E]
PR_GetThreadName+0x000001C9 [/usr/local/lib/libplds4.so.1 +0x0001AA19]
operator->+0x000007B5 [/lib/libthr.so.3 +0x00009585]
UNKNOWN 0x0
[69592] ###!!! ABORT: This Image should have already allocated data: 'mTextureClient', file /usr/obj/usr/ports/www/firefox/work/mozilla-release/gfx/layers/ipc/SharedPlanarYCbCrIma
Hit MOZ_CRASH() at /usr/obj/usr/ports/www/firefox/work/mozilla-release/memory/mozalloc/mozalloc_abort.cpp:30
Segmentation fault (core dumped)
Comment 1 Bugzilla Automation freebsd_committer freebsd_triage 2015-01-13 03:26:05 UTC
Maintainers CC'd
Comment 2 Ivan Klymenko 2015-01-19 13:45:13 UTC
...
(process:95352): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...[New Thread 818fdf000 (LWP 101374/StreamTrans #2)]
[New Thread 81c78d000 (LWP 101377/DOM Worker)]
[New Thread 81c43b000 (LWP 101413/DOM Worker)]

** (firefox:95352): CRITICAL **: gst_app_src_set_size: assertion 'GST_IS_APP_SRC (appsrc)' failed
[New Thread 8642c5400 (LWP 101656/firefox)]

** (firefox:95352): CRITICAL **: gst_app_src_set_size: assertion 'GST_IS_APP_SRC (appsrc)' failed
[New Thread 86cd2e400 (LWP 101659/firefox)]

** (firefox:95352): CRITICAL **: gst_app_src_set_size: assertion 'GST_IS_APP_SRC (appsrc)' failed

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 801902400 (LWP 101264/firefox)]
0x0000000803cfa1b1 in std::__1::__tree<std::__1::__value_type<ogg_packet*, long>, std::__1::__map_value_compare<ogg_packet*, std::__1::__value_type<ogg_packet*, long>, std::__1::less<ogg_packet*>, true>, std::__1::allocator<std::__1::__value_type<ogg_packet*, long> > >::destroy
    () from /usr/local/lib/firefox/libxul.so
(gdb) bt full
#0  0x0000000803cfa1b1 in std::__1::__tree<std::__1::__value_type<ogg_packet*, long>, std::__1::__map_value_compare<ogg_packet*, std::__1::__value_type<ogg_packet*, long>, std::__1::less<ogg_packet*>, true>, std::__1::allocator<std::__1::__value_type<ogg_packet*, long> > >::destroy () from /usr/local/lib/firefox/libxul.so
No symbol table info available.
#1  0x00000008207f8880 in ?? ()
No symbol table info available.
#2  0x00000008446c4800 in ?? ()
No symbol table info available.
#3  0x000000081e595180 in ?? ()
No symbol table info available.
#4  0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) q
The program is running.  Exit anyway? (y or n) y
Comment 3 jakub_lach 2015-01-19 22:29:02 UTC
Created attachment 151864 [details]
additional coredump when exiting DEBUG www/firefox
Comment 4 jakub_lach 2015-01-20 00:25:33 UTC
Created attachment 151869 [details]
alsa related crash bt
Comment 5 jakub_lach 2015-01-20 00:42:20 UTC
Created attachment 151871 [details]
full alsa crash with gstreamer (compiled with debug) bits
Comment 6 jakub_lach 2015-01-20 00:53:30 UTC
Created attachment 151872 [details]
the same gst bit but diffrent dump

#4  0x000000083f2183e9 in ?? () from /usr/local/lib/libgstapp-1.0.so.0
#5  0x000000083f43e62e in gst_base_sink_chain_unlocked (basesink=<optimized out>, pad=<optimized out>, obj=<optimized out>, is_list=<optimized out>) at gstbasesink.c:3430
#6  0x000000083f43dd94 in gst_base_sink_chain_main (basesink=0x820d15920, pad=<optimized out>, obj=0x85896f8e0, is_list=0) at gstbasesink.c:3538
#7  0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x83d152510, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#8  0x000000083ef77880 in gst_pad_push_data (pad=0x840576580, type=4112, data=<optimized out>) at gstpad.c:4063
#9  0x000000083f45150d in gst_base_transform_chain (pad=<optimized out>, parent=0x839b16a70, buffer=<optimized out>) at gstbasetransform.c:2260
#10 0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x83d152970, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#11 0x000000083ef77880 in gst_pad_push_data (pad=0x84a1e84b0, type=4112, data=<optimized out>) at gstpad.c:4063
#12 0x000000083ef63947 in gst_proxy_pad_chain_default (pad=<optimized out>, parent=<optimized out>, buffer=0x85896f8e0) at gstghostpad.c:126
#13 0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x8586f75f0, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#14 0x000000083ef77880 in gst_pad_push_data (pad=0x8586f7d10, type=4112, data=<optimized out>) at gstpad.c:4063
#15 0x000000083ef63947 in gst_proxy_pad_chain_default (pad=<optimized out>, parent=<optimized out>, buffer=0x85896f8e0) at gstghostpad.c:126
#16 0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x8463ad260, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#17 0x000000083ef77880 in gst_pad_push_data (pad=0x839caa0a0, type=4112, data=<optimized out>) at gstpad.c:4063
#18 0x000000083f45150d in gst_base_transform_chain (pad=<optimized out>, parent=0x821b9a4d0, buffer=<optimized out>) at gstbasetransform.c:2260
#19 0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x84ca27d10, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#20 0x000000083ef77880 in gst_pad_push_data (pad=0x84ca27ae0, type=4112, data=<optimized out>) at gstpad.c:4063
#21 0x000000083f45150d in gst_base_transform_chain (pad=<optimized out>, parent=0x821b99dd0, buffer=<optimized out>) at gstbasetransform.c:2260
#22 0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x84ca278b0, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#23 0x000000083ef77880 in gst_pad_push_data (pad=0x8549eb8f0, type=4112, data=<optimized out>) at gstpad.c:4063
#24 0x000000083ef63947 in gst_proxy_pad_chain_default (pad=<optimized out>, parent=<optimized out>, buffer=0x85896f8e0) at gstghostpad.c:126
#25 0x000000083ef77022 in gst_pad_chain_data_unchecked (pad=0x8568e5420, type=<optimized out>, data=<optimized out>) at gstpad.c:3830
#26 0x000000083ef77880 in gst_pad_push_data (pad=0x8405769e0, type=4112, data=<optimized out>) at gstpad.c:4063
#27 0x000000084d82c74e in gst_queue_push_one (queue=<optimized out>) at gstqueue.c:1169
#28 gst_queue_loop (pad=<optimized out>) at gstqueue.c:1298
#29 0x000000083efa2871 in gst_task_func (task=0x847831830) at gsttask.c:316
Comment 7 jakub_lach 2015-01-20 11:00:22 UTC
Created attachment 151897 [details]
last one

I think this is the furthest I will get here.

Recompiled all of gstreamer1* with -O -g.
Comment 8 jakub_lach 2015-01-20 16:54:18 UTC
Created attachment 151915 [details]
similar crash to the first one, only this time with full debug

full bt, gstreamer1 with debug, similar to the first one which I couldn't replicate with debug, gst bits are the same in all of them however, all crashes differ only in initial backtrace.
Comment 9 jakub_lach 2015-01-20 23:06:19 UTC
Created attachment 151934 [details]
gst and libav/h264 this time
Comment 10 jakub_lach 2015-02-02 00:43:30 UTC
Created attachment 152459 [details]
full bt, reproduced the crash with Firefox 35.0.1,1 and gstreamer1 -O0 -g

FreeBSD 10.1-STABLE #0 r277882 amd64 (FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512)
Comment 11 jakub_lach 2015-02-02 10:01:25 UTC
Created attachment 152471 [details]
similar to above, but through cubeb_alsa.c (signal 6)
Comment 12 jakub_lach 2015-02-02 10:18:31 UTC
Created attachment 152474 [details]
additional full bt of coredump on exit if www/firefox is build WITH_DEBUG set
Comment 13 Jan Beich freebsd_committer freebsd_triage 2015-02-02 14:26:14 UTC
(In reply to jakub_lach from comment #0)
> libva info: VA-API version 0.37.0
> libva info: va_getDriverName() returns 0
> libva info: Trying to open /usr/local/lib/va/i965_drv_video.so
> libva info: Found init function __vaDriverInit_0_35
> libva info: va_openDriver() returns 0

I wonder if GStreamer automatically adds VA-API decoder. Can you check with libva uninstalled? And whether non-DEBUG build also uses libva e.g.,

  $ procstat -v $(pgrep firefox) | fgrep libva
Comment 14 jakub_lach 2015-02-02 16:35:43 UTC
I have installed gstreamer1-vaapi manually, in attempt to have hardware decoding, but... it only works with 

vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD

and I didn't got it to work at all.

As far as I remember it doesn't affect anything at all (that's the first thing I uninstall if there is something fishy with gstreamer, jut in case), I didn't recompile it with a debug and libva is nowhere to be seen in bt?
Comment 15 jakub_lach 2015-02-02 16:42:25 UTC
Just to be precise

$ pkg info | grep 'gstreamer'                                           
gstreamer1-1.4.5               Media applications framework
gstreamer1-libav-1.4.5         GStreamer plug-in with many audio/video 
gstreamer1-plugins-1.4.5_1     GStreamer written collection 
gstreamer1-plugins-good-1.4.5  Good gstreamer-plugins
gstreamer1-vaapi-0.5.9         GStreamer hardware video decoding via VA-API 

As I've said, vaapi is superfluous, rest is recompiled with debug.

I wonder about different thing- somebody on gstreamer1 bugzilla have said that that new "libav" is by accident quite assuming about dependencies- and in case of them missing crashy.

Due to that, some of the gstreamer1 crashes there were "fixed" by installing all plugins blindly, I find it quite embarrassing.
Comment 16 jakub_lach 2015-02-03 10:06:55 UTC
Created attachment 152509 [details]
another cubeb_alsa.c (signal 6), this time without loaded VAAPI, setup is the same as with 152459

I've replicated core dump while VA-API plugin was unloaded (deinstalled). 

Here, Firefox is built manually with -O0 -g, the WITH_DEBUG is _not_ set, as it causes verbose writing to the console, and using webrowser is unbearably slow, however, as you can see, some optimizations slip through that way.

This is another cubeb_alsa.c (signal 6) assertion crash.
Comment 17 jakub_lach 2015-02-04 00:15:15 UTC
Just FYI, I've tried loading sem module too (just in case), though I got the same crash as last on nonetheless... cubeb_alsa.c/assertion
Comment 18 Jan Beich freebsd_committer freebsd_triage 2015-02-04 08:51:41 UTC
Do you have a rough guess when the issue started to happen?
Any changes in your environment just before?
Can you reproduce with media.gstreamer.enabled -> false in about:config ?
Can you reproduce with www/firefox-nightly (in freebsd-gecko repo)?
Can you reproduce inside poudriere jail with official packages?
Can you reproduce inside poudriere jail with parts of your environment migrated within?
Comment 19 jakub_lach 2015-02-04 12:47:49 UTC
(In reply to Jan Beich from comment #18)

>Do you have a rough guess when the issue started to happen?
>Any changes in your environment just before?

Unfortunately no. I've tested it now with clean, default profile
though. Firefox was always crashy with YouTube... but I think this
one is post 30.

>Can you reproduce with media.gstreamer.enabled -> false in about:config ?

Yes, albeit crash bt does not reference gstreamer then (only cubeb).

I'm surprised that HTML5 works this way- looks like this toggle is not what I've
thought.

>Can you reproduce with www/firefox-nightly (in freebsd-gecko repo)?

Trying to built nightly package now.

>Can you reproduce inside poudriere jail with official packages?

I have reproduced it with official 10-STABLE default packages. Obviously without debug.

>Can you reproduce inside poudriere jail with parts of your environment migrated within?

I don't have any experience with poudriere/jail setup, maybe later... 

Incidentally, all recent test were with sem module loaded. Does that mean I can rest the alsa-plugins case?
Comment 20 Jan Beich freebsd_committer freebsd_triage 2015-02-04 16:35:08 UTC
(In reply to jakub_lach from comment #19)
> (In reply to Jan Beich from comment #18)
>
>>Do you have a rough guess when the issue started to happen?
>>Any changes in your environment just before?
>
> Unfortunately no. I've tested it now with clean, default profile
> though. Firefox was always crashy with YouTube... but I think this
> one is post 30.

That's roughly when audio/alsa-* update to 1.0.28 happened. Maybe try
downgrading.

>>Can you reproduce inside poudriere jail with official packages?
>
> I have reproduced it with official 10-STABLE default packages. Obviously
> without debug.

Including alsa-* and gstreamer1* packages ? Testing only firefox package
wouldn't help if the issue is with a port option or make.conf setting
for one of its dependencies.

>
>>Can you reproduce inside poudriere jail with parts of your
>> environment migrated within?
>
> I don't have any experience with poudriere/jail setup, maybe later...

Any clean environment would do.

> Incidentally, all recent test were with sem module loaded.

sem(4) is only needed for very old libthr/libc, before 9.0-RELEASE.
At least multimedia/libvpx and security/nss depend on POSIX semaphores.
www/firefox itself used to require them for python's multiprocessing
during build but not anymore.

> Does that mean I can rest the alsa-plugins case?

Not yet because of the following

>
>>Can you reproduce with media.gstreamer.enabled -> false in about:config ?
>
> Yes, albeit crash bt does not reference gstreamer then (only cubeb).

I guess, bug 196051 needs to be FIXED first to figure out the rest.
Comment 21 jakub_lach 2015-02-04 18:29:24 UTC
Hello again, 

I finally got firefox-nightly to build- it is quite resilient to workload, and it didn't crash for quite some time... but ultimately it did too. However, due to e10s, the crash dump is produced by plugin-container, Firefox doesn't exit, albeit _all_ tabs opened with videos playing display "this tab has crashed".

Unfortunately, all debug packages of firefox-nightly I've built so far fail to start (they crash upon it instantly, have bts but that's another thing clearly), so I cannot confirm for sure it's the same issue.

I understand the value of clean environment, however I already gave it a substantial chunk of my free time, this is where I was coming from with my "maybe later" comment.

Speaking of time, thank you for time spent answering me! I hope my clueless poking with this issue didn't irritate you too much :) 

I will write again after I setup clean environment/downgrade alsa. 

For now, my hopes for relatively crash-free surfing are in e10s.
Comment 22 jakub_lach 2015-02-06 23:48:34 UTC
Created attachment 152644 [details]
after cubeb alsa patch
Comment 23 jakub_lach 2015-02-08 11:27:45 UTC
Atfer the cubeb fix, firefox-nightly doesn't crash for me at all. Spent last two days looping playlists on YouTube without single core dump.

There are rewinding/loosing audio stream/stopping playback issues, but nothing making it to core dump. 

That makes me think, I was wrong assuming (without debug) that it crashed the same way as www/firefox. There were probably the same crashes as cubeb ones (now fixed). 

In fact, that would explain why it would be crash resilient- it only crashed a half of the time compared to a regular Firefox.
Comment 24 jakub_lach 2015-03-04 14:55:28 UTC
Hello!

Some more testing pending, but looks like it's fixed in 36 as well as nightlies.
Comment 25 jakub_lach 2015-03-06 09:21:06 UTC
Actually, I take that back. Only 38 and (37?) nightly was stable for me.

39 and 36 are both crashy on YouTube playback.
Comment 26 jakub_lach 2015-03-14 13:11:23 UTC
FWIW, while 36 from ports and 39 from gecko repository at r1824 are both crashy (firefox-nightly-39.0.232571,1), r1818 is _very_ usable for me (firefox-nightly-38.0.230347,1 but in "About" 39.0a1 (2015-03-10)), it is a very marked difference, those previously mentioned are bordering on unusable in the long run, this one I would say is pretty stable for me. 

Was there any sizable difference regarding media playback between those two?
Comment 27 jakub_lach 2015-09-29 21:17:17 UTC
Not a single crash since multimedia/gstreamer1-libav update to 1.6.0 (dependency on ffmpeg in port).
Comment 28 Jan Beich freebsd_committer freebsd_triage 2016-03-04 16:59:18 UTC
Can you still reproduce?

www/firefox no longer supports gstreamer since ports r404688. It now uses lazy bindings for ffmpeg 0.8-3.0 (2.9, 3.0 since firefox 45). Hardware decoding is not supported yet[1].

www/firefox-esr will follow later as esr45, once www/firefox moves from 45.0 to 46.0 release.

Also, firefox-nightly enables e10s by default which maybe crashy due to its own bugs or missing functionality in BSDs e.g., base r296162.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1210726 (see Depends on)
Comment 29 jakub_lach 2016-03-04 17:40:22 UTC
(In reply to Jan Beich from comment #28)

You are correct, no more crashes ever since.

Speaking of e10s, I've tested it quite a while ago, but had rather bad experience (blank pages and various incompatibilities), I dread the moment it will be enabled by default.