Bug 255348

Summary: multimedia/pipewire: wip general pr
Product: Ports & Packages Reporter: Evgeniy Khramtsov <evgeniy>
Component: Individual Port(s)Assignee: Gleb Popov <arrowd>
Status: Open ---    
Severity: Affects Some People CC: evgeniy
Priority: --- Keywords: patch
Version: LatestFlags: arrowd: maintainer-feedback+
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=245321
Bug Depends on: 245321    
Bug Blocks:    
Description Flags
known good config for pw-play
screencapture-ootb-v1 (use "git am") none

Description Evgeniy Khramtsov 2021-04-23 16:11:57 UTC
Created attachment 224380 [details]

There are patches to build alsa and media session. I plan on sending them upstream this week. This depends on 245321, media session requires alsa-lib >=1.1.7. Media session also has flatpak/xdg-portal related code, this might be useful for these.

PipeWire clients e.g. pw-cat pw-play can play sound via alsa backend with edited pipewire.conf to include alsa static sink adapter pointing to "pcm.default"  (see asound.conf & posted known working config next). Pulse clients do not work yet because the pipewire-pulse daemon gets wrong audio formats from pipewire.

PIPEWIRE_DEBUG=5 is too verbose, the only catch for now is pipewire-pulse reports sink-name "null", but it seems that "null" means to use the default sink from pipewire; pactl confirms that the default sink is used. I plan to continue debugging this next weekend, the PR is here to make sure that the progress does not block on me.

If sound does not work after configuration change that was faulty, make sure that ~/.config/pipewire is rm'ed. PipeWire restore-session wasted a lot of my time before this was found out.

If one wants to build jack, then version >=1.9.10 is needed (bug 251125).

SDL2 is disabled because it is needed only for tests and gets enabled if devel/sdl20 is installed. If one wants to build it anyway, then make sure to add localbase to USES.

TODO: find out where the LD_PRELOAD libs are and/or how to build them, check if static alsa source (microphone) can be configured, xdg-desktop-portal stuff, OSS backend.

FreeBSD 14 poudriere bulk && poudriere testport w/ all options sets: OK.
Comment 1 Evgeniy Khramtsov 2021-04-23 16:15:47 UTC
Created attachment 224382 [details]
known good config for pw-play
Comment 2 Gleb Popov freebsd_committer 2021-04-23 16:21:04 UTC
It is great that you're working on this!

I'm not using PipeWire for myself, so I can't test your changes, but the patch seems fine overall.

Please, make sure to submit these changes upstream, it is pretty responsive. Also note that GitHub mirror [1] has Cirrus CI set up for testing FreeBSD builds (which is failing ATM). It'd be awesome if you fix it up, enable alsa and make sure it passes.

I'm a bit reluctant to push patches to the Ports repo and usually wait for a release that contains these changes. However, if you wish, I can commit it right away. Let me know what you prefer.

And thanks again for working on this.

[1]: https://github.com/PipeWire/pipewire
Comment 3 Evgeniy Khramtsov 2021-04-23 16:27:37 UTC
(In reply to Gleb Popov from comment #2)


> Let me know what you prefer.

The latter (wait for a release that contains these changes).

> make sure to submit these changes upstream


> fix it up, enable alsa and make sure it passes.

I'll check this next week.
Comment 4 Gleb Popov freebsd_committer 2021-04-23 16:34:03 UTC
(In reply to Evgeniy Khramtsov from comment #3)

> The latter (wait for a release that contains these changes).

Great that we're agree on this. Feel free to get me on the hook when submitting a GitLab merge request.
Comment 5 Evgeniy Khramtsov 2021-04-23 20:21:25 UTC
(In reply to Gleb Popov from comment #4)
I've just built the master branch: https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests/632
Comment 6 Gleb Popov freebsd_committer 2021-04-24 17:51:17 UTC
Nice. Can you also please fix GitHub CI? It should be as easy as adding gettext stuff to .cirrus.yml file.

It'd be also great if you enable alsa for CI.
Comment 7 Evgeniy Khramtsov 2021-04-24 19:39:33 UTC
(In reply to Gleb Popov from comment #6)

Sure, this is planned for the next week, currently away from the FreeBSD machine...

I’ll see next week if CI can build the new alsa before building the pipe. If it can’t, then the new alsa-lib has to hit the ports tree first before the CI changes.

I’ll add the alsa bug to the depends of this bug.
Comment 8 Evgeniy Khramtsov 2021-04-26 14:41:43 UTC
(In reply to Gleb Popov from comment #6)

See https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests/637

To build with alsa, the CI has to build the new alsa-lib locally (waiting for 245321 to land into ports would be too long). I'll add alsa to the CI tomorrow.
Comment 9 Gleb Popov freebsd_committer 2021-04-26 14:55:20 UTC
(In reply to Evgeniy Khramtsov from comment #8)

I'd rather wait for alsa update to be committed, but if you have spare time - why not?
Comment 10 Evgeniy Khramtsov 2021-04-27 17:15:36 UTC
(In reply to Gleb Popov from comment #9)

See: https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests/638

Building alsa-lib locally is PITA, even with 'make -d x' commands from the port build pasted directly.

I resolved this via minimal ports tree (e.g. audio/alsa-lib only and several Mk related dirs) in the compressed archive.

I'm kinda slow to reply/open pages/commit, nat64.net is overloaded for some reason today :D
Comment 11 Evgeniy Khramtsov 2021-05-02 20:22:46 UTC
Current progress:

1. Commenting 'restore-stream' in configuration files results in much improved debug information, and also gets rid of cached configuration and negotiations in ~/.config/pipewire.

2. ALSA OSS plugin abuse does not work with pipewire-pulse due to the default configured values (ex. default.clock.rate). Configuring audio.format for alsa sinks breaks format negotiation (audioadapter.c negotiate_format()) the same way for pw-play as with pipewire-pulse. The same applies to pipewire alsa, it gets configured by default, so aplay via pipewire does not work due to defaults.

3. Doxygen documentation seems to be misleading on format negotiation because it focuses on streams.

4. Continuing to abuse pcm.plug:oss is the wrong way, because PipeWire ALSA SPA plugin inserts a converter after format negotiation, which is an another can of worms if we abuse the oss plugin in alsa. No further progress in ALSA OSS abuse anymore.

To use pipewire as a pulseaudio replacement, one should write an OSS SPA plugin.

And also, this patch misses alsa-plugins, which has the required library for ALSA OSS abuse... This doesn't matter, because alsa-plugins is not needed for media session, so the patch is not updated.
Comment 12 Evgeniy Khramtsov 2021-05-17 15:03:10 UTC
(In reply to Evgeniy Khramtsov from comment #11)

This is not abandoned (couldn't touch for >week since May 2), current progress:

The bare minimum OSS code is mostly done.

PipeWire docs are incomplete about implementing new SPA plugins:

The design of SPA is complex, reading stuff from source and making sense of it
is hard (e.g. pointers named "this", SPA boilerplate, spa_hook, etc). This should take some time, though I would likely try to ask someone from PipeWire about SPA stuff.
Comment 13 Evgeniy Khramtsov 2021-05-20 18:18:28 UTC
(In reply to Evgeniy Khramtsov from comment #0)

Note to a reader:

Audio can be configured using pipewire-module-pulse-tunnel:

ALSA OSS plugin approach did not work due to the difference with native alsa:
ALSA consumes planar audio samples, but OSS consumes interleaved audio samples.
This is not a problem because conversion is done: https://git.alsa-project.org/?p=alsa-plugins.git;a=blob;f=oss/pcm_oss.c;hb=v1.2.2#l48
But pipewire-pulse uses float 32-bit planar, which does not work with ALSA OSS:

Copying JACK SPA and adapting it to OSS would be bad because pipewire has a narrow set of formats for DSP: https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/0.3.28/spa/include/spa/param/audio/raw.h#L93 https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/0.3.28/src/pipewire/impl-port.c#L560
Converting to the standard S16LE before feeding into OSS will result in quality loss.

If one wants to to make pipewire-pulse output audio through OSS without pulseaudio, adapting pipewire-module-pulse-tunnel would be a good starting point: https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/0.3.28/src/modules/module-pulse-tunnel.c#L305

After pipewire-module-pulse-tunnel was implemented, there is less motivation for me to dig deeper into pipewire, given that the code is complicated and I didn't get any response about some of my questions on DSP.

Note that xdg-desktop-portal is here https://github.com/flatpak/xdg-desktop-portal/pull/532
Comment 14 Evgeniy Khramtsov 2021-06-14 20:33:39 UTC
(In reply to Evgeniy Khramtsov from comment #13)

Thanks to the introduction of xdg-desktop-portal to ports by arrowd@:
the further progress can now be done

On the PipeWire side, there seems to be no issue currently with
the dbus, pipewire, xdg-desktop-portal-wlr, xdg-desktop-portal combo.

Started in order of:

$ ck-launch-session sway
$ dbus-daemon --session --address=unix:path=$XDG_RUNTIME_DIR/bus
$ PIPEWIRE_DEBUG=3 pipewire
$ /usr/local/libexec/xdg-desktop-portal-wlr -l DEBUG
$ /usr/local/libexec/xdg-desktop-portal --verbose -r
$ ./xdp-screen-cast.py
(from https://gitlab.gnome.org/snippets/19)

PipeWire side looks OK with GSTREAMER=on MEDIASESSION=on:

[I][000003786.849115][module-access.c:179 context_check_access()] client 0x53b6bdeb500 has trusted pid 31276
[I][000003786.849122][module-access.c:243 context_check_access()] access 0x53b6be32fc0: client 0x53b6bdeb500 'unrestricted' access granted
[I][000003795.748258][module-portal.c:162 on_portal_pid_received()] Got portal pid 40158

xdg-desktop-portal-wlr side looks OK:

2021/06/14 23:07:11 [DEBUG] - wayland: registry listeners run

xdg-desktop-portal side registers portals:

XDP: Using wlr.portal for org.freedesktop.impl.portal.Screenshot in sway
XDP: providing portal org.freedesktop.portal.Screenshot
XDP: Using wlr.portal for org.freedesktop.impl.portal.ScreenCast in sway
XDP: providing portal org.freedesktop.portal.ScreenCast
XDP: org.freedesktop.portal.Desktop acquired

Request for screencapture using xdp-screen-cast.py currently fails due to something unimplemented (?) in xdg-desktop-portal:

dbus.exceptions.DBusException: org.freedesktop.DBus.Error.AccessDenied: Portal operation not allowed: Unable to open /proc/78651/root

proc via procfs and giving access rights to user were both done for this test.

More to check with xdg-desktop-portal for screen capture.
Comment 15 Gleb Popov freebsd_committer 2021-06-15 10:18:14 UTC
(In reply to Evgeniy Khramtsov from comment #14)

This is because our procfs implementation doesn't support /proc/[pid]/root as documented in [1]. This will require a workaround, I guess.

[1] https://man7.org/linux/man-pages/man5/proc.5.html
Comment 16 Evgeniy Khramtsov 2021-06-27 12:21:01 UTC
(In reply to Evgeniy Khramtsov from comment #14)

I have to update this...

Upstream recently greatly improved SPA and modules documentation:


The better way to invoke dbus with environment variable injection would be: ck-launch-session dbus-run-session sway. Then starting "dbus-daemon --session" is not needed and the environment variables are passed to everything.

xdp_invocation_lookup_app_info_sync returns NULL, omitting check

--- a/xdg-desktop-portal-1.8.1/src/xdg-desktop-portal.c
+++ b/xdg-desktop-portal-1.8.1/src/xdg-desktop-portal.c
@@ -144,14 +144,6 @@ authorize_callback (GDBusInterfaceSkeleton *interface,
   g_autoptr(GError) error = NULL;

   app_info = xdp_invocation_lookup_app_info_sync (invocation, NULL, &error);
-  if (app_info == NULL)
-    {
-      g_dbus_method_invocation_return_error (invocation,
-                                             G_DBUS_ERROR,
-                                             G_DBUS_ERROR_ACCESS_DENIED,
-                                             "Portal operation not allowed: %s", error->message);
-      return FALSE;
-    }

will get Firefox to output screen capture pop-up and start screen capture via PipeWire, but then pipewire-media-session would segfault in spa_list_remove list.h line 72 (sorry I don't have the old coredump with symbols, should be easy to reproduce...)

Likely related due to overrun somewhere in pipewire due to unexpected data from xdg-desktop-portal (dbus) (app_info is NULL).

Progress takes too long; due to unrelated events, screen capture is now the least I have to worry about now. Unrelated events also contributed to burnout; now I lack the enthusiasm to work on something that I will never use, and I have to force myself to work on it, which doesn't work... I think anyone interested can finish screen capture, the prerequisites were done...
Comment 17 Gleb Popov freebsd_committer 2021-06-27 13:52:31 UTC
Should we close this for now? The PR would at least serve as a development roadmap for future contributors.
Comment 18 Evgeniy Khramtsov 2021-06-27 13:57:40 UTC
(In reply to Gleb Popov from comment #17)

> Should we close this for now?

It is up to you. While it is open, it is trivially searchable in bugzilla with "pipewire" keyword. Feel free to close it if you prefer so.
Comment 19 Evgeniy Khramtsov 2021-07-26 06:08:13 UTC
Created attachment 226703 [details]
screencapture-ootb-v1 (use "git am")

Screen capture tested to work with this patch "out of the box" using main ports branch:
commit 6a3cf86d09cb31d768e072fe0fc7f05126e12c20 Date: Sun Jul 25 20:33:29 2021 +0000

portlint -A: OK

poudriere bulk & testport of multimedia/pipewire and all consumers (via freshports):
12.2/i386:  OK
13.0/amd64: OK
14-CURRENT/amd64: OK

Runtime test:
14-CURRENT/amd64: OK

I can't trivially QA 13.0/aarch64 and consumers via qemu-user-static due to
lack of packages to seed after pkg 1.17.0 upgrade; pkg cluster should build new
index first. I can't QA consumers without binary package for lang/rust due to:
Comment 20 Evgeniy Khramtsov 2021-07-26 06:13:18 UTC
(In reply to Evgeniy Khramtsov from comment #19)

PORTREVISION is not bumped because of default option set change; pkg will pull the new package.
Comment 21 commit-hook freebsd_committer 2021-07-27 12:35:41 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=6e956b306e96701704da249305ab3dbfc6b823ed

commit 6e956b306e96701704da249305ab3dbfc6b823ed
Author:     Evgeniy Khramtsov <evgeniy@khramtsov.org>
AuthorDate: 2021-07-27 12:34:11 +0000
Commit:     Gleb Popov <arrowd@FreeBSD.org>
CommitDate: 2021-07-27 12:34:54 +0000

    multimedia/pipewire: Enable MEDIASESSION by default.

    PR:             255348

 multimedia/pipewire/Makefile | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)