Bug 280014 - comms/libimobiledevice version in port is obsolete
Summary: comms/libimobiledevice version in port is obsolete
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: Jason E. Hale
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-27 09:30 UTC by fabio.comolli
Modified: 2024-07-12 16:10 UTC (History)
2 users (show)

See Also:
jhale: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description fabio.comolli 2024-06-27 09:30:59 UTC
The current package is basically useless as it cannot backup any iPhones/iPads with a software newer than 14.

The same problem exists in Linux - I opened a package request in the ChaoticAUR Archlinux repository: https://github.com/chaotic-aur/packages/issues/3155 and the package they provided worked perfectly.

On Linux the git release is needed to be able to run succesful backups, not only for libimobiledevice but also for its dependencies (libplist and libusbmuxd) and possibly  usbmuxd.

This is an example: https://github.com/libimobiledevice/libimobiledevice/issues/1546
Comment 1 Jason E. Hale freebsd_committer freebsd_triage 2024-06-29 08:50:39 UTC
Newer releases of comms/libusbmuxd and devel/libplist have been made, but are not compatible with the rest of the libimobiledevice stack. I have been monitoring the situation for a while and even have a new port for libimobiledevice-glue ready, but was sort of waiting for the rest of it to catch up. Not sure what is going on upstream.

I'm on iOS/iPadOS 17.5.1 myself, so I appreciate your frustration and will try to update the ports within a week.
Comment 2 fabio.comolli 2024-06-29 08:52:44 UTC
(In reply to Jason E. Hale from comment #1)
Thank you very much. I guess you don't need my help for testing your new packages but if you need me just let me know.
Comment 3 Stephan Lichtenauer 2024-06-30 08:50:46 UTC
(In reply to Jason E. Hale from comment #1)

Thanks a lot also from me; I also watch this port and the upstream software for quite some time and would love to use it with an up to date iOS. :)
Comment 4 Jason E. Hale freebsd_committer freebsd_triage 2024-07-08 10:03:34 UTC
UPDATE: Sorry for any delays beyond my very optimistic and myopic prognosis of a week, but be assured that I am actively working on this.

It will take a little longer than I originally anticipated to update the libimobiledevice stack. Due to last week being a holiday week in the U.S., I've been swamped at $JOB and away celebrating a bit myself, both of which significantly dug into my free time that I usually spend working on FreeBSD. I also had to create another support port (security/libtatsu, https://github.com/libimobiledevice/libtatsu), which is done, but like with comms/libimobiledevice-glue is currently only in my local branch.

Pulling from git, rather than using the pre-rolled tarballs, has presented another obstacle that I also had to overcome due to upstream's choice of autotools as a build system along with the bashisms they've introduced into that environment.

Currently, I cannot get comms/py-libimobiledevice to build since the Python bindings now require Cython >= 3.0.0 which is not yet in the FreeBSD ports tree and that is beyond my control.

The current plan is to just mark the Python bindings as BROKEN since nothing in the ports tree depends on them. The rest of the stack, however, is updated and looking good from my initial build tests. Next step is more formal build tests and some basic run testing with consumer applications, which will probably take a couple more days.
Comment 5 fabio.comolli 2024-07-08 12:13:11 UTC
Thanks again for your work.
Comment 6 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:45:58 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=8a03fb6d4c504049cdc4f2a98b7f1c07cdbac881

commit 8a03fb6d4c504049cdc4f2a98b7f1c07cdbac881
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:36:04 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:31 +0000

    security/libtatsu: New port

    Library handling the communication with Apple's Tatsu Signing Server
    (TSS).

    This library is part of the libimobiledevice project and its main
    purpose is to create TSS request payloads, send them to Apple's TSS
    server, and ultimately retrieve and process the response.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 security/Makefile                 |  1 +
 security/libtatsu/Makefile (new)  | 30 ++++++++++++++++++++++++++++++
 security/libtatsu/distinfo (new)  |  3 +++
 security/libtatsu/pkg-descr (new) |  6 ++++++
 security/libtatsu/pkg-plist (new) |  7 +++++++
 5 files changed, 47 insertions(+)
Comment 7 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:45:59 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=b5256acd117dbb9692efabed6bdd5b588b1741e9

commit b5256acd117dbb9692efabed6bdd5b588b1741e9
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:46:37 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:32 +0000

    comms/usbmuxd: Update to latest git commit

    This is needed to build with the rest of the libimobiledevice stack.
    See next commit for more details.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 comms/usbmuxd/Makefile                        | 24 +++++++++++++++---------
 comms/usbmuxd/distinfo                        |  6 +++---
 comms/usbmuxd/files/patch-src_client.c (gone) | 13 -------------
 3 files changed, 18 insertions(+), 25 deletions(-)
Comment 8 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:46:01 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=9e7dc527bb360b645ea90d3e8b1edf722a08c758

commit 9e7dc527bb360b645ea90d3e8b1edf722a08c758
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:40:32 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:31 +0000

    comms/libimobiledevice-glue: New port

    Library with common code used by the libraries and tools around the
    libimobiledevice project.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 comms/Makefile                              |  1 +
 comms/libimobiledevice-glue/Makefile (new)  | 29 +++++++++++++++++++++++++++++
 comms/libimobiledevice-glue/distinfo (new)  |  3 +++
 comms/libimobiledevice-glue/pkg-descr (new) |  2 ++
 comms/libimobiledevice-glue/pkg-plist (new) | 16 ++++++++++++++++
 5 files changed, 51 insertions(+)
Comment 9 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:46:02 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=41f806cbbc2a42b16932ce9a373b6d31014305e6

commit 41f806cbbc2a42b16932ce9a373b6d31014305e6
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 01:30:36 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:33 +0000

    *.*: Bump consumers of devel/libplist

    Shlib version has be been bumped in devel/libplist and other components
    pertaining to the libimobledevice stack. Consequently, all consumers need
    to be rebuilt.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 audio/libgpod/Makefile         | 2 +-
 audio/owntone/Makefile         | 2 +-
 audio/shairport-sync/Makefile  | 1 +
 devel/kio-extras/Makefile      | 2 +-
 multimedia/droidcam/Makefile   | 1 +
 multimedia/uxplay/Makefile     | 1 +
 sysutils/fusefs-ifuse/Makefile | 2 +-
 7 files changed, 7 insertions(+), 4 deletions(-)
Comment 10 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:46:03 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=a092997dfa91fe00b4ecb79133d689ed702f2533

commit a092997dfa91fe00b4ecb79133d689ed702f2533
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:24:11 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:30 +0000

    audio/libgpod: Port to libplist_dict_set_item()

    Deprecated method plist_dict_insert_item() has been removed and
    replaced by plist_dict_set_item() in devel/libplist >= 2.3.0.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 .../files/patch-tools_ipod-lockdown.c (new)        | 33 ++++++++++++++++++++++
 1 file changed, 33 insertions(+)
Comment 11 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:46:04 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=2c3fb6ea503a1e12920220860521b0a8b8582c0f

commit 2c3fb6ea503a1e12920220860521b0a8b8582c0f
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:43:06 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:31 +0000

    comms/libusbmuxd: Update to 2.1.0

    https://github.com/libimobiledevice/libusbmuxd/releases/tag/2.1.0

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 comms/libusbmuxd/Makefile  | 14 ++++++--------
 comms/libusbmuxd/distinfo  |  6 +++---
 comms/libusbmuxd/pkg-plist |  4 ++--
 3 files changed, 11 insertions(+), 13 deletions(-)
Comment 12 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:46:05 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=208fb5d5fa8a13328502b7faaa46bb9ddf575348

commit 208fb5d5fa8a13328502b7faaa46bb9ddf575348
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:04:39 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:30 +0000

    devel/*libplist: Update to 2.6.0

    https://github.com/libimobiledevice/libplist/releases/tag/2.3.0
    https://github.com/libimobiledevice/libplist/releases/tag/2.4.0
    https://github.com/libimobiledevice/libplist/releases/tag/2.5.0
    https://github.com/libimobiledevice/libplist/releases/tag/2.6.0

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 devel/libplist/Makefile                 | 16 ++++++++-------
 devel/libplist/distinfo                 |  6 +++---
 devel/libplist/files/patch-configure.ac | 36 +++++++++++++++++----------------
 devel/libplist/pkg-plist                |  8 ++++----
 devel/py-libplist/Makefile              | 23 +++++++++------------
 5 files changed, 45 insertions(+), 44 deletions(-)
Comment 13 commit-hook freebsd_committer freebsd_triage 2024-07-11 01:46:06 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=2705dd431c33ae6cbf5f66c64f62f450f21e2619

commit 2705dd431c33ae6cbf5f66c64f62f450f21e2619
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 00:55:39 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-11 01:45:32 +0000

    comms/*libimobiledevice: Update to latest git commit

    The libimobiledevice stack has been undergoing great changes over the
    years and it has come to a head where modern iOS/iPadOS users are unable
    to make practical use of these tools in the official releases. Since most
    iOS/iPadOS users are likely using the most recent versions, I have
    decided the best course of action is to follow the git repo and created
    a couple of new support ports (comms/libimobiledevice-glue and
    security/libtatsu) needed for this to be a reality.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 comms/libimobiledevice/Makefile                    | 41 +++++++++++-----------
 comms/libimobiledevice/distinfo                    |  6 ++--
 .../libimobiledevice/files/patch-configure (gone)  | 13 -------
 .../files/patch-configure.ac (new)                 | 13 +++++++
 comms/libimobiledevice/pkg-plist                   |  7 ++++
 comms/py-libimobiledevice/Makefile                 | 29 ++++++++-------
 6 files changed, 60 insertions(+), 49 deletions(-)
Comment 14 Jason E. Hale freebsd_committer freebsd_triage 2024-07-11 02:49:37 UTC
Good news, everyone! So after enabling Developer Mode via the new idevicedevmodectl tool from comms/libimobiledevice, which seems to work just fine on my iPhone 13 Pro with iOS 17.5.1, idevicebackup2 seems to also be working. It's kind of finicky, but that might just be me since I've, admittedly, never actually used it. We're also living on the cutting edge now, so there's that. I had to restart my phone and usbmuxd a few times to get it to really roll.

Anyways, once I got it going, idevicebackup2 began diligently eating up the rest of my available disk space with gusto and I had to call it off at a point, because I really don't have that kind of space left on my devel laptop, but not before seeing recognizable data pulled directly from my phone. From my POV, "It works".

I'm curious if either of you had Developer Mode enabled prior to iOS 17 and it remained on with subsequent iOS updates? My understanding is that Apple removed this once readily-accessible on-device toggle in iOS 17, which now seems to require activation by a "real computer" (i.e., buy a Mac from us or be clever enough to make a workaround to enable). Fortunately, the latter has been done via idevicedevmodectl. I will probably never buy a Mac, but am quite fond of Apple's handheld devices.

Despite being a longtime user of Apple's mobile products (iPhone/iPad) I have never actually enabled Developer Mode on any of my devices, but these ports could probably do with some updated documentation, since it will probably not be clear to the average user that they need to enable Developer Mode on their device (now completely hidden from iOS 17+ users by default and must now be revealed via the new idevicedevmodectl to access this feature) along with other bits and bobs, which is why I'm asking.

I was also wrong about cython 3.x. We do indeed have a lang/cython3 port, but it's just not incorporated into the python framework like lang/cython. After some adjustments, the python bindings build just fine. I still have absolutely no idea if they really work, however, since nothing in the ports tree uses them. Still a win for not having to mark them BROKEN.

If all is good on your end, please close this PR as FIXED. If I don't hear a response within two weeks, I will close this PR as FIXED since it works for me.
Comment 15 fabio.comolli 2024-07-11 08:52:49 UTC
I can confirm that I never had to enable the developer mode on my SE 2020 running 17.5.1 (actually I never knew this feature existed until now).

Moreover, this procedure must be very new on the libimobiledevice side as I was an Archlinux user until two weeks ago and everything worked just fine without activating this developer mode.

I can see from the bug I opened on Chaotic AUR https://github.com/chaotic-aur/packages/issues/3155 that it was closed on June 9th. Also two days ago the new libtatsu dependency was added.

I will check later today and report back.
Comment 16 fabio.comolli 2024-07-11 18:21:53 UTC
I'm sorry to report that it doesn't work for me. I managed to get my device connected only a few times and in one case I even got the developer mode activated but the backup never starts and the connection is flaky at best.

Moreover, usbmuxd gets killed evey time I connect my phone like this:

2024-07-11T20:17:22.238883+02:00 tu45b-freebsd usbmuxd[6178] [2] Switching device 1-2 mode to 3
2024-07-11T20:17:22.365542+02:00 tu45b-freebsd usbmuxd[6178] [2] Cannot find device entry while removing USB device 0x3c5e1403e180 on location 0x10002
2024-07-11T20:17:22.401758+02:00 tu45b-freebsd kernel: ugen1.2: <Apple Inc. iPhone> at usbus1
2024-07-11T20:17:22.401773+02:00 tu45b-freebsd kernel: ugen1.2: <Apple Inc. iPhone> at usbus1 (disconnected)
2024-07-11T20:17:22.760810+02:00 tu45b-freebsd usbmuxd[6222] [1] Another instance is already running (pid 6178). exiting.
2024-07-11T20:17:22.803765+02:00 tu45b-freebsd kernel: ugen1.2: <Apple Inc. iPhone> at usbus1

it looks like that the phone connects, disconnects and connects again.

This is my device:

[20:18][fmc000@tu45b-freebsd][~]
 ⤷ $ usbconfig list|grep -i phone
ugen1.2: <iPhone 5/5C/5S/6/SE/7/8/X/XR Apple, Inc.> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA)
[20:18][fmc000@tu45b-freebsd][~]
 ⤷ $ 

and these are the 4 entries that get created under /deb/usb :

[20:19][fmc000@tu45b-freebsd][~]
 ⤷ $ ls -l /dev/usb/1.2*
crw-rw----  1 root usbmuxd  0x78 Jul 11 20:19 /dev/usb/1.2.0
crw-rw----  1 root operator 0x84 Jul 11 20:19 /dev/usb/1.2.1
crw-rw----  1 root operator 0x85 Jul 11 20:19 /dev/usb/1.2.2
crw-rw----  1 root operator 0x86 Jul 11 20:19 /dev/usb/1.2.3
[20:19][fmc000@tu45b-freebsd][~]
Comment 17 commit-hook freebsd_committer freebsd_triage 2024-07-12 06:24:56 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=f4bd2eeb661fd673294ee9a92b397b56b91ccde3

commit f4bd2eeb661fd673294ee9a92b397b56b91ccde3
Author:     Jason E. Hale <jhale@FreeBSD.org>
AuthorDate: 2024-07-11 22:36:26 +0000
Commit:     Jason E. Hale <jhale@FreeBSD.org>
CommitDate: 2024-07-12 06:24:35 +0000

    comms/usbmuxd: Workaround devd(8) race condition

    When connecting an Apple device via USB, the device connects, usbmuxd is
    started, then for reasons yet unknown, the device instantly disconnects,
    and reconnects.

    From dmesg when plugging in, for example:
    ugen0.5: <Apple Inc. iPhone> at usbus0
    <usbmuxd starts here>
    ugen0.5: <Apple Inc. iPhone> at usbus0 (disconnected)
    ugen0.5: <Apple Inc. iPhone> at usbus0

    Due to the disconnect, the instance of usbmuxd that is spawned when the
    device is first attached is subsequently killed by the unexplained
    detach because the cdce(4) devices have not yet been created, but not
    before the second instance of usbmuxd is started by the reattach. The
    second instance automatically exits because two instances cannot run
    simultaneously. The first instance then exits leaving no running
    instances.

    Adding a short sleep after the first instance is called to shutdown
    gives it enough time to completely exit before the second instance is
    spawned. Unfortunately, only one device is supported at a time until
    I can figure out why the hotplugging isn't working.

    While here, cleanup whitespace.

    PR:             280014
    Reported by:    <fabio.comolli@gmail.com>

 comms/usbmuxd/Makefile              |  1 +
 comms/usbmuxd/files/usbmuxd.conf.in | 12 ++++++------
 2 files changed, 7 insertions(+), 6 deletions(-)
Comment 18 Jason E. Hale freebsd_committer freebsd_triage 2024-07-12 06:37:51 UTC
(In reply to fabio.comolli from comment #16)
I added a workaround that mostly fixes the second problem. Oddly, I can't get idevicebackup2 to work anymore, either. I'm consistently seeing errors like:

ERROR: Could not start service com.apple.afc: Receive timeout
ERROR: Could not start service com.apple.mobile.notification_proxy: Receive timeout

Looks similar to an old semi-resolved bug filed upstream: https://github.com/libimobiledevice/libimobiledevice/issues/773

This was what I was seeing before enabling Developer Mode, which made me think this step was necessary (maybe it is) but I still have it enabled and still getting these errors. Don't really know what I did to make it work the first time. This is going to take some digging, I'm afraid. :(
Comment 19 Jason E. Hale freebsd_committer freebsd_triage 2024-07-12 06:57:49 UTC
(In reply to Jason E. Hale from comment #18)
Small addendum: enabling the INOTIFY option in comms/libusbmuxd seems to fix the hotplugging issue.
Comment 20 fabio.comolli 2024-07-12 07:04:20 UTC
I can confirm now that connecting the iPhone correctly spawns a usbmuxd process that dies on disconnect. However, still nothing works apart from idevice_id that correctly reports the serial number. Not even ideviceinfo works now ("cannot connect to lockdownd" or "mux error", it depends).

Even worse, on my phone now the "developer mode" option disappeared after a reboot :-(
Comment 21 fabio.comolli 2024-07-12 07:05:23 UTC
(In reply to Jason E. Hale from comment #19)

Should I update it now this lib? I just reinstalled everything maybe half an hour ago.
Comment 22 Jason E. Hale freebsd_committer freebsd_triage 2024-07-12 07:17:52 UTC
(In reply to fabio.comolli from comment #21)
Rebuild comms/usbmuxd to get the updated usbmuxd.conf. You will have to restart devd to get it to work. (sudo service devd restart)

Enabling INOTIFY comms/libusbmuxd isn't really necessary unless you need to connect more than one device at a time.

I haven't has any trouble with ideviceinfo. "cannot connect to lockdownd" sounds like the computer isn't trusted by the device. I accidentally revoked trust when I was testing with my iPad and the only way to reset it is Settings -> General -> Transfer or Reset iPhone/iPad -> Reset -> Reset Location & Privacy.

No idea why Developer Mode would disappear. I did a few restarts and it is still there on my iPhone and left enabled. Welcome to iHell! :P
Comment 23 Jason E. Hale freebsd_committer freebsd_triage 2024-07-12 07:24:17 UTC
I've also seen my Xorg process spike to 100% CPU and seeming to hang during my messing around, but that was easily remedied by switching to a console and restarting devd. Just something to be aware of.
Comment 24 fabio.comolli 2024-07-12 16:10:55 UTC
I tried everything you suggested to no avail. At this point I give up, feel free to close this issueif it works for you.

Thanks for everything.