Bug 245997 - emulators/qemu emulators/qemu-devel sysutils/dtc devel/valgrind: use system dtc in qemu builds + additional updates
Summary: emulators/qemu emulators/qemu-devel sysutils/dtc devel/valgrind: use system d...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Muhammad Moinur Rahman
Depends on: 239669
  Show dependency treegraph
Reported: 2020-04-28 09:55 UTC by Sean Champ
Modified: 2021-04-20 19:47 UTC (History)
5 users (show)

See Also:
linimon: maintainer-feedback? (bofh)

Patch for emulators/qemu emulators/qemu-devel sysutils/dtc devel/valgrind (9.36 KB, patch)
2020-04-28 09:55 UTC, Sean Champ
no flags Details | Diff
Linted patch for emulators/qemu emulators/qemu-devel sysutils/dtc devel/valgrind (9.05 KB, patch)
2020-04-28 10:37 UTC, Sean Champ
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sean Champ 2020-04-28 09:55:13 UTC
Created attachment 213883 [details]
Patch for emulators/qemu emulators/qemu-devel sysutils/dtc devel/valgrind

In synopsis: When building devel/qemu under poudriere, on a FreeBSD 12.1 amd64 machine I was encountering a persistent build failure with devel/qemu during staging - an excerpt from the build log
mkdir: dtc/libfdt/libfdt.h: File exists
gmake[1]: *** [Makefile:488: dtc/libfdt/libfdt.h] Error 1
gmake[1]: Leaving directory '/wrkdirs/usr/ports/emulators/qemu/work/qemu-4.1.1'

After taking a look at the qemu configure script, I discovered that it may be possible to build with an existing libfdt.so. In order to do so, I've had to update sysutils/dtc to version 1.6.0, as well as updating how the extra ldflags are provided to the qemu configure script.

The qemu build now succeeds, locally. The additional testing now available for sysutils/dtc also succeeds under make(test). For purpose of usability, I've made some additional changes in the update - as detailed, in the following.

Summary of changes:

devel/valgrind: Update vki-freebsd.h patch for changes recommended in

emulators/qemu-devel: Add dtc dependency; move debug support into a
  DEBUG port option, also setting WITH_DEBUG in this situation, enabling
  the option by default; allow build with ncurses from ports, similar to
  the emulators/qemu port; Upate how extra-ldflags are provided to the
  configure script - providing the value only once, with no quote marks,
  to ensure that the system dtc is detected during configure

emulators/qemu: Update in parallel to emulator/qemu for dtc from ports
  and debug support

sysutils/dtc: Update to upstream release 1.6.0 - in which, the
  function fdt_check_full() is now defined differently, to a side effect
  as that the function should not be stripped from libfdt.so during normal
  pkg staging. The function is used during qemu configure, for detecting
  the presence of a system dtc. Add port option for building with yaml
  support for dtc (not enabled by default); Add port option for valgrind
  support during make(test), for amd64, i386 architectures; Add
  post-patch hacks for ensuring that the build will not fail when yaml.h
  and valgrind/memcheck.h cannot be located under wrksrc - these were
  being added to make dependencies under generated *.d files, in local
  build, should otherwise be unused in the build; Add subst vars for
  port version in distinfo, as a convenience for port maint.
Comment 1 Sean Champ 2020-04-28 10:05:02 UTC
In an update to the previous remarks: I may've introduced the option to build emulators/qemu with ncurses from ports, under an earlier local change. I've not tested this change with regards to the ncurses used, beyond the initial build, for emulators/qemu or emulators/qemu-devel
Comment 2 Sean Champ 2020-04-28 10:13:57 UTC
The remark with regards to extra-ldflags at https://bugs.freebsd.org/bugzilla/attachment.cgi?id=213883&action=diff#b/emulators/qemu-devel/Makefile_sec4 should not have been published in the diff

similarly, the remark at the section

It's taken a number of days to put this patch together. Candidly, I'd thought I'd linted those remark out of the actual published diff.

Can put together an updated diff, sometime later today
Comment 3 Sean Champ 2020-04-28 10:37:02 UTC
Created attachment 213884 [details]
Linted patch for emulators/qemu emulators/qemu-devel sysutils/dtc devel/valgrind

I've updated the local patch branch such as to remove two ephemeral, local comments from the resulting diff. The updated patch is attached
Comment 4 Mark Linimon freebsd_committer freebsd_triage 2020-08-15 05:17:08 UTC
^Triage: assign to maintainer of emulators/qemu* but also Cc: maintainers of sysutils/dtc and devel/valgrind.
Comment 5 Emmanuel Vadot freebsd_committer 2020-09-11 21:07:21 UTC
dtc was updated to 1.6.0 a few weeks ago.
Comment 6 Warner Losh freebsd_committer 2020-09-11 21:16:47 UTC
I'm not the official qemu maintainer, but do generally approve of this direction FWIW.
Comment 7 Muhammad Moinur Rahman freebsd_committer 2020-09-11 23:52:23 UTC
1. We are moving qemu-devel to SNAPSHOTS version where we cannot use that specific version as it requires HEAD of dtc. Hopefully this will be committed by today or tomorrow. So I am going to add a CONFLICT with sysutils/dtc for qemu-devel.
2. Please see the latest qemu/Makefile for ncurses implementation giving the option of both. Giving the same for qemu-devel.
3. I am not keen on OPTIONIZING DEBUG for qemu-devel although will do it for qemu. Unless there is a very good reason behind it.
4. Please run portlint -abct before submitting patches. My eyes say that qemu/Makefile and qemu-devel/Makefile should throw errors with Contiguous blank lines.
5. When submitting a bug report please try to submit multiple bugs with different ports as bugzilla automagically assigns the bugs based on portname. Multiple portname causes it to have a panick attack. Depends or Blocks is easier to handle and easier to track.
Comment 8 commit-hook freebsd_committer 2020-09-12 22:25:37 UTC
A commit references this bug:

Author: bofh
Date: Sat Sep 12 22:25:07 UTC 2020
New revision: 548428
URL: https://svnweb.freebsd.org/changeset/ports/548428

  emulators/qemu-devel: Update version 4.2.0=>

  - Move to SNAPSHOT versions of qemu
  - Remove i386 support temporarily as build is failing on i386
  - Mark IGNORE for 11.X as it cannot build Position Independent Code on 11
  - OPTIONIZE NCURSES for support of both base and ports [1]
  - Add OPTIONS CAPSTONE and use CAPSTONE from ports
  - Remove PCAP related patches as was unable to create PANIC while using TSO
  - Fixes for the RISC-V PLIC that are required to boot FreeBSD/riscv64 HEAD
    after r362977

  PR:		245997 [1]
  Submitted by:	lab+bsd@thinkum.space [1]
  Reported by:	lwhsu jhb

Comment 9 Steve Wills freebsd_committer 2021-04-07 16:19:43 UTC
This seems to happen with emulators/qemu50 as well.
Comment 10 Muhammad Moinur Rahman freebsd_committer 2021-04-07 17:08:56 UTC
(In reply to Steve Wills from comment #9)
It's better not to use qemu50 as it is a placeholder for qemu 5.0.X branches in the future when 5.1.X or 5.2.X or 6.X branch moves to stable qemu will be moved to qemu50. In fact that is a reason you sill see that qemu is on 5.0.1 but qemu is still on 5.0.0.
Looking forward to work on qemu-devel in the upcoming weekend.
Comment 11 Steve Wills freebsd_committer 2021-04-08 13:14:22 UTC
(In reply to Muhammad Moinur Rahman from comment #10)
Ports shouldn't just be broken and left that way. If it's there, it should build. If it shouldn't be used, it should be marked deprecated and removed.
Comment 12 Steve Wills freebsd_committer 2021-04-08 15:49:15 UTC
Also, I'm still seeing this issue for emulators/qemu as well.
Comment 13 Muhammad Moinur Rahman freebsd_committer 2021-04-08 16:01:23 UTC
(In reply to Steve Wills from comment #12)
Give me the weekend to check this.
Comment 14 Muhammad Moinur Rahman freebsd_committer 2021-04-11 19:42:45 UTC
(In reply to Steve Wills from comment #12)
Can you share me some build logs? I can't find it in my latest builds.
Comment 15 Steve Wills freebsd_committer 2021-04-12 01:37:43 UTC
(In reply to Muhammad Moinur Rahman from comment #14)
I sent a link to the log via email, please let me know if you didn't get it.
Comment 16 commit-hook freebsd_committer 2021-04-20 19:47:12 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=9e06cd0454806020fc8e2f11df85baf500690902

commit 9e06cd0454806020fc8e2f11df85baf500690902
Author:     Muhammad Moinur Rahman <bofh@FreeBSD.org>
AuthorDate: 2021-04-20 19:41:51 +0000
Commit:     Muhammad Moinur Rahman <bofh@FreeBSD.org>
CommitDate: 2021-04-20 19:46:11 +0000

    emulators/qemu: Fix build on 13

    - Use ports dtc instead of packaged dtc [1]
    - bin/elf2dmp apparently depends on CURL [2]
    - Reduce build to only X86 architectures if X86_TARGETS is set [2]
    - rx is not an X86 architecture [2]

    PR:             245997 [1] 254385 [2]
    Reported by:    swills [1] pstef [2]

 emulators/qemu/Makefile  | 13 +++++++++++--
 emulators/qemu/pkg-plist |  6 ++++--
 2 files changed, 15 insertions(+), 4 deletions(-)