Bug 211973
| Summary: | emulators/qemu : running qemu-system-i386 -curses gives garbled output on screen | ||
|---|---|---|---|
| Product: | Ports & Packages | Reporter: | martin <martin> |
| Component: | Individual Port(s) | Assignee: | Muhammad Moinur Rahman <bofh> |
| Status: | Closed FIXED | ||
| Severity: | Affects Some People | CC: | jkim |
| Priority: | --- | Flags: | martin:
maintainer-feedback+
|
| Version: | Latest | ||
| Hardware: | amd64 | ||
| OS: | Any | ||
| Attachments: | |||
|
Description
martin
2016-08-18 21:33:20 UTC
A commit references this bug: Author: bofh Date: Fri Aug 19 20:52:58 UTC 2016 New revision: 420489 URL: https://svnweb.freebsd.org/changeset/ports/420489 Log: emulators/qemu: Update version 2.5.1.1=>2.6.1 - Remove BSD_USER related codes - Add NCURSES support [1] - Fix STATIC_LINKS code to not build with X11 support PR: 211973 [1] Submitted by: ilavsky.martin@gmail.com [1] Changes: head/emulators/qemu/Makefile head/emulators/qemu/distinfo head/emulators/qemu/files/patch-Makefile head/emulators/qemu/files/patch-configure head/emulators/qemu/files/patch-disas_libvixl_a64_disasm-a64.cc head/emulators/qemu/files/patch-disas_libvixl_vixl_a64_disasm-a64.cc head/emulators/qemu/files/patch-include_qemu-common.h head/emulators/qemu/files/pcap-patch head/emulators/qemu/files/pcap-patch-net_net.c head/emulators/qemu/pkg-plist emulators/qemu-devel will be fixed on the 2.7 version. I fetched the new version of emulators/qemu, recompiled but the issue is the same - I still can't see the proper console output:
$ qemu-system-i386 --version
QEMU emulator version 2.6.1, Copyright (c) 2003-2008 Fabrice Bellard
$
$ ldd `which qemu-system-i386`|grep curs
libncurses.so.6 => /usr/local/lib/libncurses.so.6 (0x8018bc000)
$
Do you have NCURSES option selected ? Which version of FreeBSD are you using. So far I tested with 9/10/11 and found ncurse frontend working. Yes, I have. Following options are set: $ pkg info qemu qemu-2.6.1 Name : qemu Version : 2.6.1 Installed on : Fri Aug 19 23:46:14 2016 CEST Origin : emulators/qemu Architecture : freebsd:10:x86:64 Prefix : /usr/local Categories : emulators Licenses : GPLv2 Maintainer : bofh@FreeBSD.org WWW : http://wiki.qemu.org/Main_Page Comment : QEMU CPU Emulator Options : CDROM_DMA : on CURL : on DOCS : on GNS3 : off GNUTLS : on GTK2 : off JPEG : on NCURSES : on OPENGL : off PCAP : on PNG : on SAMBA : off SASL : on STATIC_LINK : off USBREDIR : off X11 : off X86_TARGETS : off Shared Libs required: libpng16.so.16 libintl.so.8 libgnutls.so.30 libtinfo.so.6 libpixman-1.so.0 libncurses.so.6 libgthread-2.0.so.0 libglib-2.0.so.0 libnettle.so.6 libcurl.so.4 Annotations : cpe : cpe:2.3:a:qemu:qemu:2.6.1:::::freebsd10:x64 Flat size : 142MiB FreeBSD 10.3 amd64 with ncurses-6.0_2. To test the ncurses I compiled their demos (http://invisible-island.net/datafiles/release/ncurses-examples.tar.gz) and they work OK. I did recompile every dependency that qemu has but the issue is the same. I've put the status to open as the issue is the same. I did some further testing. Using pkg I installed qemu to alternate location: # pkg -r /root/qemu install qemu and executed: export LD_LIBRARY_PATH=/root/qemu/usr/local/lib:/root/qemu/usr/local/libexec:/root/qemu/usr/local/lib/.mesa /root/qemu/usr/local/bin/qemu-system-i386 -curses That worked ok. Compiled version of qemu (version 2.6.1), even when custom LD_LIBRARY_PATH was specified failed with the same issue. My /etc/make.conf: # grep -vE '^$|^#' /etc/make.conf STRIP= CFLAGS+=-fno-omit-frame-pointer NO_PROFILE=true WITHOUT_X=yes WITH_X=NO ENABLE_GUI=NO OPTIONS_UNSET=X11 # On the same system I installed qemu in jail. It worked there. One-by-one lib comparison showed this difference:
host# ldd `which qemu-system-i386` | grep curs
libncurses.so.6 => /usr/local/lib/libncurses.so.6 (0x80164a000)
jail# ldd `which qemu-system-i386`
libncurses.so.8 => /lib/libncurses.so.8 (0x80164b000)
ncurses6 is dependency on my system for other packages from ports (rtorrent, clamav, python27 and few more).
Checking config of emulators/qemu it seems it doesn't make any difference whether I choose to have ncurses or not -- it will be always compiled in. Problem is that it wants one from ports (/usr/local/lib/libncurses.so.6) and not the system one (/lib/libncurses.so.8).
I've forced pkg to remove ncurses6 leaving broken ports for a moment. I've installed qemu and now all is working OK:
host# ldd `which qemu-system-i386` | grep curs
libncurses.so.8 => /opt/qemu/lib/libncurses.so.8 (0x80164a000)
I've compiled the ncurses6 again and verified the libs:
host#pkg check --shlibs
Checking all packages: 100%
host#
Qemu doesn't work with the ncurses6 from ports, config option NCURSES doesn't seem to be doing anything.
As there is no way to edit the comment I need to create a new reply. In the example where I said it works I pasted the output where I used custom library path (/opt/qemu/lib/), but it does work with the system one too:
host# ldd `which qemu-system-i386` | grep curs
libncurses.so.8 => /lib/libncurses.so.8 (0x80164a000)
Seems like good observation. Let me check with my builder again. A commit references this bug: Author: bofh Date: Sat Aug 27 17:15:51 UTC 2016 New revision: 420986 URL: https://svnweb.freebsd.org/changeset/ports/420986 Log: emulators/qemu: Unbreak in 11+ [1] - Fix ncurses build uses to use base version and remove from CONFIGURE_ARGS for OPTIONS usage [2] PR: 212057 [1] 211973 [2] Submitted by: mat [1] ilavsky.martin@gmail.com [2] Changes: head/emulators/qemu/Makefile head/emulators/qemu/files/patch-include_qemu_atomic.h qemu now uses ncurses from base and I have removed the default ncurses from CONFIGURE_ARGS so it is an optional build. I didn't mention it here but one of my attempts before was to use ncurses:base in my custom Makefile and it didn't work. I uninstalled qemu, sourced new ports tree, checked the revision on Makefile: host# head -2 /usr/ports/emulators/qemu/Makefile # Created by: Juergen Lock <nox@jelal.kn-bremen.de> # $FreeBSD: head/emulators/qemu/Makefile 420986 2016-08-27 17:15:50Z bofh $ # I recompiled qemu but unfortunately it ignored ncurses:base settings and linked it with the ones from ports: host# ldd `which qemu-system-i386` |grep curses libncurses.so.6 => /usr/local/lib/libncurses.so.6 (0x8018bc000) I made sure that NCURSES option was not selected in config. Actually, when I try to compile it with it it showed the error: host# cd /usr/ports/emulators/qemu host# make install clean Dependency error: this port wants the ncurses library from the FreeBSD base system. You can't build against it, while a newer version is installed by a port. Please deinstall the port or undefine WITH_NCURSES_BASE. *** Error code 1 Stop. make[1]: stopped in /usr/ports/emulators/qemu *** Error code 1 Stop. make: stopped in /usr/ports/emulators/qemu host# Which is that ncurses:base option in Makefile. Anyway, right now it's still not possible to use qemu+curses if ncurses6 is already installed on the system. I'm not that familiar with the Makefile nor the port config/build rules but it seems that LDFLAGS used there don't give an option to include libs from system.
LDFLAGS when port ncurses where linked:
LDFLAGS -Wl,--warn-common -m64 -g -fstack-protector -L"/usr/local/lib"
I modified the Makefile to have this:
host# diff Makefile Makefile.orig
75c75
< CONFIGURE_ARGS+=--localstatedir=/var --extra-ldflags=-L\"/usr/lib\" --extra-ldflags=-L\"${LOCALBASE}/lib\" \
---
> CONFIGURE_ARGS+=--localstatedir=/var --extra-ldflags=-L\"${LOCALBASE}/lib\" \
host#
and recompiled the qemu. It used proper ncurses and it worked ok. I'd expect nurses:base to mean to include all needed from system but it seems it doesn't do that.
I've put bug to status open, I can replicate this on every FreeBSD machine I've tried. If ncurses package from ports is already installed on the system linking is done towards ncurses lib from port -- output is garbled. I understand that LDFLAGS solution I used is not the good one in general as it can break stuff in some scenarios ; I thought ncurses:base would give a hint to linker to use the proper ncurses location though. As a workaround the best solution seems to be uninstall ncurses from ports temporarily before installing qemu. (In reply to martin ilavsky from comment #16) I have fixed it in 2.8.0. Can you please confirm the current status? I can test FreeBSD 10 on Monday. Machine I can test it on now is FreeBSD 11 r308208. I portsnaped the latest ports and tried to build it - got immediately failure: host:(/usr/ports/emulators/qemu)# make install clean ===> License GPLv2 accepted by the user ===> Found saved configuration for qemu-2.8.0 ===> qemu-2.8.0 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by qemu-2.8.0 for building ===> Extracting for qemu-2.8.0 => SHA256 Checksum OK for qemu/2.8.0/qemu-2.8.0.tar.bz2. ===> Patching for qemu-2.8.0 ===> Applying FreeBSD patches for qemu-2.8.0 sed: /usr/ports/emulators/qemu/work/qemu-2.8.0/hw/ide/internal.h: No such file or directory *** Error code 1 Stop. make[1]: stopped in /usr/ports/emulators/qemu *** Error code 1 Stop. make: stopped in /usr/ports/emulators/qemu host:(/usr/ports/emulators/qemu)# find . -name internal.h ./work/qemu-2.8.0/include/hw/ide/internal.h ./work/qemu-2.8.0/target-ppc/internal.h host:(/usr/ports/emulators/qemu)# So I'm stuck there. I see you fixed that, now I can compile it. But no matter what is selected in config (either with or without ncurses) curses are not selected: host:(/usr/ports/emulators/qemu)# make install ===> License GPLv2 accepted by the user ===> Found saved configuration for qemu-2.8.0_1 .. .. curses support no .. It ignores completely what was selected in config menu. Compared to emulators/qemu-devel which currently is qemu-devel-2.6.0_2 there's no option to select curses from config menu but curses option gets enabled: host:(/usr/ports/emulators/qemu-devel)# make ===> License GPLv2 accepted by the user ===> Found saved configuration for qemu-devel-2.6.0_2 .. .. curses support yes Which is what I would expect to have when ncurses is selected from config menu. Note on this system (FreeBSD 11) I don't have ncurses6 installed from ports, curses from base are used. Though this PR seems to be dead note that issue is still there. emulators/qemu is currently on version 2.8.1, selecting NCURSES in config menu has no effect, qemu is compiled without it. (In reply to martin ilavsky from comment #20) Working on 2.9. Lets see what I can do. Created attachment 187886 [details]
Do not let the configure script quietly enable curses support.
When NCURSES option is enabled (default) and devel/ncurses is installed, we cannot build it any more.
Dependency error: this port wants the ncurses library from the FreeBSD
base system. You can't build against it, while a newer
version is installed by a port.
Please deinstall the port or undefine WITH_NCURSES_BASE.
*** Error code 1
Because there are too many packages depending on devel/ncurses, I had no choice but to disable NCURSES option. However, it still fails to build.
ui/egl-helpers.c:55:42: error: use of undeclared identifier 'O_CLOEXEC'
return open(rendernode, O_RDWR | O_CLOEXEC | O_NOCTTY | O_NONBLOCK);
^
ui/egl-helpers.c:75:30: error: use of undeclared identifier 'O_CLOEXEC'
r = open(p, O_RDWR | O_CLOEXEC | O_NOCTTY | O_NONBLOCK);
^
2 errors generated.
This happens because the following CFLAGS are added:
-I/usr/local/include -I/usr/local/include/ncurses -D_BSD_TYPES -D__BSD_VISIBLE -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600
Specifically, "-D_POSIX_C_SOURCE=200112L" is the root cause of this problem. Although devel/ncurses is to blame, emulators/qemu should not quietly enable curses support when the user explicitly turned it off.
Created attachment 187890 [details]
Do not let the configure script quietly enable curses support.
Apparently "--enable-curses" is not allowed. Use "NCURSES_CONFIGURE_OFF=--disable-curses" instead of "NCURSES_CONFIGURE_ENABLE=curses".
FYI, bug 223578 has a patch to fix build with devel/ncurses. If the patch is committed, we can also revert "NCURSES_USES=ncurses:base" to "NCURSES_USES=ncurses". Created attachment 187913 [details] Do not let the configure script quietly enable curses support. Now we can use devel/ncurses because bug 223578 is fixed. I currently have qemu-2.9.0_1 in my ports with Makefile version:
# $FreeBSD: head/emulators/qemu/Makefile 457024 2017-12-23 07:03:21Z antoine $
I have ncurses installed from ports, few packages require it:
# pkg info ncurses\*
ncurses-6.0.0s20171007_1
#
# pkg info -r ncurses
ncurses-6.0.0s20171007_1:
rtorrent-0.9.6_3
texinfo-6.5,1
python27-2.7.14_1
sqlite3-3.21.0_1
clamav-0.99.2_6
cmake-3.10.1
vim-8.0.1424
#
If I try to install it with NCURSES option on:
# cd /usr/ports/emulators/qemu
# make install clean
Dependency error: this port wants the ncurses library from the FreeBSD
base system. You can't build against it, while a newer
version is installed by a port.
Please deinstall the port or undefine WITH_NCURSES_BASE.
*** Error code 1
Stop.
make[1]: stopped in /usr/ports/emulators/qemu
*** Error code 1
Stop.
make: stopped in /usr/ports/emulators/qemu
#
If I disable ncurses as an option qemu is installed without it. It doesn't help to change Makefile to have NCURSES_USES=ncurses, even with the NCURSES option toggled it gets configured without them.
It seems qemu port has problems enabling ncurses when ncurses are installed from the port.
Actually now I'm not able to compile it with ncurses support at all. I've uninstalled ncurses from ports and tried to install qemu with ncurses on: # pkg info |grep curs # cd /usr/ports/emulators/qemu # make configure ===> License GPLv2 accepted by the user ===> qemu-2.9.0_1 depends on file: /usr/local/sbin/pkg - found => qemu-2.9.0.tar.bz2 doesn't seem to exist in /usr/ports/distfiles/qemu/2.9.0. => Attempting to fetch http://wiki.qemu.org/download/qemu-2.9.0.tar.bz2 qemu-2.9.0.tar.bz2 100% of 27 MB 4265 kBps 00m06s ===> Fetching all distfiles required by qemu-2.9.0_1 for building ===> Extracting for qemu-2.9.0_1 => SHA256 Checksum OK for qemu/2.9.0/qemu-2.9.0.tar.bz2. ===> Patching for qemu-2.9.0_1 ===> Applying FreeBSD patches for qemu-2.9.0_1 ===> qemu-2.9.0_1 depends on executable: gmake - found ===> qemu-2.9.0_1 depends on package: pkgconf>=1.3.0_1 - found ===> qemu-2.9.0_1 depends on executable: bison - found ===> qemu-2.9.0_1 depends on file: /usr/local/bin/python2.7 - found ===> qemu-2.9.0_1 depends on executable: python2 - found ===> qemu-2.9.0_1 depends on file: /usr/local/libdata/pkgconfig/pixman-1.pc - found ===> qemu-2.9.0_1 depends on package: perl5>=5.24<5.25 - found ===> qemu-2.9.0_1 depends on shared library: libnettle.so - found (/usr/local/lib/libnettle.so) ===> qemu-2.9.0_1 depends on shared library: libfontconfig.so - found (/usr/local/lib/libfontconfig.so) ===> qemu-2.9.0_1 depends on shared library: libfreetype.so - found (/usr/local/lib/libfreetype.so) ===> qemu-2.9.0_1 depends on shared library: libvdeplug.so - found (/usr/local/lib/libvdeplug.so) ===> qemu-2.9.0_1 depends on shared library: libcairo.so - found (/usr/local/lib/libcairo.so) ===> qemu-2.9.0_1 depends on shared library: libglib-2.0.so - found (/usr/local/lib/libglib-2.0.so) ===> qemu-2.9.0_1 depends on shared library: libintl.so - found (/usr/local/lib/libintl.so) .. .. CFLAGS -g -O2 -pipe -fno-omit-frame-pointer -fstack-protector -fno-strict-aliasing QEMU_CFLAGS -I/usr/local/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -pthread -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -I/usr/ports/emulators/qemu/work/qemu-2.9.0 -I/usr/local/include -DPREFIX=\""/usr/local\"" -Wno-string-plus-int -Wno-initializer-overrides -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/local/include LDFLAGS -Wl,--warn-common -m64 -g -fstack-protector -L"/usr/lib" -L"/usr/local/lib" curses support no It seems it's ignoring this toggle altogether. work/qemu-2.9.0/config.log shows: # QEMU configure log Tue Dec 26 21:45:15 CET 2017 # Configured with: './configure' '--localstatedir=/var' '--extra-ldflags=-L"/usr/lib"' '--extra-ldflags=-L"/usr/local/lib"' '--disable-libssh2' '--enable-debug' '--prefix=/usr/local' '--cc=cc' '--enable-docs' '--disable-kvm' '--disable-linux-user' '--disable-linux-aio' '--disable-xen' '--enable-debug-info' '--python=/usr/local/bin/python2.7' '--extra-cflags=-I/usr/ports/emulators/qemu/work/qemu-2.9.0 -I/usr/local/include -DPREFIX=\""/usr/local\""' '--disable-curl' '--disable-gnutls' '--disable-gtk' '--disable-vte' '--disable-vnc-jpeg' '--disable-opengl' '--disable -vnc-png' '--disable-vnc-sasl' '--disable-usb-redir' '--disable-sdl' '--disable-strip' # It's not toggling curses at all. OS changed since I posted this bug, now I'm on 11.1-RELEASE-p1 r324609, ports sourced from head (portsnap) . I played around with this issue and found out why it's failing to detect ncurses.
The configure script has a code to check if the (n)curses is installed on the system. In my case it sets the include and lib env variables:
curses_inc_list="$($pkg_config --cflags ncursesw 2>/dev/null):-I/usr/include/ncursesw:"
curses_lib_list="$($pkg_config --libs ncursesw 2>/dev/null):-lncursesw:-lcursesw"
It creates small curses demo C program and tries to compile it. It fails because:
test.c:12:3: warning: implicit declaration of function 'add_wch' is invalid in C99 [-Wimplicit-function-declaration]
add_wch(WACS_DEGREE);
^
test.c:12:11: error: use of undeclared identifier 'WACS_DEGREE'
add_wch(WACS_DEGREE);
As the compilation fails on all possible flags configure script assumes no curses are installed.
This function is defined in curses.h. This program fails using either system libs or port ones (ncurses 6.0.0s20171007_1).
I cheched the defined function in both libs (output in grep is omitted for brevity):
# readelf -Wa /usr/lib/libncursesw.so | grep add_wch
534: 000000000001cac0 26 FUNC GLOBAL DEFAULT 11 add_wch
# readelf -Wa /usr/local/lib/libncursesw.so.6 | grep add_wch
523: 00000000000128b0 26 FUNC GLOBAL DEFAULT 11 add_wch
After some playing around I tried to downgrade the ncurses to ncurses-6.0.0s20171007 (version that pkg installs) and it worked! I was able to compile the demo program and qemu found the ncurses.
Comparing the include/lib variables I noticed following preprocessor macros are included in the previous versions:
-D_BSD_TYPES -D__BSD_VISIBLE -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600
And one required to compile the demo program being: -D_XOPEN_SOURCE=600
As a quick dirty test I patched the configure script to expand QEMU_CFLAGS just to include the above macro. qemu compilation failed, I think this macro unleashes other dependencies as the generated code differs.
But as the qemu was working on previous version of ncurses I tried to expand the QEMU_CFLAGS variable in configure script by:
QEMU_CFLAGS="-D_BSD_TYPES -D__BSD_VISIBLE -D_XOPEN_SOURCE=600 $QEMU_CFLAGS"
And compilation succeeded.
Now the Makefile is using ncurses:base and therefore I can't use devel/ncurses. So we have a catch 22 problem here. Some ports do need it.
Leaving my patched configure script I did following tests:
1) change ncurses:base into ncurses in emulators/qemu/Makefile and tried to compile it. Problem is that pkg-config returns paths for devel/ncurses first but final
binary is linked against system one.
qemu starts but output is garbled ( which was why I opened bug in the first place )
2) I used the original Makefile, patched configure script but temporarily removed devel/ncurses from system.
qemu works as expected (hooray!)
3) Installed back devel/ncurses
all works as expected, pkg check --shlibs is OK
Bottom line: those removed macros from ncurses-6.0.0s20171007 are still needed. configure script has to be able to generate curses_inc_list/curses_lib_list depending on
whether user wants ncurses from base or from devel/ncurses.
Technically there's no problem using either curses - both system and port ones work flawlessly. I tested this on my machine. We can define option in qemu/Makefile for user to choose no ncurses, ncurses base or ncurses ports (radio button choice). Depending on the choice proper curses_inc_list/curses_lib_list can be generated in configure script. Right now the problem is the staging phase where all is linked together, linker flags are: -L"/usr/lib" -L"/usr/local/lib" Which is a problem as linker will find system ncurses first all the time. Changing the order of the libs when port ncurses is selected would be an option (this is how I tested it) but I don't like that approach as it could introduce other problems ( do we have other libraries such as ncurses in both locations? ) I'm not sure if there's a better way though ; never did customize any port before, so I need to get familiar with the process a bit more. |
