Bug 277230 - emulators/wine: Testing experimental wow64 mode
Summary: emulators/wine: Testing experimental wow64 mode
Status: Closed Not Accepted
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Gerald Pfeifer
URL: https://github.com/shkhln/freebsd-win...
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-22 22:17 UTC by Alexander Vereeken
Modified: 2024-08-01 06:05 UTC (History)
7 users (show)

See Also:
gerald: maintainer-feedback-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Vereeken freebsd_triage 2024-02-22 22:17:29 UTC
This is an excerpt of the work from Alex S <iwtcex@gmail.com> for using the new wow64 mode on FreeBSD.

During my tests so far I have only encountered minor problems and can say that so far the functionality is good.

The discussion about issues takes mainly place in the FreeBSD Discord.

Differential Revision: https://reviews.freebsd.org/D44028
Comment 1 Alex S 2024-02-23 14:45:20 UTC
This was never intended to submitted here, it's just my working copy.
Comment 2 Alex S 2024-02-23 14:56:46 UTC
We can keep this issue for tracking, I suppose. But we are not switching to the _experimental_ wow64 mode in 9.0.
Comment 3 Alexander Vereeken freebsd_triage 2024-02-23 15:21:49 UTC
Sure, i'll change this ticket accordingly, i was probably a bit too hasty, sorry.
Comment 4 Gerald Pfeifer freebsd_committer freebsd_triage 2024-02-23 15:30:03 UTC
How about doing this work on wine-devel, first and foremost? That's one
of the usecases we have -devel ports for. :-)
Comment 5 Alex S 2024-02-23 16:54:23 UTC
(In reply to Gerald Pfeifer from comment #4)

> How about doing this work on wine-devel

Maybe later.
Comment 6 Brendan Shanks 2024-02-23 18:47:49 UTC
Great! I'll give this a test when I have a chance. I briefly tried to add this support but my understanding of GDT/LDT is a bit lacking.
Ideally we can polish this up and get it upstream.

And yes the wow64 mode is still experimental, if this gets added to the port it should probably be a (defaulted-off) config option.
Comment 7 alt2600 2024-02-25 23:24:08 UTC
(In reply to Alexander Vereeken from comment #0)

since I haven't had any luck with wine nor wine-devel since 7 was removed I gave this a go with the raw diff against my tree. It created a prefix, wine internal programs work ok. I was able to update mono to 9.0.0 and install gecko 2.47.4 . but when I try to run 32 bit or 64 bit code I generally get a stack error or illegal instruction. I'm not sure exactly what this all means but maybe someone would understand. Still the closest to having wine going again. This was on a westmere running amd64 13.2 . Not sure if I'd have better luck with 14.0, when 13.3 comes out I'll likely update, but if the old ones or this may benefit from 14.0 and the newer libc/libc++ runtimes I'm game. Just need to plan out the build time of ports is all. Have folks had luck even with the old way on both 13.2 and 14.0 running separate 32bit and 64bit builds? Not sure if I'm just having issues with ASLR or some other security features, or if my machine is just too old. I mean it is old, not gonna deny that.


anyway 
wrfsbase 
seems to be the offending instruction

https://www.felixcloutier.com/x86/wrfsbase:wrgsbase
this is all I can see about it, but it seems to definitely imply it isn't valid for protected mode, which I believe the kernel runs in. 


32 bit program:
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/local/bin/wine Lunar\ Magic.exe
process 23007 is executing new program: /usr/local/bin/wine
010c:fixme:ntdll:create_logical_proc_info stub
010c:fixme:ntdll:init_cpu_info Failed to get logical processor information, status 0xc0000002.
010c:err:environ:init_peb starting L"Z:\\home\\mydude\\Projects\\Console\\SNES\\Lunar Magic\\Lunar Magic.exe" in experimental wow64 mode

Program received signal SIGILL, Illegal instruction.
Privileged opcode.
0x00000000034d18e4 in __wine_syscall_dispatcher () from /usr/local/bin/../lib/wine/x86_64-unix/ntdll.so
(gdb) disassemble
Dump of assembler code for function __wine_syscall_dispatcher:
   0x00000000034d17e0 <+0>:     mov    %gs:0x328,%rcx
   0x00000000034d17e9 <+9>:     pop    0x70(%rcx)
   0x00000000034d17ec <+12>:    pushf
   0x00000000034d17ed <+13>:    pop    0x80(%rcx)
   0x00000000034d17f3 <+19>:    movl   $0x0,0xb4(%rcx)
   0x00000000034d17fd <+29>:    mov    %rax,(%rcx)
   0x00000000034d1800 <+32>:    mov    %rbx,0x8(%rcx)
   0x00000000034d1804 <+36>:    mov    %rdx,0x18(%rcx)
   0x00000000034d1808 <+40>:    mov    %rsi,0x20(%rcx)
   0x00000000034d180c <+44>:    mov    %rdi,0x28(%rcx)
   0x00000000034d1810 <+48>:    mov    %r12,0x50(%rcx)
   0x00000000034d1814 <+52>:    mov    %r13,0x58(%rcx)
   0x00000000034d1818 <+56>:    mov    %r14,0x60(%rcx)
   0x00000000034d181c <+60>:    mov    %r15,0x68(%rcx)
   0x00000000034d1820 <+64>:    mov    %cs,0x78(%rcx)
   0x00000000034d1823 <+67>:    mov    %rsp,0x88(%rcx)
   0x00000000034d182a <+74>:    mov    %ss,0x90(%rcx)
   0x00000000034d1830 <+80>:    mov    %rbp,0x98(%rcx)
   0x00000000034d1837 <+87>:    subq   $0xe,0x70(%rcx)
   0x00000000034d183c <+92>:    mov    0xb0(%rcx),%r14d
   0x00000000034d1843 <+99>:    test   $0x3,%r14d
   0x00000000034d184a <+106>:   je     0x34d18a8 <__wine_syscall_dispatcher+200>
   0x00000000034d184c <+108>:   mov    $0x7,%eax
   0x00000000034d1851 <+113>:   xor    %edx,%edx
   0x00000000034d1853 <+115>:   mov    %rdx,0x2c0(%rcx)
   0x00000000034d185a <+122>:   mov    %rdx,0x2c8(%rcx)
   0x00000000034d1861 <+129>:   mov    %rdx,0x2d0(%rcx)
   0x00000000034d1868 <+136>:   test   $0x2,%r14d
   0x00000000034d186f <+143>:   je     0x34d189e <__wine_syscall_dispatcher+190>
   0x00000000034d1871 <+145>:   mov    %rdx,0x2d8(%rcx)
   0x00000000034d1878 <+152>:   mov    %rdx,0x2e0(%rcx)
   0x00000000034d187f <+159>:   mov    %rdx,0x2e8(%rcx)
   0x00000000034d1886 <+166>:   mov    %rdx,0x2f0(%rcx)
   0x00000000034d188d <+173>:   mov    %rdx,0x2f8(%rcx)
   0x00000000034d1894 <+180>:   xsavec64 0xc0(%rcx)
   0x00000000034d189c <+188>:   jmp    0x34d18b0 <__wine_syscall_dispatcher+208>
   0x00000000034d189e <+190>:   xsave64 0xc0(%rcx)
   0x00000000034d18a6 <+198>:   jmp    0x34d18b0 <__wine_syscall_dispatcher+208>
   0x00000000034d18a8 <+200>:   fxsave64 0xc0(%rcx)
   0x00000000034d18b0 <+208>:   lea    0x98(%rcx),%rbp
   0x00000000034d18b7 <+215>:   mov    0x28(%rsp),%r12
   0x00000000034d18bc <+220>:   mov    0x30(%rsp),%r13
   0x00000000034d18c1 <+225>:   lea    0x38(%rsp),%r15
   0x00000000034d18c6 <+230>:   mov    %rcx,%rsp
   0x00000000034d18c9 <+233>:   test   $0xc,%r14d
   0x00000000034d18d0 <+240>:   je     0x34d18e9 <__wine_syscall_dispatcher+265>
   0x00000000034d18d2 <+242>:   mov    $0x13,%rsi
   0x00000000034d18d9 <+249>:   mov    %esi,%fs
   0x00000000034d18db <+251>:   mov    %gs:0x320,%rsi
=> 0x00000000034d18e4 <+260>:   wrfsbase %rsi
--Type <RET> for more, q to quit, c to continue without paging--


64bit programs seems to stop here with a different error:
Thread 3 received signal SIGUSR1, User defined signal 1.
Sent by thr_kill() from pid 22987 and user 1029.
[Switching to LWP 680201 of process 23068]
_read () at _read.S:4
4       PSEUDO(read)
(gdb) disassemble
Dump of assembler code for function _read:
   0x00000000031ca4c0 <+0>:     mov    $0x3,%eax
   0x00000000031ca4c5 <+5>:     mov    %rcx,%r10
=> 0x00000000031ca4c8 <+8>:     syscall
   0x00000000031ca4ca <+10>:    jb     0x31c72a8 <.cerror>
   0x00000000031ca4d0 <+16>:    ret
End of assembler dump.

lldb gives a little more information, but not much for 64bit

This version of LLDB has no plugin for the language "assembler". Inspection of frame variables will be limited.
Process 23084 stopped
* thread #3, name = 'wine', stop reason = signal SIGUSR1
    frame #0: 0x00000000031ca4c8 libc.so.7`__sys_read at _read.S:4
   1    /* @generated by libc/sys/Makefile.inc */
   2    #include "compat.h"
   3    #include "SYS.h"
-> 4    PSEUDO(read)
   5            .section .note.GNU-stack,"",%progbits


wine control
wine notepad
all run as expected. right now things are in windows 10 mode. switching to windows 7 doesn't make a difference
Comment 8 Alex S 2024-02-25 23:31:38 UTC
(In reply to alt2600 from comment #7)

> wrfsbase seems to be the offending instruction

With my latest revision of the patches or before it?
Comment 9 alt2600 2024-02-26 00:02:09 UTC
(In reply to Alex S from comment #8)

here is the link
https://reviews.freebsd.org/D44028?download=true

The download raw link from the initial post. I did look at the github link, but I couldn't recall how to do a diff from the 8.1 to the latest there from the website without cloning, and wanted to try the simple way first. If there is updated patches I'd love to try them. anyway a diff can be attached here, or should I just clone 

https://github.com/shkhln/freebsd-wine/tree/master/emulators/wine-devel

and make a diff between head and the 8.1 commit hash in it? just not sure where the latest would be.
Comment 10 Alex S 2024-02-26 00:08:58 UTC
What diff? It's a complete port.
Comment 11 alt2600 2024-02-26 14:47:47 UTC
the ports git diff that comes from this link to take wine 8.1 to wine 9.0 with experimental wow64

https://reviews.freebsd.org/D44028?download=true

its a ports diff, its not staged on a new port, its staged against emulators/wine . I wanted to try it, given nothing else works for me right now.
Comment 12 Alex S 2024-02-26 15:05:18 UTC
(In reply to alt2600 from comment #11)

My code is on GitHub, that other link as far as I'm concerned doesn't exist.
Comment 13 alt2600 2024-02-27 01:42:03 UTC
(In reply to Alex S from comment #12)

not quite a straight diff, but just cloned and replaced wine-devel in my tree, didn't want to work out the rebase, but you on the right path for sure

github code has Lunar Magic working in 32bit mode fine. I see your ASM updates are checking so I see this message in console:
CPU doesn't support wrfsbase
so guessing maybe thats a "newer" ASM feature, least "newer" from my old junk.

only glitch I saw was window came up with a copy of the underlying desktop, resizing window redrew the window correctly, might have always been glitched, maybe an issue in wine, I thought apps in windows were supposed to internally call win api redraw window when they are ready for input. just being thorough, don't see this as a problem per-say.ADFOpus didn't exhibit this, drew normally, likely a issue with the old copy of Lunar Magic I have.

64 bit still throwing Illegal instruction, for some stuff. maybe my old machine or old libc from 13.2, beyond my understanding of low level stuff. Seems it doesn't like the syscall setup is my only guess, or if I'm reading the right correctly potentially unicode in RCX???. googling this signal error has led to rpcbind service, which I don't use, but it network related I don't recall network ever working for me with wine in the past, just clicking on battlenet generally crashed Diablo 2 on me for instance in other wines. 

Thread 3 received signal SIGUSR1, User defined signal 1.
Sent by thr_kill() from pid 17282 and user 1024.
[Switching to LWP 710979 of process 17280]
_read () at _read.S:4
4       PSEUDO(read)
(gdb) disassemble
Dump of assembler code for function _read:
   0x00000002421a94c0 <+0>:     mov    $0x3,%eax
   0x00000002421a94c5 <+5>:     mov    %rcx,%r10
=> 0x00000002421a94c8 <+8>:     syscall
   0x00000002421a94ca <+10>:    jb     0x2421a62a8 <.cerror>
   0x00000002421a94d0 <+16>:    ret
End of assembler dump.
(gdb) i r
rax            0x3                 3
rbx            0x1002fea00         4298107392
rcx            0x2425a3008         9703141384
rdx            0x10                16
rsi            0x1002fea00         4298107392
rdi            0x16                22
rbp            0x1002fe9f0         0x1002fe9f0
rsp            0x1002fe9b8         0x1002fe9b8
r8             0x0                 0
r9             0x0                 0
r10            0x2425a3008         9703141384
r11            0x3c6c1fd40         16219503936
r12            0x16                22
r13            0x242812700         9705694976
r14            0x10                16
r15            0x1002fea00         4298107392
rip            0x2421a94c8         0x2421a94c8 <_read+8>
eflags         0x246               [ PF ZF IF ]
cs             0x43                67
ss             0x3b                59
ds             0x3b                59
es             0x3b                59
fs             0x13                19
gs             0x1b                27
fs_base        0x3c6c22120         16219513120
gs_base        0x7ffa0000          2147090432


64 bit notepad++ ran fine, so its def something up with this random game using Unity Engine.

I'll keep checking out your github, and seeing how things go of if I see total success, but thanks man!!! first time anything win32/win64 launched on my machine since wine7 was removed. not going to see if my backup wine7 prefix will update quite yet, but this is awesome.
Comment 14 Gerald Pfeifer freebsd_committer freebsd_triage 2024-03-03 14:13:37 UTC
I believe we agreed that this is too early for emulators/wine, and
Alex is still developing it.

A logical step in the FreeBSD Ports context would be tackling wine-devel
first, the wine port then at a later point.


In any case I see quite some changes which appear upstream-worthy; 
Alex and Alexander, that would be great to push, even if only some
bits.
Comment 15 alt2600 2024-07-26 21:30:12 UTC
(In reply to Gerald Pfeifer from comment #14)

Wish this would be reconsidered, as with the current work with wine 9.8 from about 2 months ago it is running most of the things it wasn't previously. win32 software GL still is slow, but I think this is mainly a known issue with wine and this kind of build. This is the only way I can use wine as no version in ports will function for me at all. I had some issue with games based on nwjs, but all the unity stuff is seemingly resolved. It was even launching and playing games that had never worked with wine for me in the past. I think if this could go into ports as like wine-experimental or something it is a way for folks like me to continue to use wine and work in the ports tree, instead of relying on local hackery. At some point this will be the required way, as support for i386 is removed. Not sure the status of the github from URL but the one directly from Alex's github is what I plan to run for now.
Comment 16 Gerald Pfeifer freebsd_committer freebsd_triage 2024-07-29 22:55:48 UTC
(In reply to alt2600 from comment #15)
> Wish this would be reconsidered, as with the current work with 
> wine 9.8 from about 2 months ago it is running most of the things
> it wasn't previously. 

Why not focus on wine-devel then? I just committed a patch that seems 
to help a number of users and plan on updating to Wine 9.13 this week.

> I think if this could go into ports as like wine-experimental or
> something it is a way for folks like me to continue to use wine 
> and work in the ports tree, instead of relying on local hackery. 

We don't have enough fingers on the keyboard to properly maintain the
Wine ports we already have.

(One problem is too much local hackery as opposed to seeing to get
upstream work on FreeBSD out of the box.)

Bug 280448 aims in the same direction, though I have what I consider
a more acceptable patch.
Comment 17 alt2600 2024-07-31 23:44:28 UTC
(In reply to Gerald Pfeifer from comment #16)

I understand where you are coming from. That PE build fix from PR 280448 you cited seemingly would be needed to even attempt function on my machine. 

For me, its 6-8 hours of poudriere building llvm15, llvm17, gcc12 or 13 (forgot these days) just to even attempt and not even getting a prefix created is a lot of time just to principally play diablo 2 once in a while. wine 8 has never worked for me, and it seems the PR is supposed to get back how wine 8 was working. Pulling down from Latest on the main repo always had enough lag that it would break wine while I waited for it to catch up in its build cycle, but when It was working I dealt with poudriere and didn't mind, only two compilers in those days were needed.

I don't know how I can help with the current wine-devel if I cannot even get a prefix. I don't understand enough of the low level detail in coding, to know more then maybe slightly more then the basics of things. Even when I have issues, others seem to have things work fine the times I've reported issues. I'll give it a whirl when the PE build issue hits ports, and I'll report if I have issues. Any maybe when I finally take the plunge with a nice shiny new threadripper system wine from ports will just work, if is an age issue, with my westmere antique. At some point the experimental WoW64 build will be default though, when Linux likely ends i386 multi-lib, which seems closer on the horizon then not these days. maybe even in Releng 15 or 16 32bit compat may require source builds.
Comment 18 Alex S 2024-08-01 06:05:20 UTC
Proton is not yet compatible with the new wow64 mode, among other things. I really can't afford switching before that.