x86 Emulation: UTM issues

Anything about Mac emulation that does not belong in the above categories.

Moderators: Cat_7, Ronald P. Regensburg

Post Reply
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

x86 Emulation: UTM issues

Post by adespoton »

Well, I've jumped in and am running UTM on an M1 Mac running Monterey.

I figured out how to launch individual VMs programmatically using

Code: Select all

open utm://start?device=<device%20name>
which means I can use my regular Finder-clickable icons to launch everything.

For x86 emulation, I'm going the route of using TianoCore from the OpenCore repo and the following EFI booters from OpenCore:
EFI-MODERN: macOS 10.15 and newer supported
EFI-LEGACY: Mac OS X 10.6 through 10.14 supported
EFI-i386: Mac OS X 10.4 through 10.7 supported

Basic instructions are on Mykola's Blog, but I found I needed to make some tweaks.

UTM has updated since the blog was written in January, so now under System:Show Advanced Settings, you can select Penryn for CPU, and select sse4.1, sse4.2 and ssse3 CPU flags there as well and ignore the changes on the QEMU page.

However, I've had no luck with automatic keyboard detection, and so have added

Code: Select all

-usbdevice keyboard
to the QEMU list.

This brings me to the last bits that I haven't figured out yet:
1. Networking. The blog says to use vmxnet3, but I've had zero luck with that. The one that seems to sometimes work for me is usb-net -- but then it randomly vanishes while the emulated machine is running, and I need to tear everything down and reboot my Mac to get it back.
2. Legacy boot. I've only succeeded in getting 10.7 and later running; 10.4 x86, 10.5 x86 and 10.6 (plus server versions) are being stubborn. I figure there's some CPU flags I don't have right yet, or some tweaks that need to be added to EFI that I haven't figured out yet. I'll have to go over what I did to get these booting under VMWare, Parallels and VirtualBox and that should help on here.

If anyone wants the UTM config files I'm successfully using, let me know and I'll post them on my GitHub page.
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: x86 Emulation: UTM issues

Post by adespoton »

I've figured out the 10.4-10.6 issue partially... have to use i386 instead of x86_64 as the CPU type, and have the OVMF as a disk image instead of type BIOS. This allows me to at least select the boot drive and start the boot process, bringing up an apple.
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

adespoton wrote: Thu Dec 02, 2021 6:06 pm If anyone wants the UTM config files I'm successfully using, let me know and I'll post them on my GitHub page.

Thanks for your hard work and for uploading them! I created a Mountain Lion VM in QEMU (on Intel macOS Ventura host) based on your config. Works a treat!

Originally it was a Snow Leopard Server VM on Parallels that I created on my old Hackintosh with HDD years ago. After moving it to the new one with SSD just a few years ago I had to run the host with UnSolid.kext injected to prevent Parallels from prepending “APPLE SSD” to the model name of the VM’s virtual hard disk. ’Cause changing that (or a bunch of other parameters of the VM’s system) triggered the registration dialog of a certain application that tied its activation to those hardware parameters when I installed it years ago.

Snow Leopard (Server) doesn’t run on UTM/QEMU yet, so I updated the VM to Mountain Lion. I populated the “Platform Info” fields of OpenCore’s config.plist with the details gathered from the old VM. Same with some stuff in the QEMU startup command file (the hard disk attributes “Model”, “Revision”, and “Serial” can’t be added to UTM, so I had to use QEMU directly). I changed the network card to the e1000-82545em submodel to reflect the vendor and device ID my activated software expects.

And hey, it runs without grouching!
(I can even shut down the VM with my keyboard’s power button …)

Just one question though:
How to change the screen dimensions?
They seem to be hardcoded.



EDIT:
By the way: At apple.com there are new installers available for Lion and Mountain Lion. No trouble with expired certificates:
https://support.apple.com/kb/DL2077
https://support.apple.com/kb/DL2076
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

I’ve found this: viewtopic.php?t=11094

But currently I have only one screen resolution available in System Preferences, namely 1280×720. Since qemu_vga.ndrv, as referenced under the link above, contains more resolutions, it seems to be the wrong file.

Which is the correct one for hex editing the resolution?


EDIT:
This is my start command for QEMU, in case it is relevant:

Code: Select all

#!/bin/bash
cd "$(dirname "$0")"

port=45001

qemu-system-x86_64 \
-L /usr/local/Cellar/qemu-x86/7.1.0/share/qemu \
-nodefaults \
-vga none \
-device e1000-82545em,mac=xx:xx:xx:xx:xx:xx,netdev=net0 \
-netdev user,id=net0 \
-cpu Penryn,+ssse3,+sse4.1,+sse4.2 \
-smp cpus=2,sockets=1,cores=2,threads=1 \
-machine q35,vmport=off,vmport=off \
-accel tcg,tb-size=512 \
-global ICH9-LPC.disable_s3=1 \
-m 2048 \
-usb \
-device usb-tablet,bus=usb-bus.0 \
-device usb-mouse,bus=usb-bus.0 \
-device usb-kbd,bus=usb-bus.0 \
-device ich9-usb-ehci1,id=usb-controller-0 \
-device ich9-usb-uhci1,masterbus=usb-controller-0.0,firstport=0,multifunction=on \
-device ich9-usb-uhci2,masterbus=usb-controller-0.0,firstport=2,multifunction=on \
-device ich9-usb-uhci3,masterbus=usb-controller-0.0,firstport=4,multifunction=on \
-bios "./OVMF.bin" \
-device "ide-hd,bus=ide.0,drive=drive3,bootindex=1,model=Snow Leopard Server-0,ver=F.A5JNMR,serial=XXXXXXXXXXXXXXXXXXXX" \
-drive "if=none,media=disk,id=drive3,file=./Snow Leopard Server-0.qcow2,discard=unmap,detect-zeroes=unmap" \
-name "- Van Dale - Port $port" \
-monitor tcp:localhost:$port,server,nowait \
-uuid AABBCCDD-EEFF-GGHH-IIJJ-KKLLMMNNOOPP \
-rtc base=localtime \
-device VGA,vgamem_mb=128 \
-usbdevice keyboard
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

Found it:
https://github.com/kholia/OSX-KVM/blob/ ... n-opencore


Much easier than I thought – no hex editing and such.
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: x86 Emulation: UTM issues

Post by adespoton »

mabam wrote: Sun Dec 11, 2022 6:39 pm Found it:
https://github.com/kholia/OSX-KVM/blob/ ... n-opencore


Much easier than I thought – no hex editing and such.
Glad you found it :) I should probably add a note to my configs about that too.

Let me know if you have any ideas on getting 10.4-10.6 x86 to work on an M-series CPU. The latest OpenCore version adds some more support that might be enough to get at least 10.6 booting, but I've had no time this autumn to dig into it and produce a usable updated OpenCore image.
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

10.6 is more lightweight than 10.8, would be better for my ported VM as well. The original in Parallels is 10.6 Server and I had to update it for QEMU. Couldn’t update to 10.7 as its installer refused to run without me also providing the 10.7 installer for the server applications. The 10.8 installer doesn’t have that restriction and just deactivates the server stuff.

Not sure, but I might have a look if I can get 10.6 to work. Regarding that, when you say that the latest OpenCore version adds some more support, is there anything in particular you have in mind?

adespoton wrote: Mon Dec 12, 2022 4:10 pm Glad you found it :) I should probably add a note to my configs about that too.
That’s a good idea.
You may also want to point to the following links:
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

adespoton wrote: Thu Dec 02, 2021 6:06 pm I figured out how to launch individual VMs programmatically using

Code: Select all

open utm://start?device=<device%20name>
which means I can use my regular Finder-clickable icons to launch everything.

I tested this line in Terminal but it only launched UTM, not the VM.
By “device” you mean the name given to the VM in UTM, right?
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: x86 Emulation: UTM issues

Post by adespoton »

mabam wrote: Wed Dec 14, 2022 2:06 am
adespoton wrote: Thu Dec 02, 2021 6:06 pm I figured out how to launch individual VMs programmatically using

Code: Select all

open utm://start?device=<device%20name>
which means I can use my regular Finder-clickable icons to launch everything.

I tested this line in Terminal but it only launched UTM, not the VM.
By “device” you mean the name given to the VM in UTM, right?
Yes; it should launch UTM if it isn't running and then run the named VM. It works for me for UTM 2.x through 4.x. You may need to play around with quotes/no quotes to get it to pass the string in correctly. UTM has an entire collection of URI handlers for different things -- you can pause/start/stop/suspend and also have access to the QEMU monitor via the serial command.

Details here: https://github.com/utmapp/UTM/blob/main ... MAPI.swift
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

I have my Terminal set to use bash (latest version installed via Homebrew). On a different Mac also running Ventura but using zsh, the above command throws an errors as it sees the “?” in it as a wild card:

Code: Select all

open utm://start?device=<device%20name>
zsh: no matches found: utm://start?device=<device%20name>

Putting quotes around fixes the error

Code: Select all

open utm://'start?device=<device%20name>'
but still only opens UTM, not the VM.

Since UTM 4.1.2 released a few days ago there’s a command line tool included. Using that with the following command does the job:

Code: Select all

/Applications/UTM.app/Contents/MacOS/utmctl start <device%20name>

On a different VM ran in QEMU directly (not via UTM) I’m using Karabiner Elements to grab the Power Key when QEMU is frontmost and send the following to the VM’s monitor:

Code: Select all

TCPPort=45000
export TCPPort
expect -c 'spawn nc localhost $::env(TCPPort) -w 1
expect "(qemu) "
send -- "sendkey power\r"
interact;'
(I have attached the TCP port to the VM’s name as an argument when executing the QEMU binary. The variable “TCPPort” is actually parsed from the running VM’s window title using osascript.)


Now I’m trying to figure out how to send a command to the QEMU monitor of the VM in UTM. Because for the UTM VM I want to do the same for cmnd-Q, cmnd-W, etc. (so these aren’t executed by UTM but by its VM).

EDIT:
I does work attaching the argument

Code: Select all

-monitor tcp::45000,server,nowait
in the VM’s UTM settings –> QEMU and using the multi line command above.
However, with more than one VM in UTM each needs to have their own TCP port number. In that case that port number can be added to the VM’s name and parsed by substituting the first line of the multi line command by:

Code: Select all

TCPPort=$(osascript -e 'return word -1 of (name of window 1 of application "UTM" as text)' | grep -o '[^ ]*$')

But what I’m wondering:
Isn’t there an easier way to connect to the VM’s monitor by just using the VM’s name with utmctl or UTM’s AppleScript terminology?
It should be possible as UTM’s AppleScript dictionary states:
  • serial interfaceenum : Serial port interface.
    ptty : Pseudo TTY port.
    tcp : TCP port.
    unavailable : Serial interface is currently unavailable or is in use by the GUI.
But I can’t figure it out somehow.
Last edited by mabam on Mon Dec 26, 2022 2:45 am, edited 3 times in total.
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

adespoton wrote: Wed Dec 14, 2022 3:51 pm Yes; it should launch UTM if it isn't running and then run the named VM. It works for me for UTM 2.x through 4.x. You may need to play around with quotes/no quotes to get it to pass the string in correctly. UTM has an entire collection of URI handlers for different things -- you can pause/start/stop/suspend and also have access to the QEMU monitor via the serial command.

Details here: https://github.com/utmapp/UTM/blob/main ... MAPI.swift

By the way: The above link changed with the last version of UTM. It is now https://github.com/utmapp/UTM/blob/main ... MCtl.swift.
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: x86 Emulation: UTM issues

Post by adespoton »

mabam wrote: Sun Dec 25, 2022 7:30 pm I have my Terminal set to use bash (latest version installed via Homebrew). On a different Mac also running Ventura but using zsh, the above command throws an errors as it sees the “?” in it as a wild card:

Code: Select all

open utm://start?device=<device%20name>
zsh: no matches found: utm://start?device=<device%20name>

Putting quotes around fixes the error

Code: Select all

open utm://'start?device=<device%20name>'
but still only opens UTM, not the VM.

Since UTM 4.1.2 released a few days ago there’s a command line tool included. Using that with the following command does the job:

Code: Select all

/Applications/UTM.app/Contents/MacOS/utmctl start <device%20name>

On a different VM ran in QEMU directly (not via UTM) I’m using Karabiner Elements to grab the Power Key when QEMU is frontmost and send the following to the VM’s monitor:

Code: Select all

TCPPort=45000
export TCPPort
expect -c 'spawn nc localhost $::env(TCPPort) -w 1
expect "(qemu) "
send -- "sendkey power\r"
interact;'
(I have attached the TCP port to the VM’s name as an argument when executing the QEMU binary. The variable “TCPPort” is actually parsed from the running VM’s window title using osascript.)


Now I’m trying to figure out how to send a command to the QEMU monitor of the VM in UTM. Because for the UTM VM I want to do the same for cmnd-Q, cmnd-W, etc. (so these aren’t executed by UTM but by its VM).

EDIT:
I does work attaching the argument

Code: Select all

-monitor tcp::45000,server,nowait
in the VM’s UTM settings –> QEMU and using the multi line command above.
However, with more than one VM in UTM each needs to have their own TCP port number. In that case that port number can be added to the VM’s name and parsed by substituting the first line of the multi line command by:

Code: Select all

TCPPort=$(osascript -e 'return word -1 of (name of window 1 of application "UTM" as text)' | grep -o '[^ ]*$')

But what I’m wondering:
Isn’t there an easier way to connect to the VM’s monitor by just using the VM’s name with utmctl or UTM’s AppleScript terminology?
It should be possible as UTM’s AppleScript dictionary states:
  • serial interfaceenum : Serial port interface.
    ptty : Pseudo TTY port.
    tcp : TCP port.
    unavailable : Serial interface is currently unavailable or is in use by the GUI.
But I can’t figure it out somehow.
It might be worth bringing this up on the Discord channel; documentation has been rather spotty and this area has been moving pretty fast lately -- I've been away, and haven't kept up with it myself.
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

I’m aware your UTM config files on https://github.com/adespoton/utmconfigs are for emulation on Apple Silicon hardware, not for virtualisation on an x86_64 host. But I’m running into an issue: When checking “Use Hypervisor” in UTM this changes -accel tcg,tb-size=512 in the qemu command to -accel hvf. That makes the VM much faster even during boot, but it also leaves me with it not responding to mouse and keyboard anymore.

Do you have any suggestions?
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: x86 Emulation: UTM issues

Post by adespoton »

mabam wrote: Fri Nov 03, 2023 11:47 am I’m aware your UTM config files on https://github.com/adespoton/utmconfigs are for emulation on Apple Silicon hardware, not for virtualisation on an x86_64 host. But I’m running into an issue: When checking “Use Hypervisor” in UTM this changes -accel tcg,tb-size=512 in the qemu command to -accel hvf. That makes the VM much faster even during boot, but it also leaves me with it not responding to mouse and keyboard anymore.

Do you have any suggestions?
I wasn't aware that the hypervisor was even usable on x86_64... looks like Osy tied it back to QEMU/KVM hypervisor? Probably best to check the UTM GitHub instance. I know there have been tweaks recently related to the hypervisor code and input devices, but I think that was only for the Apple Silicon hypervisor.
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: x86 Emulation: UTM issues

Post by mabam »

Actually, it’s the same whether I run the VM in UTM or in QEMU directly.

But it does work for a VM I created following https://web.archive.org/web/20230209014 ... rd-utm.php, both, in UTM and QEMU. It obviously uses a Snow Leopard distro as plain Snow Leopard would crash in QEMU. But that shouldn’t be the reason why HVF works.

Not sure how to amend your UTM config accordingly as yours uses a BIOS file while the above doesn’t seem to do that.
User avatar
adespoton
Forum All-Star
Posts: 4227
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: x86 Emulation: UTM issues

Post by adespoton »

mabam wrote: Fri Nov 03, 2023 6:00 pm Actually, it’s the same whether I run the VM in UTM or in QEMU directly.

But it does work for a VM I created following https://web.archive.org/web/20230209014 ... rd-utm.php, both, in UTM and QEMU. It obviously uses a Snow Leopard distro as plain Snow Leopard would crash in QEMU. But that shouldn’t be the reason why HVF works.

Not sure how to amend your UTM config accordingly as yours uses a BIOS file while the above doesn’t seem to do that.
My x86 config files are designed specifically to work with the TCG emulator; if you're using HVF, you should just be able to pass through the host config and make it work in most situations. There may be a few specific virtual device values that need to be set though for specific bits of hardware, just like there are for VirtualBox, Parallels and VMWare Fusion.

If you're using HVF, the config files will be more of a hinderance than a help, as they're specifically designed to get TCG working on an Apple Silicon Mac with older macOS versions.

On X86_64, I found it significantly easier to just use one of the above mentioned VMs, which will run everything from 10.4.x through 14.x (although VMWare Fusion requires some tweaking for 10.4-10.6 and VirtualBox needs some additional parameters set).

TCG in QEMU 8.x also seems to have some changes that can break OS X 10.8 through 10.12; seems to be something to do with video handling. UTM 4.3+ has rolled back to QEMU 7.x to mitigate these issues (but then introduces a few new bugs of its own).
Post Reply