[Bug]: Hang/crash in iPXE when resetting VM after loading saved state

[logiclrd]

Version

trunk/main

Host OS Type

Windows

Host OS name + version

Windows 10 22H2 Build 19045.6332

Host Architecture

x86

Guest OS Type

other

Guest Architecture

x86

Guest OS name + version

MS-DOS 6.22

Component

Other

What happened?

In testing the fix for #610, I encountered a crash while doing some basic exercising of the VM. I initially reported this issue with the full steps I followed that produced the crash, but I have since been able to investigate what minimally reproduces the crash. What it boils down to is that if the VM's state was saved in 7.2.10 and then restored in 7.2.97, then when Reset, the VM fails to boot, hanging or crashing in the iPXE phase:

The guest being MS-DOS may not even be relevant to the crash, but that is the configuration that has been tested and observed crashing at this time.

In subsequent testing, I have found that I can reproduce the crash with saved states from the same version. They do to be saved in a previous VirtualBox version.

With my configuration, VirtualBox is running using the native Hypervisor API ("green turtle" icon).

How can we reproduce this?

0. Have an MS-DOS 6.22 VM set up and VirtualBox-7.2.97-174214-Win.exe installed.
1. Run VirtualBox and start the VM.
2. Once it has booted to the DOS prompt, save the state of the VM.
3. Restore the VM.
4. Reset the VM (e.g. Host Key+R).

At this point, the VM begin the pre-boot steps, but when it gets to the iPXE portion, it either hangs or crashes. (If it hangs, resetting again rolls the dice again, and, in my testing, it will eventually crash. Usually doesn't take too many tries.)

It should be noted that the iPXE code is running even though "Network" is unchecked under "Boot Device Order" and no network cards are enabled.

Here is a VBox.log from a crash produced with these steps:

VBox.log

Here is an earlier VBox.log from a crash produced with additional testing exercising #610

VBox.crash.log

Did you upload all of your necessary log files, screenshots, etc.?

• Yes, I've uploaded all pertinent files to this issue.

[MichalN-O]

I have been able to reproduce the "wrong" display in 80X25.BAS, but if I run 320X200F.BAS, it either works or pops back into QBASIC with an overflow message, which I guess is the expected behavior. Resetting the VM is no problem.

When you first ran into the crash, were you using Hyper-V (NEM) or the native VirtualBox hypervisor?

[MichalN-O]

I was able to decode the stack trace from the VBox.crash.log you provided, but unfortunately I'm not really any wiser. It's not crashing anywhere near the VGA code, and it's not apparent that graphics is at all involved. Could you please retry with network booting disabled, i.e. no iPXE?

[logiclrd]

When you first ran into the crash, were you using Hyper-V (NEM) or the native VirtualBox hypervisor?

I appear to be set up to use the native API. Green turtle. 😛

[logiclrd]

Could you please retry with network booting disabled, i.e. no iPXE?

I checked the VM settings. Network boot is already unchecked in the boot order. I tried disabling the NIC entirely. If there's another way to disable iPXE more directly, let me know.

With this change, the boot still runs iPXE code:

A reset from this state didn't crash immediately, but instead hung the same way, but a second reset reproduced the crash.

[logiclrd]

Okay, I just had a few cycles to see about trimming irrelevant things, and I optimistically tried the leanest possible run: start up the VM in 7.2.10, immediately save state, upgrade to 7.2.97, restore state, reset. None of the stuff I did on the run where I originally produced the crash.

And the crash still happens.

So all of that stuff can be discarded, I think. 🙂

I will update the reproduction steps.

[logiclrd]

I think this means there's more than one bug here.

After restoring a previously-saved VM state:

1. The Reset gets hung up and sometimes crashes in the iPXE phase.
2. The 80X50.BAS text display is corrupted somehow.
3. The mode switch QuickBASIC performs when a math error occurs in the execution of 320X200F.BAS causes a hang.

2 and 3 might be related. It seems likely that 1 is not related to either of them.

[logiclrd]

I have updated the main issue description including reproduction steps.

[MichalN-O]

I don't think problems 2 and 3 are related because I have seen 2 but not 3. Actually I have an idea what 3 might be, and if I'm right then it's completely unrelated to saved states or VGA (it's probably related to FPU exception handling).

Does problem 2 occur even when not loading a 7.2 saved state? One problem with saved states is that sometimes emulation changes also require BIOS changes, in which case things just aren't going to work and the solution is to close the VM and start again. There could be some hardware/BIOS versioning and blah blah blah but it's just way too much work for near zero payoff.

[logiclrd]

I just tried starting the VM directly in the test build, and also saving the state in the test build and then restoring it, and the text mode corruption in 80X50.BAS does not reproduce. So, my interpretation is that this is very likely a minor incompatibility (with significant consequences under specific circumstances) in the exact format of the VGA emulation layer's state data between 7.2.10 and 7.2.97.

I just did several test runs, and have found that the iPXE hang/crash on reset happens in 7.2.97 if the VM was restored from a saved state, regardless of whether the saved state was from 7.2.97 or 7.2.10.

I also had one oddball test run where when I ran 320X200F.BAS, it hung in the same place, but unlike the previous testing, using the 7.2.10 saved state, this time it was responsive to Ctrl+Break. It shouldn't have hung in the first place, but when Ctrl+Break was sent, it responded:

Note that it did not switch to this screen until Ctrl+Break was sent, and it should have done immediately upon hitting the runtime error.

This was a run started from saved state, and after getting this, when I used Host+R to reset the VM, just like all the other runs from saved states, it encountered the iPXE issues. It hung on startup 3 times in a row, and on the 4th attempt, the VirtualBoxVM.exe process crashed.

I tried another run from a 7.2.97 saved state, and the Ctrl+Break responsiveness did not occur, but the iPXE hang/crash did.

[MichalN-O]

Could you please try this using the native VirtualBox hypervisor? I want to make sure that the problem is specific to running on top of Hyper-V because I still can't see any hangs/crashes.

Also, how exactly is your DOS VM configured? Could you maybe post your CONFIG.SYS? The memory manager etc. likely matters for the overflow handling.