Windows Server 2025: Windows Terminal: Crash in OpenConsole.exe (0xc0000094, division by zero) causes programs to fail with 0xC0000142. How to detect and work around this Windows Terminal version?

[denisbider]

Windows Terminal version

1.18.2401

Windows build number

10.0.26100.2314

Other Software

Bitvise SSH Server 9.39 installer
Bitvise SSH Client 9.39 installer

Steps to reproduce

This problem is frequently observed by users of Bitvise software who attempt to install on Windows Server 2025 before the Windows Terminal has updated to a more recent version. The problem has also been reported with Windows 11.

To reproduce:

Issue version 1:

• Ensure the installed Windows Terminal version is the one that ships with the OS (e.g. version 1.18 for Windows Server 2025)
• Download Bitvise SSH Server installer: https://bitvise.com/ssh-server-download
• Open Command Prompt in Windows Terminal with administrative rights
• Run "BvSshServer-Inst.exe -?" for command-line help
• Observe that the Windows Terminal window closes abruptly
• Observe that events from "Application Error" and "Windows Error Reporting" appear in the Windows Event Log
• The error events show OpenConsole.exe crashing with exception code 0xc0000094 (integer division by zero)

Issue version 2:

• Run Bitvise SSH Server installer from Windows File Explorer
• The installer is a console application with graphical UI. Windows Terminal window opens in background for the console output
• Attempt to install the SSH Server
• Installation fails when the installer tries to run BvSshServer.exe to register the Windows service
• Error code from BvSshServer.exe is 0xC0000142, corresponding to DLL initialization issue
• Observe that events from "Application Error" and "Windows Error Reporting" appear in the Windows Event Log
• The error events show OpenConsole.exe crashing with exception code 0xc0000094 (integer division by zero)

Both problems appear to resolve themselves after Windows Terminal auto-updates to a newer version. This auto-update appears to happen unpredictably in the background, without notice to the user that something has changed.

After Windows Terminal auto-updates to 1.21.2911, the installers work - the OpenConsole crash no longer occurs.

Since Windows Server 2025 comes bundled with Windows Terminal 1.18, and the Windows Terminal version frequently does NOT update before the user attempts to install Bitvise software:

• We need a way to detect the application is running under Windows Terminal
• We need a way to detect the version of Windows Terminal, so that issues in the version that ships with the OS can be avoided

Minimal Reproducible Example

We have identified the cause of the crash in Windows Terminal 1.18 and wrote a trivial C++ program which reproduces it reliably:

TestConsoleBuf.txt

Conditions for the crash are set up if a program calls SetConsoleScreenBufferSize to enlarge the screen buffer height. The crash occurs when another program is run in the same terminal window.

This is fixed in newer Windows Terminal versions, but Windows Terminal 1.18 ships with Windows Server 2025. It is not necessarily updated before the user tries to run software in it.

It would be useful to have a documented mechanism to:
(A) detect that the program runs under Windows Terminal, and
(B) detect the Windows Terminal version.

In the absence of such a feature, our current workaround is to detect that screen buffer height == window height, and avoid enlarging the screen buffer height in this circumstance. We assume that problems might arise if the heights are equal and become unequal.

[lhecker]

Generally speaking, I'd recommend removing the SetConsoleScreenBufferSize call entirely from your application. Is it realistic that users in this day and age still have console buffers <300 lines? It should be <0.3% according to what I know. Because of that, I think it would be beneficial to respect the wishes of the user, even if the wish is kind of nonsensical.

In either case though, I'd recommend not calling the function at all, if you know that you're running under Windows Terminal, no matter its version. The only time it works under Windows Terminal is when you call it on an off-screen console buffer.
It's difficult to fix the SetConsoleScreenBufferSize implementation, because applications can deadlock if they notice that the resize failed, if Windows Terminal rejected the request. They sometimes check the buffer size and then change it if it doesn't match the target value. This would then repeat forever. Perhaps I'll just make it ignore the request anyway in the future, and we'll have to see how many bugs this results in. 🤔

If you're certain that you want to keep the SetConsoleScreenBufferSize call, I'd be happy to provide a way to detect Windows Terminal. 🙂

[denisbider]

Generally speaking, I'd recommend removing the SetConsoleScreenBufferSize
call entirely from your application. Is it realistic that users in this day and age
still have console buffers <300 lines?

For Bitvise SSH Server, it is realistic, because it is commonly installed on Windows Server versions 2016 - 2022 which use the traditional Windows console by default. The SSH Server installer is commonly started as a browser download, or from Windows File Explorer. In this case, Windows will open a console window with a default screen buffer height, which is small and insufficient to capture all of the installation details. If anything goes wrong during the installation, important parts of the output go missing.

I'd recommend not calling the function at all, if you know that you're
running under Windows Terminal, no matter its version.

The main problem is that we don't know whether we are running under Windows Terminal. We will detect it now by checking whether screen buffer height == window height, but that is a fallible heuristic. On Windows Server 2025, Windows Terminal does not appear as a parent for the installer process, so it cannot be identified using that heuristic, either.

Developers are missing an officially supported API that could be invoked to detect that the application is running under Windows Terminal, because it has very different behaviors from the traditional Windows console.

We also have the stermc command-line terminal client, which is part of Bitvise SSH Client. It performs well in the traditional Windows console, but performs poorly under Windows Terminal because of dramatic conceptual differences. We will need to implement special accommodations in stermc to support Windows Terminal. For this, a robust detection mechanism would be essential.

Because Windows Terminal behaviors (including crashes!) differ across versions, we would need the version as well. We can't assume that OS components are up to date; our software often runs in offline environments which are separated from the internet and are often conservative about updating. These environments will be invisible in telemetry metrics.

Perhaps only 0.1% of the software runs in a nuclear power plant. But the 0.1% that runs in a nuclear power plant is somewhat important.

[lhecker]

For Bitvise SSH Server, it is realistic, because it is commonly installed on Windows Server versions 2016 - 2022 which use the traditional Windows console by default.

I was quite certain that Windows 10 all the way back to RS1 set the default HKCU:\Console\ScreenBufferSize to 0x23290078 aka 9001x120. That would include Windows Server 2016. Perhaps I misremembered? Is Windows Server configured differently?

A possible workaround would be to use something like this:

const HWND hwnd = GetConsoleWindow();
wchar_t name[128] = {};
GetClassNameW(hwnd, &name[0], ARRAYSIZE(name));
if (wcscmp(&name[0], L"ConsoleWindowClass") == 0) {
    // ... call SetConsoleScreenBufferSize ...
}

I would not test for the opposite case (i.e. work around Windows Terminal by testing for "PseudoConsoleWindow"), because I think that Windows will keep the UNIX-style terminal architecture for the foreseeable future, but it may not always be Windows Terminal. I also don't think there will be a time where under any circumstance the default buffer height is <300 again either.

[DHowett]

Just a quick thought: I would not expect the SSH Server process to be running inside Terminal under any circumstances, nor would I expect the sessions it spawns to be.

In a typical Bitvise setup, which processes are running where and under which console hosts?

[denisbider]

Leonard:

I was quite certain that Windows 10 all the way back to RS1 set the default HKCU:\Console\ScreenBufferSize to 0x23290078 aka 9001x120. That would include Windows Server 2016. Perhaps I misremembered? Is Windows Server configured differently?

I did some testing, and it looks like you're correct. The small default screen buffer heights are on older Windows versions.

Our software still supports Windows Server 2003 and Windows XP SP3. That sounds old, but as recently as May this year, we received an inquiry from a customer who wanted to use our software on a network running Windows 2000.

if (wcscmp(&name[0], L"ConsoleWindowClass") == 0)

That is golden - thank you, much appreciated! We might need this to adapt stermc for Windows Terminal.

The situation with stermc is that it's a console-mode SSH terminal which expects to manage the screen buffer, but now Windows Terminal is the console-mode terminal which expects to manage the screen buffer. So, I'm thinking that for Windows Terminal, we'll probably just pipe the stream from the server to stdout. I hope that works.

[denisbider]

Dustin:

I would not expect the SSH Server process to be running inside Terminal under any circumstances, nor would I expect the sessions it spawns to be.

The issue described above relates specifically to our installers. Our installers are Windows console applications because they need to be scriptable from the command-line. If run with command-line parameters, they perform an unattended installation. If run without parameters, they open an interactive graphical installer UI.

Our design approach is anti-Apple. Our software is for administrators, so we favor exposing details for savvy users, rather than hiding details for inexperienced users. So, all of the details of the installation process go into the console window. If anything goes wrong, the user can copy & paste the content into a support inquiry.

In a typical Bitvise setup, which processes are running where and under which console hosts?

Once installed, the SSH Server runs as a Windows service in session 0. We don't currently have a problem with Windows Terminal there. Terminal sessions initiated by SSH users open in session 0 in a traditional Windows console. The console is monitored by the SSH Server to translate console events into terminal escape sequences.