VSCode remote SSH to container always fails/times out in Windows

[jansselt]

What happened?

devpod starts container on remote host and opens VSCode but then is never able to SSH into the container.

What did you expect to happen instead?

VSCode to open and connect to the container. This works without issue for me in Linux. In Windows it times out no matter how long I set the timeout maximum.

How can we reproduce the bug? (as minimally and precisely as possible)

For me, create any workspace and open in VSCode. I am using an SSH provider.

My devcontainer.json:

{"image":"mcr.microsoft.com/devcontainers/javascript-node"}

Local Environment:

• DevPod Version: 0.5.4
• Operating System: windows
• ARCH of the OS: AMD64

DevPod Provider:

• Local/remote provider: ssh

Anything else we need to know?

I am experiencing an issue similar to this issue I believe #850

The container is created just fine. I can use VSCode Browser just fine. However if I try to use VSCode, it always fails to SSH connect to the container.

[08:13:20] info Workspace resume-generator already exists
[08:13:20] debug Acquire workspace lock...
[08:13:20] debug Acquired workspace lock...
[08:13:20] info Creating devcontainer...
[08:13:20] debug Inject and run command: '/tmp/devpod/agent' helper ssh-server --stdio --debug
[08:13:20] debug Attempting to create SSH client
[08:13:20] debug execute inject script
[08:13:20] debug download agent from https://github.com/loft-sh/devpod/releases/download/v0.5.4
[08:13:20] debug Run command provider command: ${SSH_PROVIDER} command
[08:13:21] debug Received line after pong: done
[08:13:22] debug SSH client created
[08:13:22] debug SSH session created
[08:13:22] info Execute SSH server command: bash -c '/tmp/devpod/agent' agent workspace up --workspace-info 'H4sIAAAAAAAA/+xTXW/iOhD9K1fznA9Ki9TmDQUu5d5uqWhY7a4iIdcegsGxI4+Tdrfiv69MoE1pV/u0b/uU2HPO+MzMmWd4NHZLFeM4s7KQGhJIkzxfEFrK8w3TRHkeCWwqI/KcG+3wyVGeC1yxWrk8f+FTnlukusSwQI2WOWM70WhDRkPw+hwkzyAFJHBKggDqk0DIV+yKQwCV5K62CAmsnasoieNCunX9ELUfRoSOIm7KuP09hENlChMOBnzwcMXOVjjoR5UufD5rGinQejGalT4x0RoCMJWTRpO/H07Gt9nybphd+1PDVO1hsSuruG1LzArUDnYBjGbp/+P5O6wwfIvWA8Zfsvlw+e/NcHLfBYTqH1Vv0VdOaO9aUQISZ2vcBXA9u8+6cFqjUlEhf5RG1HwbPUr9K+r0dphm08/T7Osym34azxY+0f7+v3GaLQ+C0/l4NL7NpsObN7J8DngFT6bZ75B3s/kbpf3+x8J2uwBKxtdSY6tHCuzMoCFuxD4hmdq2Ximkm2NlSDpjv783gJ/53q2oXHxqKW8Pn41bZH6umSyRHCsrL7HXvwh752G/n52dJ73LpHf5DQJQjNyCvOATxNl50u95xGEX/HzbXfAveNoIm9Rox6RGmxq9koXXz1/+ZMkKX2XJbVRKbg2ZldsXILDhRybFG9Yw4lZWLtSHdlTMrSGBqAtsN2sXQGvD5PmIem/RAIR51MowsZjffNxDZVYupPWRZ1EhI6T4SIybXjSILiAAqTfI3US61KJA7SRTdLTCMTraO/9DAD4hb2ffrodCou65U8jhwiclx5SCBFZM0b4jvCZnSk/0ByVnr4u791THS393+g/v9E8AAAD//wEAAP//o/eT/k0GAAA=' --debug
[08:13:22] info No root required, because neither docker nor agent daemon needs to be installed
[08:13:22] info Use /home/luke/.devpod/agent/contexts/default/workspaces/resume-generator as workspace dir
[08:13:22] debug Created logger
[08:13:22] debug Received ping from agent
[08:13:22] debug Credentials server started on port 16161...
[08:13:23] debug Incoming client connection at /
[08:13:23] debug Credentials server started...
[08:13:23] debug Wrote docker credentials helper to /home/luke/.devpod/agent/contexts/default/workspaces/resume-generator/.cache/nodoleoyznkt/docker-credential-devpod
[08:13:23] debug Workspace Folder already exists /home/luke/.devpod/agent/contexts/default/workspaces/resume-generator/content
[08:13:23] debug Workspace exists, skip downloading
[08:13:24] debug Using docker command 'docker'
[08:13:24] debug execute inject script
[08:13:24] debug Received line after pong: done
[08:13:24] debug done exec
[08:13:24] debug done inject
[08:13:24] debug done injecting
[08:13:24] debug Done InjectAgentAndExecute
[08:13:25] debug Injected into container
[08:13:25] info Setup container...
[08:13:25] debug Inject and run command: '/usr/local/bin/devpod' helper ssh-server --stdio --debug
[08:13:25] debug Attempting to create SSH client
[08:13:25] debug SSH client created
[08:13:25] debug SSH session created
[08:13:25] info Execute SSH server command: bash -c '/usr/local/bin/devpod' agent container setup --setup-info 'H4sIAAAAAAAA/7RWbW/iOBD+K8ifQ5yEBIilfuAgS9HxJl5277ZUkYkHyDaxke2w7Vb895MTWujLrfbuet/smWdmPM/YM35EPTh0Bdc05SC7gm/S7ZdU76ZU7xB5REkpMas0p1tABOWJtPM0kUKJjbYTkWMGh+TJg8Lf6IGqRKZ7XeeCATpaaF86Q/Yl0P6mBDfKEcgtsO5zHAm50LBUIBFBpQcLJYXSIk9/UJ0KrgzqoBKjIjePCO41cFUpbhBbUyhykHaFqIPKUq7R7dH6ZeTt0fov2c6LtdKpLsxZDbNwr82JL3ke9BBBElSRQ30LvJ5saJggCw1FQrMvQt6pPU3gk8hYyQLeiRxwVtwBNhzuBcN0C1zjpHKvMIMNLTKNvz/ZKnx2D5JqISsw18hCz+d4J9RPPCALRfxQ5vLbch7Po/l8MBnHZt3p9WbRfI4IKnh6T0zBr7AsOC4USOw6joPXhUIW6kWfp5Ne3OlH40W8nA0RQTut94pgvE31rliXHGdio+tqh0/JSsiAKlCYie88E5Thg2MHtn/21x8s4utoOI1m8XQyWyCC3KbbdA1g0v09msXdyfjToP9hXNoJTXaAuWAiA/Hwg98ZXq8no+hFCFPSztiEBR4v5/Zy8aneNsJJf9wpsSfUtLO4NpaFkjgzlwCrdcrJxf55e1aUi2r7LNvSHNSl3UmgON1XqP8x/+mX3uv059fR0BTZxMZrqnalbPjZyByznl/H3eEgGpc181q2Yzu2W2v4rteseY7nPmEm43HUXQwm43dw5/3JYjmPZmdy/+j149lyvBiMorg3MIqXV/MEebrP3WGnusmmBb1Ule/WD8NX4sWfU1NLrR+QhWLjX+d7fMmu6QvPj20kCq4N/mEPV+uUM0uJQiZw9aHv3NJUbkFf/fRFHy+aQQ80TbOyu5Z5Nn3WDFpJsGaNMGgy1qJNykIKwYa64IaevwnBbwaMbhLfb4WhF7Y2rcBhXhL4Xss1zHclUA0MEeQ5nl93GnXPW7gN4rSJ59stv93y2p4TfjVF1lSDiW0WhTJP5j41tqVKamAd/a6fwPa8Rui1nSD4esrnNEmGdA1VPgwO9rlp2yl7v/2+gp3aDiLo4NgNu+G9hVSFe7+JvRwU5UBRbz3oNAelab5HBC0KsGpOUBtRaW6yX3NbJGgTt1nrjxZvTQ9UppRXrNTXQtx9FzKvYOcxm4OmjGqKCLp5XKGUrRBZoe0ukXYqXp1xA1QXEhRORJ4LXi90minirdDR+mXTbaqJ+48szNw0Jtbq1ZxfIfK4Ok36an0e4CtEblZ/M8JX6PZ4LA9w/kuUB+Glow+Pc7ytSN8LZhfVv0UKYRriAaTxYlqWaU+OuZ/LF4jBv/1nHI9/AQAA//8BAAD//1/lX2S7CQAA' --container-workspace-info 'H4sIAAAAAAAA/6pWykxJVbKqVspLzE1VslIqK07OT0lVqtVRSs7J9C8oyczPKwZJg1UhyabkJ2enFuWkFoNka2sBAAAA//8BAAD///hy5ipHAAAA' --chown-workspace --debug
[08:13:25] debug Created logger
[08:13:25] debug Received ping from agent
[08:13:25] debug Start setting up container...
[08:13:25] debug Patch etc environment & profile...
[08:13:26] debug Run post create commands...
[08:13:26] debug Done setting up environment
[08:13:26] debug Setup vscode...
[08:13:26] info Install extensions 'dbaeumer.vscode-eslint' in the background
[08:13:26] debug Connection to SSH Server closed
[08:13:26] debug Done executing ssh server helper command
[08:13:26] debug Done setting up container
[08:13:26] debug done inject
[08:13:26] debug Connection to SSH Server closed
[08:13:26] debug done injecting
[08:13:26] debug Done executing ssh server helper command
[08:13:26] debug Done InjectAgentAndExecute
[08:13:26] debug Done creating devcontainer
[08:13:26] info Run 'ssh resume-generator.devpod' to ssh into the devcontainer
[08:13:26] debug No dotfiles repo specified, skipping
[08:13:26] info Starting VSCode...
[08:13:26] debug done exec
[08:13:26] debug Run vscode command code --disable-extension ms-vscode-remote.remote-containers --new-window --folder-uri=vscode-remote://ssh-remote+resume-generator.devpod//workspaces/resume-generator

This only happens for me on Windows. Linux works with no issues. I don't have a mac, so can't test that.

[jansselt]

Also may be related to #521 in which case I see the response that theres need for work on the windows SSH and potentially missing unix utilities. So that may be the cause. If so, I suspect this can be closed and I can just +1 that.

[pascalbreuninger]

Hi @jansselt, thanks for reporting the issue. Could you try to increase the vscode SSH timeout and see if that helps? (Second comment on this stackoverflow question: https://stackoverflow.com/questions/59978826/why-ssh-connection-timed-out-in-vscode)

[jansselt]

I actually stumbled across that SO while trying to figure it out myself. I did try this and it did not work for me. I let it run for 10 mins or so. And it just keeps logging "Waiting for devpod agent to come up" occasionally.

However, I will try once more later when I'm back in my network to make sure that is still the case.

[pascalbreuninger]

If it doesn't work out we'll take a look, thanks 👍

[jansselt]

Alright, let er run for about 10 mins this morning.

[rodenj1]

I just wanted to add to this bug. I am running into the same issue. Windows 11 Running DevPod. Provider is ssh to a linux system. Web VScode works fine, but local VScode never connects with same error reported earlier. I did test increasing the timeout, but that did not help. Here are all the different logs

DePod

VSCode Log

Doing a direct SSH to the pod also fails with the same VSCode messages

However, if I use the DevPod CLI to connect via ssh, it works!

[pascalbreuninger]

thanks @rodenj1

[Analect]

I can confirm the same problems per other users above ... works with vscode browser and devpod ssh ... but fails with desktop version of vscode. Set timeout from 15s to 60s. I set up provider connection as [email protected]. Just wondering, does it somehow expect a vscode user to be pre-established on the remote machine? Thanks.

[09:15:53.351] Log Level: 2
[09:15:53.373] SSH Resolver called for "ssh-remote+idpbuilder-ref-impl-idp-macmini-2204.devpod", attempt 1
[09:15:53.374] "remote.SSH.useLocalServer": false
[09:15:53.375] "remote.SSH.useExecServer": false
[09:15:53.375] "remote.SSH.showLoginTerminal": false
[09:15:53.376] "remote.SSH.remotePlatform": {"192.168.192.13":"linux","192.168.192.49":"linux","192.168.192.63":"linux","192.168.192.21":"linux","192.168.192.50":"linux","192.168.192.48":"macOS","192.168.192.27":"macOS","192.168.192.42":"linux","192.168.192.56":"linux"}
[09:15:53.376] "remote.SSH.path": undefined
[09:15:53.376] "remote.SSH.configFile": undefined
[09:15:53.376] "remote.SSH.useFlock": true
[09:15:53.377] "remote.SSH.lockfilesInTmp": false
[09:15:53.377] "remote.SSH.localServerDownload": auto
[09:15:53.377] "remote.SSH.remoteServerListenOnSocket": false
[09:15:53.377] "remote.SSH.showLoginTerminal": false
[09:15:53.378] "remote.SSH.defaultExtensions": []
[09:15:53.378] "remote.SSH.loglevel": 2
[09:15:53.378] "remote.SSH.enableDynamicForwarding": true
[09:15:53.378] "remote.SSH.enableRemoteCommand": false
[09:15:53.379] "remote.SSH.serverPickPortsFromRange": {}
[09:15:53.379] "remote.SSH.serverInstallPath": {}
[09:15:53.387] VS Code version: 1.87.2
[09:15:53.387] Remote-SSH version: [email protected]
[09:15:53.387] win32 x64
[09:15:53.485] SSH Resolver called for host: idpbuilder-ref-impl-idp-macmini-2204.devpod
[09:15:53.485] Setting up SSH remote "idpbuilder-ref-impl-idp-macmini-2204.devpod"
[09:15:53.511] Using commit id "863d2581ecda6849923a2118d93a088b0745d9d6" and quality "stable" for server
[09:15:53.519] Install and start server if needed
[09:15:56.238] Checking ssh with "C:\Program Files (x86)\Google\Chrome\Application\ssh.exe -V"
[09:15:56.241] Got error from ssh: spawn C:\Program Files (x86)\Google\Chrome\Application\ssh.exe ENOENT
[09:15:56.242] Checking ssh with "C:\Program Files (x86)\VMware\VMware Workstation\bin\ssh.exe -V"
[09:15:56.247] Got error from ssh: spawn C:\Program Files (x86)\VMware\VMware Workstation\bin\ssh.exe ENOENT
[09:15:56.247] Checking ssh with "C:\WINDOWS\system32\ssh.exe -V"
[09:15:56.254] Got error from ssh: spawn C:\WINDOWS\system32\ssh.exe ENOENT
[09:15:56.254] Checking ssh with "C:\WINDOWS\ssh.exe -V"
[09:15:56.257] Got error from ssh: spawn C:\WINDOWS\ssh.exe ENOENT
[09:15:56.258] Checking ssh with "C:\WINDOWS\System32\Wbem\ssh.exe -V"
[09:15:56.260] Got error from ssh: spawn C:\WINDOWS\System32\Wbem\ssh.exe ENOENT
[09:15:56.260] Checking ssh with "C:\WINDOWS\System32\WindowsPowerShell\v1.0\ssh.exe -V"
[09:15:56.263] Got error from ssh: spawn C:\WINDOWS\System32\WindowsPowerShell\v1.0\ssh.exe ENOENT
[09:15:56.263] Checking ssh with "C:\WINDOWS\System32\OpenSSH\ssh.exe -V"
[09:15:56.340] > OpenSSH_for_Windows_8.6p1, 
[09:15:56.341] > LibreSSL 3.4.3

[09:15:56.355] Running script with connection command: "C:\WINDOWS\System32\OpenSSH\ssh.exe" -T -D 11088 "idpbuilder-ref-impl-idp-macmini-2204.devpod" bash
[09:15:56.357] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[09:16:17.081] > 09:16:17�[38;5;6minfoWaiting for devpod agent to come up...
> �]0;C:\WINDOWS\System32\cmd.exe�
[09:16:17.081] Got some output, clearing connection timeout
[09:16:40.110] > 09:16:40 �[38;5;6minfo Waiting for devpod agent to come up...
[09:17:03.137] > 09:17:03 �[38;5;6minfo Waiting for devpod agent to come up...
[09:17:26.147] > 09:17:26 �[38;5;6minfo Waiting for devpod agent to come up...
[09:17:49.163] > 09:17:49 �[38;5;6minfo Waiting for devpod agent to come up...

[eduyubero]

+1

I am facing the exact same problem my peers have.

The following is my setup:

• A laptop with Microsoft 11 as base OS.
• An Ubuntu VM launched with Multipass.
• DevPod provider with SSH to the Ubuntu VM.

Results are the same described above.

• IntelliJ works (with Jetbrains Gateway)
• VSCode Web works.
• VSCode Desktop returns same timeout error logs shared by @rodenj1 .

[jzazo]

Same experience as me. I am connecting via tunnel to my remote machine and I am forwarding ssh ports manually (2222). Web vscode works though.

[fzxu]

I have the same problem. vscode shows 'Opening Remote...' forever.

...
[23:10:38] debug Created logger
[23:10:38] debug Received ping from agent
[23:10:38] debug Start setting up container...
[23:10:39] debug Credentials server started on port 15235...
[23:10:40] debug Incoming client connection at /
[23:10:40] debug Credentials server started...
[23:10:40] debug Patch etc environment & profile...
[23:10:40] debug Run post create commands...
[23:10:41] debug Done setting up container
[23:10:41] debug Connection to SSH Server closed
[23:10:41] debug Done executing ssh server helper command
[23:10:41] debug Done creating devcontainer
[23:10:41] info Run 'ssh abc-web.devpod' to ssh into the devcontainer
[23:10:41] debug No dotfiles repo specified, skipping
[23:10:41] info Starting VSCode...
[23:10:42] debug Run vscode command code --disable-extension ms-vscode-remote.remote-containers --new-window --folder-uri=vscode-remote://ssh-remote+abc-web.devpod//workspaces/abc-web

Windows 11 Pro 23H2
64-bit operating system, x64-based processor

Please looking into this or if there is any workaround. Really appreciated.

[89luca89]

Hi all!

Can you please attach the following tests:

• try running devpod ssh --debug name-of-project.devpod in order to see if this problem occurs also with regular ssh, or it's a vscode thing

• also try adding --debug to rhe ProxyCommand in the ssh config, and try again from vscode, in order to get more logs

Thanks a lot!

[fzxu]

devpod ssh --debug name-of-project.devpod is working fine from my knowledge, it can ssh and keep output 'Start refresh' token or something.

try from vscode, there is no error, it just hang at Install and start server if needed

[00:20:16.811] Log Level: 2
[00:20:16.819] SSH Resolver called for "ssh-remote+abc.devpod", attempt 1
[00:20:16.819] "remote.SSH.useLocalServer": false
[00:20:16.819] "remote.SSH.useExecServer": true
[00:20:16.819] "remote.SSH.showLoginTerminal": false
[00:20:16.819] "remote.SSH.remotePlatform": {}
[00:20:16.819] "remote.SSH.path": undefined
[00:20:16.820] "remote.SSH.configFile": undefined
[00:20:16.820] "remote.SSH.useFlock": true
[00:20:16.820] "remote.SSH.lockfilesInTmp": false
[00:20:16.820] "remote.SSH.localServerDownload": auto
[00:20:16.820] "remote.SSH.remoteServerListenOnSocket": false
[00:20:16.820] "remote.SSH.showLoginTerminal": false
[00:20:16.822] "remote.SSH.defaultExtensions": []
[00:20:16.822] "remote.SSH.loglevel": 2
[00:20:16.822] "remote.SSH.enableDynamicForwarding": true
[00:20:16.822] "remote.SSH.enableRemoteCommand": false
[00:20:16.823] "remote.SSH.serverPickPortsFromRange": {}
[00:20:16.823] "remote.SSH.serverInstallPath": {}
[00:20:16.830] VS Code version: 1.89.0
[00:20:16.830] Remote-SSH version: [email protected]
[00:20:16.830] win32 x64
[00:20:16.834] SSH Resolver called for host: abc.devpod
[00:20:16.834] Setting up SSH remote "abc.devpod"
[00:20:16.838] Using commit id "b58957e67ee1e712cebf466b995adf4c5307b2bd" and quality "stable" for server
[00:20:16.840] Install and start server if needed

I have --debug added but seems no more logs. I also increased the remote ssh timeout to 60s from 15s.

[jzazo]

Yeah, I confirm I can ssh with devpod ssh name-of-project.devpod as well.

[fzxu]

update from my side. It was actually a false alarm from my side. For windows there is additional selection popup (I didn't noticed). After choosing 'Linux' it works for me. Thanks!