Initial release specification

[jefflill]

The world needs an easy way to write and debug Raspberry Pi .NET Core applications using Visual Studio. There isn't really a good way to do this right now because there's a significant amount of setup required on both the Windows and Raspberry sides and the edit, build, debug inner loop is really clunky.

The idea is to build a Visual Studio VSIX that manages all of this as transparently as possible:

• Add an VS Options panel where users can manage their Raspberry Pi connections. This would include specifying the host/ip address, the SSH port, as well as the username and password (we'll consider doing SSH key authentication in the future. This dialog will include these connection functions:

  • Add
  • Edit
  • Test: verifies that we can connect using the settings
  • Remove
  • Read/write the settings as JSON to %USERPROFILE%\.pi-debug\connections.json. I spent a couple hours trying to figure out how to persist this to the VS registry but couldn't make it work. I'm going to rationalize this solution as being better because it will keep credentials out of the VS registry that might end up being exported and shared (which would be bad?).
     

• Maintain a catalog of .NET Core SDKs. This will include the name, version, architecture, binary download link, and SHA512 hash. We'll use this for ensuring that the correct SDK is installed on the Raspberry.

  I originally thought that the plug-in would read the catalog from GitHub at runtime, but we'll embed it as a resource instead so things will work for disconnected developers. This means will need to do a new release to support new SDKs (which only happens once or twice a month).

• Add a Debug/Start Debugging Raspberry command to begin remote debugging. This will need to:

  • Verify that Windows native OpenSSH client is installed and offer to install it when it isn't present
  • Identify the startup project
  • Verify that the startup project targets .NET Core
  • Publish the project for ARM32
  • Connect to the Raspberry via SSH
  • Verify that the referenced SDK is installed, installing it if necessary
  • Verify that vsdbg is installed, installing it if necessary
  • Copy the application files to the Raspberry
  • Some operations take some time (generating SSH keys, installing SDKs, etc). We need to present some kind of progress indicator.
  • We're going to need to generate a SSH key pair to be able to use the windows ssh command to interact with the debugger:
    • We'll store keys in %USERPROFILE%\.pi-debug\keys
    • Connections keep track of the username, password, and public/private SSH keys (after they've been generated).
      • If only the password is known when connecting, it will be used to connect and generate/configure SSH keys. Subsequent connections will use the keys.
      • If SSH key authentication fails, then we'll try the password again and configure the existing key as authorized. This will make it easy for users to reimage their Raspberry and start debugging again.
  • Start debugging
    • Have the user create a connection if one doesn't exist yet
    • Read the launchSettings.json file from the project's Properties folder
    • Install the SDK on the Raspberry if necessary
    • Install the debugger on the Raspberry if necessary
    • Upload the program binaries
    • Launch the debugger
      Info: Offroad Debugging of .NET Core on Linux OSX from Visual Studio
      • Handle command line arguments with single/double quotes and escapes
      • Don't allow spaces in the program or assembly names
      • Get VS to launch the debugger
      • Verify that the special @RASPBERRY environment variable works
      • Verify that environment variables work in general
         

• Project Properties: My original plan was to add a custom Debug Raspberry property page to the project being debugged. This page would allow the user to specify the target Raspberry connection, command line arguments and environment variables. I couldn't figure out a way to do this. I looked briefly into extending/modifying the existing Console and ASPNET Core project systems to create my own that adds my custom debug property page (or replaces the default one), but this really seems like overkill.

  After thinking about this, I realized there's a simple and not terribly hacky solution for the Start Debugging Raspberry command:

  • Use the default Raspberry connection by default, prompting the user to create one if necessary.
  • Use the command line arguments and environment variables from the launchSettings.json file
  • Users with multiple Raspberry connections will specify the hostname of the target via a reserved environment variable: @RASPBERRY=hostname. This variable will not be passed to the program being debugged.
     
    So, this integrates very nicely for users with only one Raspberry. For developers who need to target different Raspberries from different projects, they just need to add the @RASPBERRY environment variable, which isn't too icky, It would have been nice to intercept the usual debugging commands, but frankly that would have been complicated and risky and having a separate command dedicated to debugging Raspberry means that the default debug commands can be used as usual to run locally, etc.
     

• Raspberry filesystem

  • Raspberry connection information will be stored on the workstation at: %USERPROFILE%\.raspberry
  • .NET SDKs will be installed to /lib/dotnet
  • Add /lib/dotnet to the PATH
  • vsdbg will be installed to /lib/dotnet/vsdbg
  • Debugged .NET binaries will be uploaded to ~/vsdbg/PROJECTNAME where PROJECTNAME is the name of the Visual Studio project being debugged (where any spaces have been converted to underscores). This will allow debugging of multiple projects at the same time.
  • Raspberry project properties will be persisted as `$solutiondir/.vs/raspberry-projects.json
     

• Second Pass: Now that basic debugging is working, I think we may be able to provide a more seamless Run-F5 experience.

  • Intercept the debugging commands like this. We're going to use the ExtensibilityEssentials2019 extension to discover the command IDs because the way described in the stackoverflow post doesn't work any more.

    • Debug.Start: 5EFC7975-14BC-11CF-9B2B-00AA00573819, 0x127

    • Debug.StartWithoutDebugging: 5EFC7975-14BC-11CF-9B2B-00AA00573819, 0x170

      UPDATE: I don't believe this command makes sense for remote debugging. The problem is that launching the process will lock its program files on the remote machine which will block any subsequent binary uploads for the program. One possible way to deal with this would be to create separate temporary directories for each program run by adding a GUID to the program name or something. The problem with this is that these directories could accumulate quickly and there isn't an easy way to purge these without installing a service script or something. I'm not actually sure this command is that valuable on a Raspberry, so we'll just tell the user that it's not supported.

      Putting this on the backlog Future work #3.

    • Debug.StartDebugTarget (aka Attach...): 6E87CFAD-6C05-4ADF-9CD7-3B7943875B7C, 0x101

      UPDATE: Putting this on the backlog Future work #3.

    • Debug.Restart: 5EFC7975-14BC-11CF-9B2B-00AA00573819, 0x128 UPDATE: Debug.Restart is disabled while debugging via vsdbg, so we don't need to worry about this command.
       

  • Add a Project/Raspberry Debug Settings command and have this display a simple dialog (we could do a tool window, but I don't think its worth the effort). This dialog would allow the user to enable/disable Raspberry debugging for the project and also allow the user to select the target Raspberry.

  • The new properties will be stored in $solutiondir\.vs\raspberry-debug.json. .gitignore ignores the.vs directory by default, so this file won't get committed, which we wouldn't want because these are really developer specific settings. This single file will include a map of settings for each project keyed by project unique names. This avoids file pollution and also allows users to rename projects without a hitch. We could also prune entries for projects that no longer exist within the new properties dialog.

  • Remove the @RASPBERRY environment variable hack.

  • Remove our Start Debugging on Raspberry command

  • Support ASPNET: This is almost working already:

    • We need to setup the listening address as 0.0.0.0 so we can reach the server from the development workstation
    • Launch a browser on the workstation
       

  • NETCORE 5.0: The .NET Core 5.0 preview download links from MSFT are not stable. They seem to come and go. It would be really nice to support this because of Blazor. I realized last night that we could easily host the 5.0 preview binaries ourselves as GitHub releases.

    UPDATE: I'm not going to bother supporting .NET 5.0 prereleases. MSFT is already on RC2 and I suspect we'll see an official release soon.

  • Verify that a Visual Basic project works too

  • Pressing F5 while debugging fails because we should realize that we're debugging and not intercept the commands. We can detect that the IDE is in debug mode via dte.Mode == vsIDEMode.vsIDEModeDebug

  • Rename the VSIX from RaspberryDebug --> RaspberryDebugger (that's more natural)

  • Change back to the referencing the Neon.SSH.NET nuget package.

  • Tweak the progress bar UI: Change the caption to "Raspberry Debugger" and put the operation on the secondary line. It looks weird now.

  • Connections should be named by user@host

  • It takes several seconds for the package to load resulting in F5 not being intercepted immediately after starting VS.

    [ProvideAutoLoad] is the the answer. I specified both UIContextGuids80.NoSolution and UIContextGuids80.SolutionExists in separate attributes. UIContextGuids80.NoSolution is what did the trick; my package was loaded when VS presented the user with the initial what do you want to do? dialog.

  • Can we hide or disable the Raspberry Debug Settings... command if the current project type is not supported? This can be deferred if necessary because we do check and show a message box. It's not ideal though.

  • ConnectionDialog needs to ensure that connection names (user@host) are unique.

  • Verify the .NET SDK SHA512 hash before installation
     

• Log to the debug output window so users will have an idea of what happened when things go wrong.

• Cleanup: Make most types internal

• Publish the VSIX: info

  • Publish Neon.SSH.NET to nuget and restore the package reference
  • Purchase an EV code signing certificate. These are pricey but we can also use it to sign the neonDESKTOP binaries for Windows and OS/X.
  • Write some documentation
  • Flesh out the VSIX manifest
  • Review the Raspberry Trademark Rules and apply any changes
    • It looks like we can't use the Raspberry Pi icon anywhere. I'll find another image to use
    • Include these statements in the VSIX manifest as well as in the GitHub README.md:
       
      • The RaspberryDebugger extension is compatible with Raspberry Pi
      • Raspberry Pi is a trademark of the Raspberry Pi Foundation
         
  • VSIX Icon: a combination of the neonFORGE icon and a raspberry?
  • The Raspberry Settings... menu and dialogs should display the icon.

Limitations

• We're not going to honor global.json files. We'll just assume that the latest version of the specified framework SDK installed on the workstation should be used.

• Debugging ASPNET applications listening on HTTPS is not supported.

• Only 32-bit Raspberry Pi OS is supported

• Only .NET Core 3.1.x applications are supported. We'll support .NET Core 5.0 when it is formally released.

References

https://www.hanselman.com/blog/remote-debugging-with-vs-code-on-windows-to-a-raspberry-pi-using-net-core-on-arm
https://github.com/OmniSharp/omnisharp-vscode/wiki/Attaching-to-remote-processes
https://github.com/OmniSharp/omnisharp-vscode/wiki/Remote-Debugging-On-Linux-Arm
https://microsoft.github.io/debug-adapter-protocol/ - overview of remote debugging
OmniSharp Debugger Documentation: launch.json
OmniSharp: launch.json details

This documents launch.json for Visual Studio Code. I'm not convinced that this is the same for Visual Studio:

https://code.visualstudio.com/Docs/editor/debugging#_launchjson-attributes

It may be possible to support F5-Run and perhaps attach to process using custom tasks. The trick would be having a task be able to execute code in the VSIX. Something to investigate:

https://docs.microsoft.com/en-us/visualstudio/ide/customize-build-and-debug-tasks-in-visual-studio?view=vs-2019

[jefflill]

DONE (a while ago)