Docker and lsp-mode?

[xendk]

I discovered this repository while trying to dockerize my language servers for lsp-mode. Sadly it doesn't support the ones I use. But I have some thoughts/findings/suggestions.

First off, from my point of view it would be nice if lsp-mode would just use containers for language servers. Around here we've already moved all development into containers, and doing the same for language servers would get around issues around language versions and, for instance, having to have a node environment available in order to use the Dockerfile language server. But I understand this would probably be met with some opposition.

Next best, again, in my opinion, would be if lsp-mode supplied docker images for all the language servers it supported, and could just run those as an alternative to installing it.

In my own mucking about I've found that it's pretty simple to make lsp-mode use a langage server in docker. I've just fixed the server configuration to point at a script that mounts in root directory at the same location in the docker image and then run the same command that lsp-mode would. For TCP based language servers one just have to bind the port with the -p switch to docker (and ensure that the server accepts connections from non-localhost).

[yyoncho]

These are all good suggestions and we pretty much what we want to achieve with this project. The reason for not having these suggestions is the lack of resources. ATM there are 72 language servers, so dockerizing them is a lot of effort.

[xendk]

72 is quite a number, didn't know there was that many. But I assume at lot of those have been contributions, and I was assuming the same would be the case for the docker versions. Some might already be available upstream, I found that the docker-langserver already had an image I could use. But I think there's a visibility issue here, I wasn't aware of this project until two days ago, and I've been an lsp-mode user for two years now.

I would think that having docker language servers supported directly in lsp-mode would mean that more noticed that possibility and more contributed images.

I was thinking one could extend lsp-dependency with a :docker stanza which specifies the image and any needed meta-information. Only about 19 language servers seems to be using lsp-dependency at the moment but it looks like only 4 servers that support download doesn't use lsp-dependency (mostly Windows stuff, but also eslint, that, at a glance, seems to be able to use :download). And perhaps a simple clients/images directory with Dockerfiles that gets built and pushed to Githubs image registry.

I'd suggest having a global "use docker" defcustom which would make lsp-mode just pull images when needed, without prompting the user. I think when you've decided to use docker, you've kinda drunk the cool-aid.

If I understand the doc of lsp-dependency correctly, one could add an image to a language server from ones init.el, so you don't even have to touch lsp-mode to get started.

With some documentation and a few good examples, I think the community will pick up on it and start rolling images.

[yyoncho]

These are all good suggestions. I was considering implementing them as well, but it seems to me that there is no huge demand for docker related solutions(although, they have a lot of advantages).

If I understand the doc of lsp-dependency correctly, one could add an image to a language server from ones init.el, so you don't even have to touch lsp-mode to get started.

Not exactly since some of the commands use a bit more complex way to build the command to run. The whole lsp-dependency and lsp-mode way to build the command is not designed very well for historical reasons.

But in general, we may fix lsp-mode to allow implementing what you are proposing.

[xendk]

Not exactly since some of the commands use a bit more complex way to build the command to run.

Well, yeah, I saw typescript-language-server requiring tsserver, which one would bundle into the same image. Which could be dealt with in two ways: The simple :new-docker-connection way, or having an lsp-dependency stanza that tells lsp everything it needs in order to start the image (to tell lsp what arguments it should pass a TCP language server to specify port, for instance).

but it seems to me that there is no huge demand for docker related solutions

Well, I would think that as containers seems to be sneaking into all kind of projects that being able to run a container might actually be more common between developers than, say, having a nodejs installed... Well, OK, nodejs might be the one that beats docker, at least in the web development department.

Maybe you're right, and I should just carry on with my "wrap a command in a container" script. Has the advantage of being reusable for flycheck and general cli commands.

[aidalgol]

Next best, again, in my opinion, would be if lsp-mode supplied docker images for all the language servers it supported, and could just run those as an alternative to installing it.

@xendk I have made a start on this, independent of emacs-lsp: https://github.com/aidalgol/dockered-language-servers
It only has Dockerfiles for elixir-ls for Elixir 1.10 and a.11 so far, and I am really unsure how best to structure the repository to accommodate the different needs for other language server images, so I welcome any assistance with this effort.

[factyy]

@xendk I have opened a few PRs that add support for this features (at least partially). Take a look at the now-revamped readme

[xendk]

@factyy Interesting. Seems that we're a few that's gone down similar paths.

I'm still using rid as it has the advantage that I can use the same command for running cli commands in docker (almost) seamlessly.

The thing is, I think this is way more complicated than it should be. lsp-mode has already made as simple as it can be, provided you have the ecosystem for the language server (ie, nodejs for the bash-language-server) installed. Using images ought to be simpler.

In my mind lsp-docker should just make lsp ask if it should look for a language server image when lsp determines it hasn't got one. Determining whether one exists would be consulting a list of language names to image names which could be either provided with lsp-docker or fetched over HTTP. When the user has made a choice, the image would be pulled and run with the root of the project mounted in under the same name (to avoid dealing with path translations), the uid/gid of the user (for good measure), and then just let lsp manage the run command like it does any other language server.

Then the rest of the work would just be adding language servers to @aidalgol's https://github.com/aidalgol/dockered-language-servers and add them to the list (dockered-language-servers might even build the list and publish it somewhere).

Images could communicate extra abilities (like supporting a TCP port argument) through image labels.

Well, that's my dream, anyway....

[factyy]

@xendk Well, these are indeed interesting points. I think that an ideal situation in my opinion looks quite similar to yours, except for path mounting (I believe it is really important to have the freedom to choose which paths to mount and which not to mount, e.g. not to bring build caches and other artifacts back to the host).

At the same time I believe that containerized language servers out-of-the-box part is quite hard to achieve. There have to be lots and lots of actively maintained images, I don't know if it is possible to achieve this goal at all considering the available resources. So I decided to help users to use their own containers as language servers...

One question: could you clarify a bit what do you mean by using the same command for running cli in docker seamlessly?

[xendk]

(I believe it is really important to have the freedom to choose which paths to mount and which not to mount, e.g. not to bring build caches and other artifacts back to the host).

I'll wager that most people won't care, as long as "it works". You have any language servers producing build artifacts? I don't have any, but I'd think if one has accepted the fact that one creates build artifacts inside the project directory when run standalone, they'd accept it for a dockerized server too. Might even consider it a feature.

At the same time I believe that containerized language servers out-of-the-box part is quite hard to achieve. There have to be lots and lots of actively maintained images, I don't know if it is possible to achieve this goal at all considering the available resources.

Why do you think it's harder? Getting a server running in some random container requires insight into containers too, getting all the moving parts in line. Writing a Dockerfile is basically just scripting the process. The situation now is that everybody is starting from scratch trying to figure out how to run a language server in a container.

So I decided to help users to use their own containers as language servers...

If they have a container they can use, they're one step away from making an image. The majority of images will be a three step process: 1: Select a base image, 2: Install the language server package and 3: Set the entrypoint. Having a library of language servers will provide plenty of example for people to create new images.

One question: could you clarify a bit what do you mean by using the same command for running cli in docker seamlessly?

I'm using https://github.com/xendk/rid. It's just a wrapper that compiles all the needed arguments to docker to make it seem that the command run in the container was run on the host system. So when I type composer install, it's actually running composer in a PHP container, with all the mounts, uids and env variables to make it "just work". Allows for setting images per directory, so I can match PHP version per project.

I'm using the same command for language servers, so when lsp-mode starts serenata, it doesn't know it's actually running in a container. But rather than hacking up an image for each language server I want to use, it would be much nicer just being able to pull standard images, or, better, have lsp-mode figure it out for me entirely.