Running language servers in containers

Body

I develop in various programming languages using Emacs as my text editor. Each language and development framework require a number of libraries and programs to be installed in order to have a fully featured development environment to work in. Moreover, each application that I develop might require different versions of libraries and programs than other applications. It would be a mess and insecure to have everything installed directly on the host ... well ... like we used to do 10 years ago.

I therefore have containerized my entire development workstation. My text editor runs in a container, and each application runs in one or more containers in its own pod. I also run language servers in separate pods/containers.

After I recently updated my entire Emacs configuration, I reviewed my language servers setup. Here is where I currently am with that.

The default operation mode for language servers is using STDIO to communicate with the text editor (client). That is, the server reads client requests from stdin and writes its responses on stdout. This is problematic with containers because the text editor container wouldn't have access to the standard input/output of another container.

Fortunately, there is a fairly easy way to expose input/output via a port using the socat tool. "Socat is a command line based utility that establishes two bidirectional byte streams and transfers data between them." - from https://linux.die.net/man/1/socat. Socat basically allows you to connect data sources of different types and we use that to send data received in a port to stdin and send data printed on stdout to the same (or another) port.

Let's see an example with the Haskel language server that only supports STDIO.

socat tcp-listen:"8893",reuseaddr exec:"haskell-language-server-wrapper --lsp"

And here is an example Containerfile with an entrypoint that uses the above command: https://github.com/krystalcode/docker-dev/tree/master/language-servers/haskell

The next step is to grant access to the text editor to the port on the language server container. I have a container network called lsp and I add the text editor pod and all language server pods to it. The text editor can then access the language server using the name of the its pod e.g. lsp_haskell:8893.

The remaining step now is to instruct the text editor to automatically connect to the language server whenever a Haskell file is loaded. In Emacs - when using Eglot - this is done with the following lisp code in your configuration:

(use-package eglot
  :ensure t
  :config
  (add-to-list 'eglot-server-programs '((haskell-mode) "lsp_haskell" 8893))
  :hook
  (haskell-mode . eglot-ensure))

The above code will do the following:

  • Ensure that the Eglot package is installed.
  • It will add the "lsp_haskell:8893" host:port combination as the language server for the Haskell mode.
  • It will connect to the language server when a file is loaded in a buffer in haskell-mode.

This has worked fine so far. Do note that the port needs to be given as an integer i.e. 8893 and not as a string i.e. "8893" - I wasted a few hours troubleshooting this until I figured it out.

I did find an issue with this setup though in that this works with only one project. That is:

  • I start the language server.
  • I load a Haskell file in project A - a connection is established to the server.
  • I load another file in project A - everything works as expected.
  • I load another Haskell file in project B - establishing a connection is refused by the server.

This is likely due to the fact that a separate connection is established by the language server for each project - which makes sense, and that does not work with the setup above.

For Haskell projects, it seems that it is best to run the server with the same GHC and Cabal versions that the project uses. Moreover, the server needs access to the project files, and - even though probably possible using volumes and some more customization in the requests sent by Emacs - it may make better sense to run a language server within each development project.

For other language servers though, like Bash for example, this is not acceptable. You may have a Bash script or two within a, for example, PHP project, and you wouldn't want to run a separate language server just for that. That's something, therefore, that I still have to figure out. For now, I have to restart the language server and reconnect.