Bindings from latex layer are not available when editing latex files

[dankessler]

Description

Even with the latex layer enabled, when editing latex files (that end in .tex) none of the major mode bindings are available and also lsp falls down.

Reproduction guide 🪲

• Start Emacs
• Visit a file ending in .tex
• Press SPC m

Observed behaviour: 👀 💔
RE key bindings, observe this in minibuffer

Minibuffer output: SPC m-  which-key: There are no keys to show

RE LSP: this message appears (after visiting a .tex file)

LSP :: There are no language servers supporting current mode `LaTeX-mode' registered with `lsp-mode'.
This issue might be caused by:
1. The language you are trying to use does not have built-in support in `lsp-mode'. You must install the required support manually. Examples of this are `lsp-java' or `lsp-metals'.
2. The language server that you expect to run is not configured to run for major mode `LaTeX-mode'. You may check that by checking the `:major-modes' that are passed to `lsp-register-client'.
3. `lsp-mode' doesn't have any integration for the language behind `LaTeX-mode'. Refer to https://emacs-lsp.github.io/lsp-mode/page/languages and https://langserver.org/ .
4. You are over `tramp'. In this case follow https://emacs-lsp.github.io/lsp-mode/page/remote/.
5. You have disabled the `lsp-mode' clients for that file. (Check `lsp-enabled-clients' and `lsp-disabled-clients').
You can customize `lsp-warn-no-matched-clients' to disable this message.

Expected behaviour: ❤️ 😄
which-key pop-up showing the many useful commands I can run when editing a latex file.

Also, I expect lsp support to work (since I have texlab installed)

System Info 💻

• OS: darwin
• Emacs: 29.1
• Spacemacs: 0.999.0
• Spacemacs branch: develop (rev. df8fef8)
• Graphic display: t
• Running in daemon: nil
• Distribution: spacemacs
• Editing style: hybrid
• Completion: helm
• Layers:

((auto-completion :disabled-for org)
 better-defaults
 (bibtex :variables bibtex-enable-ebib-support t)
 csv emacs-lisp
 (ess :variables ess-assign-key "\255" ess-use-flymake nil)
 eww git graphviz haskell
 (helm :variables helm-use-fuzzy 'source helm-buffer-max-length nil)
 html json latex lsp
 (major-modes :variables matlab-shell-command "/Applications/MATLAB_R2021b.app/bin/matlab")
 markdown osx
 (org :variables org-adapt-indentation nil spaceline-org-clock-p t org-want-todo-bindings nil)
 outshine pandoc pdf python
 (shell :variables shell-default-height 30 shell-default-position 'bottom)
 spell-checking syntax-checking
 (version-control :variables version-control-global-margin nil)
 (treemacs :variables treemacs-use-scope-type 'Perspectives treemacs-use-follow-mode nil treemacs-recenter-after-tag-follow 'on-distance))

• System configuration features: ACL GNUTLS JSON LIBXML2 MODULES NOTIFY KQUEUE PDUMPER SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER XWIDGETS ZLIB

Explanation

This seems to be a consequence of recent changes in AUCTeX (that are live in the version on ELPA) and explained in the AUCTeX documentation.

In brief, AUCTeX has changed conventions for its major mode names. Whereas previously its modes had names like {tex,latex}-mode, it now names its major modes like {TeX,LaTeX}-mode. It then makes use of emacs 29.1+'s major-mode-remap-alist so that e.g., activating latex-mode will instead call LaTeX-mode.

However, spacemacs sets up its bindings using spacemacs/set-leader-keys-for-major-mode. The key binding setup for latex is in the latex layer's packages.el file, but it refers to these modes like {tex,latex}-mode. So, it does all of its configuration in the spacemacs-latex-mode-map, but I suspect that when I open a .tex file, due to the remapping it's now looking for a spacemacs-LaTeX-mode-map that does not exist.

This could be fixed quickly by just updating the case used when referring to major mode's throughout the latex layer's packages.el file.

A deeper fix would involve adjusting either spacemacs/set-leader-keys-for-major-mode or the way spacecmacs loads keymaps to be aware of major-mode-remap-alist.

[dankessler]

After digging into this more, the place for a deeper fix would be upstream and most likely in bind-map, but I'm not even sure it makes sense for it to try to compensate for major mode remapping. The simplest fix is probably to just tweak the latex layer to cover both the old (for people using older emacs/AUCTeX) and new major mode names.

[dankessler]

The issue with lsp would need to be fixed in the upstream package lsp-latex; I'll try to fix it there.

UPDATE: upstream PR is now submitted

[tigerjack]

Do the auctex changes affect also the local variables definition?
While your pull request works inside the main file (that is, TeX-master: t), it does not from other ones (that is, TeX-master: "main").

EDIT: It seems that the TeX-master is properly set when the C-c _ keybinding is invoked, and the following is written into the tex file

%%% Local Variables:
%%% mode: LaTeX
%%% TeX-master: "slides"
%%% End:

However, it is not read again when reopening the file later. May be it due to the fact that the mode is called LaTeX and it is expecting something else? I tried to change it to latex, but the effect is the same.

[dankessler]

@tigerjack the local variable TeX-master seems to be working correctly for me, e.g., if I open a file that has

%%% Local Variables:
%%% mode: LaTeX
%%% TeX-master: "../main"
%%% End:

where there is a main.tex file in the parent directory, then things work properly, e.g., compiling with SPC a u compiles the parent file, not the child file which is open.

Your issue did make me realize that I had missed some keybinds in my PR, e.g., those dealing with reftex features (e.g., SPC R t does not give the reftex-toc overview. To avoid making it too kludgy, I'm going to rework the PR so that at the top of the latex layer's packages.el file, it figures out whether it should set configuration for latex-mode or LaTeX-mode (similarly for tex-mode and context-mode), sets some private variables appropriately, and then relies on those throughout.

[dankessler]

This is a bigger pain than I thought. When packages.el is run, AUCTeX has not yet been loaded, so it's somewhat challenging (to me at least) to figure out whether things should be configured for latex-mode or LaTeX-mode.

Configuring for both does kind of cover our bases, but it makes the code pretty ugly and duplicative. In addition, if dotspacemacs/user-config sets key bindings (e.g., I use (spacemacs/set-leader-keys-for-major-mode 'latex-mode "oc" 'cdlatex-mode)), then that config would need to get updated (or duplicated) too.

Instead, I'm now of the opinion that it'd be cleaner for bind-map to be updated so that it is aware of major-mode-remap-alist. For example, suppose I have configured keybindings for foo-mode, but foo-mode has been remapped to Foo-mode. If Foo-mode is activated (either directly or on redirecton from foo-mode then I want bind-map to activate my bindings for foo-mode. Chaos might abound if I've configured two different keymaps for foo-map and Foo-map and remapped foo-map to Foo-map, but that seems like a configuration that is asking for trouble anyway.

I've locally patched bind-map to behave as described above and it's working for me, so in due course I'll open that as a PR upstream.

In related news, my fix to lsp-latex was merged, so that part of the issue is now resolved.

I'm going to leave this issue open and its associated PR in draft status for now, because if we can get bind-map changed then this issue is resolved and the PR is moot, but if for whatever reason that doesn't work then we can try to work around it in spacemacs.

[dankessler]

I have opened a PR against bind-map that should hopefully fix these issues without requiring further modifications to spacemacs.

[tigerjack]

@tigerjack the local variable TeX-master seems to be working correctly for me, e.g., if I open a file that has

%%% Local Variables:
%%% mode: LaTeX
%%% TeX-master: "../main"
%%% End:

where there is a main.tex file in the parent directory, then things work properly, e.g., compiling with SPC a u compiles the parent file, not the child file which is open.

Your issue did make me realize that I had missed some keybinds in my PR, e.g., those dealing with reftex features (e.g., SPC R t does not give the reftex-toc overview. To avoid making it too kludgy, I'm going to rework the PR so that at the top of the latex layer's packages.el file, it figures out whether it should set configuration for latex-mode or LaTeX-mode (similarly for tex-mode and context-mode), sets some private variables appropriately, and then relies on those throughout.

I am not sure of what's happening behind the scenes to be honest. It seems that some files are opened in LaTeX/PS major mode, while others in TeX or TeX/PS. In the latter, the local variables are not updated.

When I try to force the activation of the LaTeX major mode and revert the buffer, the mode seems to be applied, but I get another error from helm saying helm-M-x-execute-command: Cannot open load file: No such file or directory, tex-buf. And of course the local variables are not update either.

On the other hand, if I do a C-c _ to set the master, the TeX-master variable is correctly applied.

I am still unsure if the behaviour is related to the auctex update, but I did not have this issue before. If you need some testing, I will be glad to help.

[dankessler]

@tigerjack I suspect the issue is with the way your major-modes are getting specified. If I understand correctly, in the new version of AUCTeX, TeX-mode isn't really a mode you should ever be in explicitly. From the "Changes" section of the AUCTeX manual:

          Note that ‘TeX-mode’ isn't meant for use for end users.  It is
          only meant for the base mode for other major modes.  Its role
          is to provide base keymap, hook and syntax table under the
          same name with the former AUCTeX and run the common
          initialization code.

Can you confirm which version of AUCTeX you are using with SPC h d v auctex-version RET?

My understanding of the way AUCTeX handles mode mapping (with new AUCTeX and emacs 29+) is as follows.

1. Open a file that ends with .tex
2. There is an entry in auto-mode-alist that maps this (via regexp) to tex-mode
3. An entry in major-mode-remap-alist remaps tex-mode to TeX-tex-mode
4. Per its documentation, TeX-tex-mode is essentially a virtual mode that tries to guess the appropriate mode to activate

When you somehow land in TeX-mode, something has presumably gone wrong in one of these steps. I suspect it's happened in step 4, but I can't be sure. If I were you, I'd look at the documentation for TeX-tex-mode (with SPC h d f TeX-tex-mode RET) to understand how it chooses which mode to activate based on various logic and variables, and you can see if you can understand why you're not landing in LaTeX-mode.

[James-Hanson]

Is there a quick fix for this issue at the moment?

[dankessler]

@James-Hanson asks:

Is there a quick fix for this issue at the moment?

One hack is to (painstakingly) duplicate the logic for latex-mode for LaTeX-mode, which is what I originally tried to do in #16283, but it turned out to be more painstaking than I was willing to stomach and so I came up with an upstream fix to bind-map.

I have "fixed" this for myself locally by using the version of bind-map that has my patch (which is currently sitting as a PR).

You can accomplish this by adding this entry to dotspacemacs-additional-packages

     (bind-map :location (recipe
                            :fetcher github
                            :repo "dankessler/emacs-bind-map"
                            :branch "feat-remap-aliased-modes"))

After restarting emacs, if you update packages with SPC f e U, it should offer to update bind-map using my patched version.

I frequently use this approach (of telling spacemacs to use a version of a package that I have patched) whenever I find and fix bugs for packages but am waiting for the PR to be adopted.

Unfortunately, the renaming of latex-mode -> LaTeX-mode has subtle ramifications for many packages that relied on major-mode being precisely equal to latex-mode (e.g., see this PR for {helm,ivy}-bibtex.

PS: the lsp part of this bug report has been resolved thanks to adoption of this PR.

EDIT: added quotes around the repo name; thanks @jajpater for catching that

[tigerjack]

Can you confirm which version of AUCTeX you are using with SPC h d v auctex-version RET?

Its value is "14.0.2"

My understanding of the way AUCTeX handles mode mapping (with new AUCTeX and emacs 29+) is as follows. [...]

I suspect it was related to a failure of the undo-tree history (a similar error is reported here).

Despite that, even if I force the LaTeX-mode, the TeX-master variable keeps having the value t instead of main.
The same thing happens with a file for which the LaTeX-mode is enabled automatically.

[tigerjack]

Upon further investigation, the problem seems to be related to this line of my configuration file

(add-hook 'LaTeX-mode-hook #'latex-extra-mode)

So I think it's a bug of the latex-extra-mode

EDIT: I fixed this error here Malabarba/latex-extra#38

[tigerjack]

@dankessler @James-Hanson
Upon further usage, I think an easy workaround would be to either declare buffer or directory variables. Let me know if it works for you as well.

File variable

Inside the .tex file, declare the following

%%% Local Variables: 
%%% mode: LaTeX
%%% End: 

Directory variable

Create a .dir-locals.el file in the directory where your .tex files are, and put the following content.

((latex-mode
  (mode . LaTeX )))

[dankessler]

Upon further investigation, the problem seems to be related to this line of my configuration file

(add-hook 'LaTeX-mode-hook #'latex-extra-mode)

So I think it's a bug of the latex-extra-mode

EDIT: I fixed this error here Malabarba/latex-extra#38

@tigerjack glad you got it sorted out and were able to submit a fix upstream

[dankessler]

File variable

Inside the .tex file, declare the following

%%% Local Variables: 
%%% mode: LaTeX
%%% End: 

Directory variable

Create a .dir-locals.el file in the directory where your .tex files are, and put the following content.

((latex-mode
  (mode . LaTeX )))

@tigerjack I'm a little surprised that this works for you, because I would expect that after the mode becomes LaTeX-mode, then the change-major-mode-after-body-hook fires, which causes bind-map to decide that you're not in latex-mode anymore, set spacemacs-latex-mode-map-active to nil, and disable all of the spacemacs-provided bindings.

[tigerjack]

@dankessler I honestly don't know what is the sequence of instructions that leads to this behaviour. What I can add, that maybe can help you in debugging the issue, is

1. The effect seems to be the same either if I put latex or LaTeX
2. The snippets are working
3. The compilation process (either C-c C-c xelatex or , b) works, but it opens in a tex-shell buffer. C-c C-l command does not seem to work properly, and the C-c ` is undefined.
4. , v to visualize the output pdf file works only after the first compilation, even if the pdf file is already present.

Let me know if I can help in any other way.

[atreyasha]

@dankessler thanks a lot for your bind-map patch, it fixes the issue for me:

#16282 (comment)

[jajpater]

Also a big thanks from me. It indeed fixes the issue for me. I had to add quotation marks to the repo part:

(bind-map :location (recipe :fetcher github :repo "dankessler/emacs-bind-map" :branch "feat-remap-aliased-modes"))

[James-Hanson]

@dankessler Thank you for the help, but it doesn't seem to be changing anything for me. I also tried it with quotes around the repo.

Is there some way to check to see if the version of bind-map my installation is using is actually your branch? I'm not seeing anything helpful in list-packages and I'm not seeing anything in the messages buffer indicating that it's actually pulling your branch.

I realized I was trying to do this with Emacs 28.2. I built 29.2 and was able to get it to work. Thanks again.

[dankessler]

@dankessler Thank you for the help, but it doesn't seem to be changing anything for me. I also tried it with quotes around the repo.

Is there some way to check to see if the version of bind-map my installation is using is actually your branch? I'm not seeing anything helpful in list-packages and I'm not seeing anything in the messages buffer indicating that it's actually pulling your branch.

I realized I was trying to do this with Emacs 28.2. I built 29.2 and was able to get it to work. Thanks again.

@James-Hanson thanks for letting me know and for sharing this update. I thought my fix would work for emacs 28, too, so I may take a slightly closer look to see if I can adjust the patch accordingly.

[dankessler]

Also a big thanks from me. It indeed fixes the issue for me. I had to add quotation marks to the repo part:

(bind-map :location (recipe :fetcher github :repo "dankessler/emacs-bind-map" :branch "feat-remap-aliased-modes"))

@jajpater thanks for noticing that! For whatever reason, it works for me without quotes, but it looks like surrounding with quotes is indeed the idiomatic way to specify recipes. I'll updated my earlier comment to use quotes for anyone who is copy-pasting it.

[tigerjack]

@tigerjack the local variable TeX-master seems to be working correctly for me, e.g., if I open a file that has

%%% Local Variables:
%%% mode: LaTeX
%%% TeX-master: "../main"
%%% End:

where there is a main.tex file in the parent directory, then things work properly, e.g., compiling with SPC a u compiles the parent file, not the child file which is open.

I am coming back on this again. In my case, while the SPC m a combination works well, the SPC m v returns the error message Output file "main" does not exist even if the file "main.pdf" exists. However, if I first build and then view, the error goes away. Am I the only one affected by this problem?

[vitaminace33]

Just here to say that I am expriencing this issue with

• Emacs 29.2
• Spacemacs Develop
• AUCTeX 14.0.3
• Texlab 5.12.4

I worked around it by downgrading AUCTeX to 13.3.0, which is obviously not a fix, but I depend too much on it.

Regards.

[dankessler]

My PR to bind-map has been merged, which fixes the primary complaint that this issue raised.

It sounds like some people (@tigerjack) are having different, but likely closely-related, issues with the newest version of AUCTeX. I'd suggest that we close this issue and open a fresh one to track any of those specific problems.

Also, I'd suggest people remove this hack from their config:

     (bind-map :location (recipe
                            :fetcher github
                            :repo "dankessler/emacs-bind-map"
                            :branch "feat-remap-aliased-modes"))

restart emacs, and then update packages to grab the latest version of bind-map which has this change integrated. You'll then continue to get any new updates from the upstream bind-map

[LailaElbeheiry1]

Just here to say that I am expriencing this issue with

• Emacs 29.2
• Spacemacs Develop
• AUCTeX 14.0.3
• Texlab 5.12.4

I worked around it by downgrading AUCTeX to 13.3.0, which is obviously not a fix, but I depend too much on it.

Regards.

Can you explain how you downgraded AUCTeX?

[tigerjack]

Even though I do not find any latex-related change in the latest emacs version (29.3), the upgrade nonetheless fixed all my problems when used with auctex 14.0.3.2024-03-17

[dankessler]

Even though I do not find any latex-related change in the latest emacs version (29.3), the upgrade nonetheless fixed all my problems when used with auctex 14.0.3.2024-03-17

In brief, this is because auctex renamed many of its major modes, e.g., latex-mode -> LaTeX-mode. If emacs >= 29, it uses major-mode-remap-alist in order to register this remapping. However, bind-map, which spacemacs relies on to handle much of its keymapping, was not aware of major-mode-remap-alist. I fixed that in a recent PR, so that now bind-map will activate keymaps for latex-mode when LaTeX-mode is activated provided this remapping was registered in major-mode-remap-alist.

However, AFAIK, bind-map has not been patched to be aware of the way that auctex registers the remapping for emacs < 29, which is probably why things don't work correctly if user emacs < 29.

[tigerjack]

@dankessler yes, I was on board with all these problems on 29.2 :)

However, I was still experiencing strange behaviour that you couldn't reproduce (as an example, , v to visualize the PDF output was not working when opening the .tex file, even though the PDF was there. However, building the document using , b was then making the , v shortcut to function).

Today's upgrade to emacs 29.3 fixed those strange behaviours as far as I can tell; maybe it was simply due to a recompilation of all the packages?!. On the other hand, opening .tex files now require considerable more time.

[dankessler]

@dankessler yes, I was on board with all these problems on 29.2 :)

Good; sorry for over-explaining :)

However, I was still experiencing strange behaviour that you couldn't reproduce (as an example, , v to visualize the PDF output was not working when opening the .tex file, even though the PDF was there. However, building the document using , b was then making the , v shortcut to function).

Today's upgrade to emacs 29.3 fixed those strange behaviours as far as I can tell; maybe it was simply due to a recompilation of all the packages?!. On the other hand, opening .tex files now require considerable more time.

Interesting! I'm on emacs 29.1 and not encountering any issues, but glad that upgrading to 29.3 fixed things for you!

[zzjjzzgggg]

A tex file uses \input to include another tex file. Moving the cursor to the included file name, and pressing SPC f f, I find that emacs cannot automatically select the included file, which is not the case before auctex being updated. Does anyone also suffer from this issue?

[tigerjack]

A tex file uses \input to include another tex file. Moving the cursor to the included file name, and pressing SPC f f, I find that emacs cannot automatically select the included file, which is not the case before auctex being updated. Does anyone also suffer from this issue?

I never used this shortcut, and I don't have it in my key bindings. What is the command executed?

[zzjjzzgggg]

SPC f f runs the command spacemacs/counsel-find-file.

[tigerjack]

@zzjjzzgggg which is an Ivy command. I never used it, nor it. But in theory, you should be able to use [ f for find-file-at-point. I tried it, and while it works on the scratch buffer, it does not work inside a latex buffer. I wonder if Ivy uses find-file-at-point internally.

[smile13241324]

I have worked a bit with my latex layer and it seems that with the many fixes from @dankessler this issue is obsolete now. If you run into other strange behaviors please open a new issue for them and I am sure folks will have a look at it.

[fnussbaum]

With Emacs 30, this problem appears again due to the introduction of major-mode-remap-defaults. A PR with a fix is submitted upstream: justbur/emacs-bind-map#11.