CIDER completion system overhaul

[toniz4]

I'm starting this discussion topic to gather some opinions and findings regarding the CIDER completions.

Some context of this is at #3006 #3226 #3019 #3653

[toniz4]

I think I got CIDER to play nicer with flex completion:

In that picture I was using (add-to-list 'completion-category-overrides '(cider (styles flex)))

The main thing is not relying on the prefix passed by the completion function:

cider/cider-completion.el

Line 196 in 176a8e7

(lambda (prefix pred action)

But getting the prefix using symbol-at-point

(defun cider--get-prefix ()
  "Return the prefix at point as a string"
  (symbol-name (symbol-at-point)))

And using this in a function to return the completions, instead of using cider-complete directly in complete-with-action

(defun cider--capf-complete ()
  (let* ((prefix (cider--get-prefix))
         (results (cider-complete prefix)))
    (if results
        (setq cider--completion-cache results)
      cider--completion-cache)))

I'm using cider--completion-cache there because it's possible that the prefix passed to cider-complete returns no completions, like when typing util/relongfuncname for example. But we can just return the last completions results an let the "frontend" do the filtering.

Here's how the capf is, it's basically the same, I just remade the behavior of complete-with-action to ease debugging, and used cider--capf-complete instead of cider-complete directly.

(defun cider-complete-at-point ()
  "Complete the symbol at point."
  (when-let* ((bounds (bounds-of-thing-at-point 'symbol)))
    (when (and (cider-connected-p)
               (not (or (cider-in-string-p) (cider-in-comment-p))))
      (list (car bounds) (cdr bounds)
            (lambda (prefix pred action)
              ;; When the 'action is 'metadata, this lambda returns metadata about this
              ;; capf, when action is (boundaries . suffix), it returns nil. With every
              ;; other value of 'action (t, nil, or lambda), 'action is forwarded to
              ;; (complete-with-action), together with (cider-complete), prefix and pred.
              ;; And that function performs the completion based on those arguments.
              ;;
              ;; This api is better described in the section
              ;; '21.6.7 Programmed Completion' of the elisp manual.
              (cond ((eq action 'metadata) `(metadata (category . cider))) ;; defines a completion category named 'cider, used later in our `completion-category-overrides` logic.
                    ((eq (car-safe action) 'boundaries) nil)
                    ((null action) (try-completion prefix (cider--capf-complete)))
                    ((eq action t) (all-completions prefix (cider--capf-complete)))
                    (t (test-completion prefix (cider-complete prefix)))))
            :annotation-function #'cider-annotate-symbol
            :company-kind #'cider-company-symbol-kind
            :company-doc-buffer #'cider-create-compact-doc-buffer
            :company-location #'cider-company-location
            :company-docsig #'cider-company-docsig))))

I've been using this for some time now, seems to be working fine. Still needs to be tested with orderless and other completions styles and compatibility with company-mode (I use corfu)

@minad do you think this looks sound?