Feat: Improved module search

[Ell1ott]

Previously, when searching for the modules and you made a typo, it would not show it. This is now fixed/improved using the typo-safe-search npm package

Examples:

It is still not perfect, but I much prefer this behaviour.

[1zun4]

Is there any reason it is still on draft?

[Ell1ott]

Is there any reason it is still on draft?

Not anymore. Just wanted to make sure that there weren't any edge cases where it wouldn't work, but is now ready to be merged.

[SenkJu]

I see the library you are using is made by you. What distance function did you implement? It doesn't seem to be one I know.

[Ell1ott]

Well, I chose to write my own library and distance function because I couldn't find any good libraries that did both of the following:

• Would allow typos (as in inserting a character into the query that doesn't exist in the item). I, for example, found command-score, which does something similar but doesn't allow incorrect characters.
• Was expecting the user not to have completed the query. When the user searches for a module, they are very likely only to type the first few letters. The few distance functions that allowed typos expected the query string to already be complete. An example of this is the levenshtein

My distance function is slightly inspired by command-score but otherwise created by myself to solve the above-listed problems.

[SenkJu]

Hm, basic distance functions are generally considered a solved problem (see Levenshtein distance). Your implementation appears rather inefficient to me. Consider using something like the Wagner-Fischer algorithm for much higher efficiency.

[Ell1ott]

I ran some tests with the js-levenshtein library and got results where I would prefer the old system (just ranking it depending on if the query existed in the item). Let's take the following example. The user wants to search for 'poison' in a list of words and starts by typing 'poi.' Here, js-levenshtein would recommend the following words as the best for the query 'poi':

• pool
• spit
• war
• poison
• dose

Whereas my distance function would recommend these:

• poison
• productive
• proportion
• pool
• proof

Regarding the algorithm's performance, js-levenshtein was about twice as performant as my implementation in my tests. It took Levenshtein about 4364 ms to sort a list of 140 random words 60000 times. It took my implementation 7811 ms to do the same. In the end, this efficiency difference won't be noticeable to the end user because they both are extremely efficient, with mine being able to sort the list of 140 items 7600 times per second. And when testing it in the client i did not feel any speed difference to the old system.

If you know any alternatives to Levenshtein that would give better results, I would be happy to take a closer look at them 🙂

[Ell1ott]

After trying to further improve the performance of my algorithm, I have managed to bring it down to about 6000 ms to sort the list of 140 words 60000 times.

[Ell1ott]

@SenkJu Do you still have concerns or could we maybe merge?