Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert byte offset to char, to fix wrong highlight in non ascii text #95

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Convert byte offset to char, to fix wrong highlight in non ascii text #95

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
54c68ec
convert char x shift to bytes
artur-shaik Nov 30, 2020
87d4c07
update readme
artur-shaik Nov 30, 2020
a40fc16
add idents
artur-shaik Dec 1, 2020
450b30c
enable char to byte convertion by default
artur-shaik Dec 1, 2020
126163d
use match case in regexp
artur-shaik Dec 1, 2020
3e35c1e
use match case in regexp
artur-shaik Dec 1, 2020
057485f
Merge branch 'byte-to-char' of github.com:artur-shaik/vim-grammarous …
artur-shaik Dec 2, 2020
aa2caff
get rid of hacky code
artur-shaik Mar 3, 2021
9bb3945
cleanup
artur-shaik Mar 26, 2021
deade37
add `line-by-line` argument
artur-shaik Mar 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,10 @@ Please set `g:grammarous#show_first_error` to `1`. It opens an information windo
Please set `g:grammarous#use_location_list` to `1`. It sets all grammatical errors to location list.
This variable is set to `0` by default to avoid conflicts of location list usage with other plugins.

### I want to highlight error by char count not byte

Please set `g:grammarous#convert_char_to_byte` to `1`. This should fix wrong highlight in UTF-8 text.

## Automatic installation

This plugin attempts to install [LanguageTool](https://www.languagetool.org/) using `curl` or `wget` command at first time.
Expand Down
52 changes: 42 additions & 10 deletions autoload/grammarous.vim
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ let g:grammarous#hooks = get(g:, 'grammarous#hooks', {
let g:grammarous#languagetool_cmd = get(g:, 'grammarous#languagetool_cmd', '')
let g:grammarous#show_first_error = get(g:, 'grammarous#show_first_error', 0)
let g:grammarous#use_location_list = get(g:, 'grammarous#use_location_list', 0)
let g:grammarous#convert_char_to_byte = get(g:, 'grammarous#convert_char_to_byte', 0)
artur-shaik marked this conversation as resolved.
Show resolved Hide resolved

highlight default link GrammarousError SpellBad
highlight default link GrammarousInfoError ErrorMsg
Expand Down Expand Up @@ -180,6 +181,30 @@ function! s:set_errors_to_location_list() abort
endtry
endfunction

function! s:convert_char_to_byte(errs) abort
for e in a:errs
let line_from = getline(str2nr(e.fromy)+1)
let line_to = getline(str2nr(e.toy)+1)
let ch_from = strcharpart(line_from, e.fromx, 1)
let e.errorlength = len(strcharpart(line_from, e.fromx, e.errorlength))
artur-shaik marked this conversation as resolved.
Show resolved Hide resolved
let e.fromx = byteidx(line_from,str2nr(e.fromx))+1
let e.tox = byteidx(line_to,str2nr(e.tox))
if ch_from =~ '\(\s\|[`<>!@#$%^&*(){}\[\].,:;\"''\\/]\)'
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check looks very ad-hoc.

  • What is a purpose of this condition? I could not read the intention
  • Please use =~# since =~ behavior depends on user's configuration

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said in first message byteidx sometimes return not enough bytes, I couldn't determine why. That's why I used such hacky solution to shift highlight if first character is a symbol or whitespace.

Copy link
Owner

@rhysd rhysd Dec 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not understanding the problem well. Would you help me to understand it by showing some example which requies this hack?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is example result without this conditional shift:

2020-12-14-113438_863x345_scrot

Second match is missed one character. Sometimes byteidx result is not enough for matchaddpos for one byte. That's why I used this hack to find if we are on word's start position.

And here result with shifting:
2020-12-14-113957_895x347_scrot

let e.fromx += 1
if e.fromy == e.toy
let e.tox += 1
endif
endif
endfor
endfunction

function! s:shift_x(errs) abort
for e in a:errs
let e.fromx += 1
let e.tox += 1
endfor
endfunction

function! s:set_errors_from_xml_string(xml) abort
let b:grammarous_result = grammarous#get_errors_from_xml(s:XML.parse(substitute(a:xml, "\n", '', 'g')))
let parsed = s:last_parsed_options
Expand All @@ -194,11 +219,17 @@ function! s:set_errors_from_xml_string(xml) abort
return
endif

if g:grammarous#convert_char_to_byte
call s:convert_char_to_byte(b:grammarous_result)
else
call s:shift_x(b:grammarous_result)
endif

let len = len(b:grammarous_result)
echomsg printf('Detected %d grammatical error%s', len, len > 1 ? 's' : '')
call grammarous#highlight_errors_in_current_buffer(b:grammarous_result)
if parsed['move-to-first-error']
call cursor(b:grammarous_result[0].fromy+1, b:grammarous_result[0].fromx+1)
call cursor(b:grammarous_result[0].fromy+1, b:grammarous_result[0].fromx)
endif

if g:grammarous#enable_spell_check
Expand Down Expand Up @@ -396,9 +427,9 @@ function! s:matcherrpos(...)
return matchaddpos('GrammarousError', [a:000], 999)
endfunction

function! s:highlight_error(from, to)
function! s:highlight_error(from, to, length)
if a:from[0] == a:to[0]
return s:matcherrpos(a:from[0], a:from[1], a:to[1] - a:from[1])
return s:matcherrpos(a:from[0], a:from[1], a:length)
endif

let ids = [s:matcherrpos(a:from[0], a:from[1], strlen(getline(a:from[0]))+1 - a:from[1])]
Expand All @@ -419,9 +450,10 @@ function! grammarous#highlight_errors_in_current_buffer(errs)
if !g:grammarous#use_fallback_highlight
for e in a:errs
let e.id = s:highlight_error(
\ [str2nr(e.fromy)+1, str2nr(e.fromx)+1],
\ [str2nr(e.toy)+1, str2nr(e.tox)+1],
\ )
\ [str2nr(e.fromy)+1, str2nr(e.fromx)],
\ [str2nr(e.toy)+1, str2nr(e.tox)],
\ e.errorlength
\ )
endfor
else
for e in a:errs
Expand Down Expand Up @@ -560,7 +592,7 @@ function! s:binary_search_by_pos(errors, the_pos, start, end)
endif

let m = (a:start + a:end) / 2
let from = [a:errors[m].fromy+1, a:errors[m].fromx+1]
let from = [a:errors[m].fromy+1, a:errors[m].fromx]
let to = [a:errors[m].toy+1, a:errors[m].tox]

if s:less_position(a:the_pos, from)
Expand All @@ -582,7 +614,7 @@ endfunction

function! grammarous#fixit(err)
if empty(a:err)
\ || !grammarous#move_to_checked_buf(a:err.fromy+1, a:err.fromx+1)
\ || !grammarous#move_to_checked_buf(a:err.fromy+1, a:err.fromx)
\ || a:err.replacements ==# ''
call grammarous#error('Cannot fix this error automatically.')
return
Expand Down Expand Up @@ -774,7 +806,7 @@ endfunction

function! grammarous#move_to_next_error(pos, errs)
for e in a:errs
let p = [e.fromy+1, e.fromx+1]
let p = [e.fromy+1, e.fromx]
if s:less_position(a:pos, p)
return s:move_to_pos(p)
endif
Expand All @@ -785,7 +817,7 @@ endfunction

function! grammarous#move_to_previous_error(pos, errs)
for e in reverse(copy(a:errs))
let p = [e.fromy+1, e.fromx+1]
let p = [e.fromy+1, e.fromx]
if s:less_position(p, a:pos)
return s:move_to_pos(p)
endif
Expand Down
10 changes: 5 additions & 5 deletions autoload/grammarous/info_win.vim
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ let s:save_cpo = &cpo
set cpo&vim

function! grammarous#info_win#action_return()
call grammarous#move_to_checked_buf(b:grammarous_preview_error.fromy+1, b:grammarous_preview_error.fromx+1)
call grammarous#move_to_checked_buf(b:grammarous_preview_error.fromy+1, b:grammarous_preview_error.fromx)
endfunction

function! grammarous#info_win#action_fixit()
Expand All @@ -13,7 +13,7 @@ function! grammarous#info_win#action_remove_error()
let e = b:grammarous_preview_error
if !grammarous#move_to_checked_buf(
\ b:grammarous_preview_error.fromy+1,
\ b:grammarous_preview_error.fromx+1 )
\ b:grammarous_preview_error.fromx )
return
endif

Expand All @@ -24,7 +24,7 @@ function! grammarous#info_win#action_disable_rule()
let e = b:grammarous_preview_error
if !grammarous#move_to_checked_buf(
\ b:grammarous_preview_error.fromy+1,
\ b:grammarous_preview_error.fromx+1 )
\ b:grammarous_preview_error.fromx )
return
endif

Expand All @@ -34,7 +34,7 @@ endfunction
function! grammarous#info_win#action_next_error()
if !grammarous#move_to_checked_buf(
\ b:grammarous_preview_error.fromy+1,
\ b:grammarous_preview_error.fromx+1 )
\ b:grammarous_preview_error.fromx )
return
endif

Expand All @@ -46,7 +46,7 @@ endfunction
function! grammarous#info_win#action_previous_error()
if !grammarous#move_to_checked_buf(
\ b:grammarous_preview_error.fromy+1,
\ b:grammarous_preview_error.fromx+1 )
\ b:grammarous_preview_error.fromx )
return
endif

Expand Down