Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support persian characters #4

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Conversation

hmmftg
Copy link

@hmmftg hmmftg commented Jul 4, 2023

No description provided.

@ravarage
Copy link

ravarage commented Sep 7, 2023

I will test this against Kurdish chars, they are mostly either Arabic or persian thank you

@ravarage
Copy link

ravarage commented Sep 9, 2023

I don't know what you did, I tried this and a lot of thing even in Arabic got broken, so I made my own solution, I want to compare accuracy with original library, not this fork, guess what?, the original beside missing chars, it is working great, now my solution does support Persian too, not sure if I should release it or add it to this library, I will do some benchmark and compare success rate between them, then I decide
it support Arabic, Urdu,Persian and Kurdish
it support remove harakat , shift harakat, remove tatweel , zero width chars and use unshaped instead of isolate

@hmmftg
Copy link
Author

hmmftg commented Oct 9, 2023

I don't know what you did, I tried this and a lot of thing even in Arabic got broken, so I made my own solution, I want to compare accuracy with original library, not this fork, guess what?, the original beside missing chars, it is working great, now my solution does support Persian too, not sure if I should release it or add it to this library, I will do some benchmark and compare success rate between them, then I decide it support Arabic, Urdu,Persian and Kurdish it support remove harakat , shift harakat, remove tatweel , zero width chars and use unshaped instead of isolate

can you commit your changes to let us now your solution?

@ravarage
Copy link

https://github.com/ravarage/go-arabic-reshaper
its very slow compare to this solution, just by making it one loop it improve the performance by %50, I Will try to make it feature reach by adding linguist chars but for right now I am too busy , and this got the job done for me

@DearRude
Copy link

Any updates on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants