UTF-8 compatibility #24
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixed all functions in Cenrallix's SQL language to support UTF-8. The character encoding is detected at startup and only the UTF-8 or the previous single-byte functions are registered. Also adding some initial support for mapping what the system calls certain character sets and what other software packages call them.
Re-added the UTF-8 functions to a rolled-back version of exp_functions.c to get proper code format and get rid of complete refactoring of code.
Really applied utf-8 set of functions ot exp-functions.c. The last commit did not actually include these.
This update added stSeparate for pStructInf files so one can pull out a specific subtree so that it is not freed along with the rest of the tree.
Added functionality to load a charsetmap file so that charsets can be added to the system so that a system administrator can define the equivalent charset names for their system charset.
This adds a basic charsetmap.cfg file with a lot of comments.
Also added a new utility module with the prefix chr. The header is utility/charsets.h and it contains a function for looking up the charset to be used by a module and some macros to specify attribute names that are allowed in charsetmap.cfg. Added one other macro to define the key in centrallix.conf that specifies the charsetmap file.
Changed some JavaScript files that add UTF-8 suppot to server.
Also added a little note in the QPrintf documentation documenting about JSSTR.
Also modified the corresponding functions in exp_functions.c.
- Added conversion to UTF-16 and glyph table lookup for postscript output (which also includes PDF output). - Glyph table obtained from Adobe, license compatible with GPL. - Glyph table modified for canonicalization and importation into C code - Still need to auto-select the correct font that has the requested glyph(s) in it. For instance - CJK fonts, etc. Right now, output formatter only uses times, helvetica, and courier, which are usually encoded in iso-8859-1.
… needed) - This commit causes the server to count UTF-8 encoded characters properly as just one character. However, we are still not looking up the correct font metric -- only the lower ASCII codes have proper font metrics in the server (grr).
…l. Updated mtlexer to properly handle reading long lines that could split utf-8 chars. Changed mlx_internal_WillSplitUTF8 to mlx_internal_willCharFitUTF8, and updated the function to match new name
…s. Added test for util's numBytesInChar.
…rlong tests to reflect current functionality.
…r unixuser driver. Fixed typo in pop3.
…g for mysql database to properly handle UTF-8
…ent headers. Added ability for gzip to intelligently truncate/replace .tar and .tgz file extensions
… uneeded comments in gzip driver and test.
…nd ux to be less flaky. Added nVerifyUTF8 for use with non null terminating strings. Fixed some spacing in mtlexer.h
… with failed PUt requests.. This includes change in net_http_rest.c. Updated name for report test #8
…unction to prevent crashes and conform to style. Fixed typo in mysql test. Updated mysql test readme. Fixed uxFiles test
…tf-8 Fixed a typo from the website
Final merge before pull request. Previous merge forgot to pull master.
|
I just re-merged with master (forgot to pull master last time) and everything works. Should be good to go |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #23
This branch should ideally be broken up into several smaller branches + pull requests for different parts the UTF-8 compability project.
There has been some "cursory" review of this code.