rup.diff

329d328
< Updated ignore patterns. 2019-10-23T19:03:04+00:00
335d333
< Force unix line endings, to make sure it works ok also on the Windows subsystem for Linux. 2019-10-07T17:52:29+00:00
344d341
< Updating svn ignores for tools/analysers/. 2019-06-14T06:39:23+00:00
349,350d345
< Updating svn ignores. 2019-05-24T09:58:34+00:00
< Updating svn ignores. 2019-05-24T09:45:30+00:00
360d354
< Updated svn ignores. 2019-02-27T10:22:06+00:00
372d365
< Ignore more files, including files that are automatically added to svn when populating a new language. This is done to avoid them showing up as noise for external languages, in which case these files might not be in our svn (but in the external svn repo instead). 2019-01-08T07:01:30+00:00
402d394
< Updated svn ignores. 2018-09-25T08:02:11+00:00
407d398
< More general ignore pattern for tools/mt/apertium/tagsets/. 2018-09-10T11:17:14+00:00
410d400
< Updated svn ignore patterns. 2018-09-08T05:21:34+00:00
420d409
< Updated svn ignores. 2018-08-30T16:17:17+00:00
423d411
< Updated svn ignores. 2018-08-29T05:36:10+00:00
425d412
< Updating svn ignores. 2018-08-28T10:49:42+00:00
439d425
< More things to ignore. 2018-05-14T10:30:20+00:00
453,454d438
< Added svn:ignore for files being output of testing. 2018-03-01T06:37:16+00:00
< Added svnignore pattern for sigma.txt. 2018-02-21T10:00:49+00:00
457d440
< Two more files to ignore. 2018-02-06T09:46:57+00:00
468d450
< Updated svn ignores. 2018-01-31T12:15:22+00:00
497d478
< Updated svn ignores. 2017-12-11T12:54:11+00:00
518,519d498
< Updated svn ignores for tokenisers and grammar checkers + subdirs. 2017-10-11T11:59:35+00:00
< Updated svn ignores for tokenisers and grammar checkers + subdirs. 2017-10-11T11:06:09+00:00
531d509
< Updating svn ignores. 2017-08-25T10:17:49+00:00
545,546d522
< Updating svn ignores. 2017-06-29T00:05:40+00:00
< Updated svn ignores. 2017-06-28T22:59:36+00:00
565d540
< Updated svn ignores. 2017-03-01T12:03:10+00:00
581d555
< Updated svn ignores. 2017-01-30T09:58:10+00:00
638a613,614
> Fix the lav and rup links once again, this time hopefully correctly 2016-07-15T12:20:44+00:00
> Fix symbolic links 2016-07-13T18:24:22+00:00
649d624
< Updated svn ignores. 2016-06-09T20:05:58+00:00
667d641
< Setting svn ignore patterns on tools/spellcheckers/filters/. 2016-05-10T01:01:42+00:00
688d661
< Ignore more preprocessor files = fst’s. 2016-04-14T16:01:35+00:00
692d664
< Updated svn ignores. 2016-03-15T19:55:06+00:00
695d666
< Use a more general svn ignore pattern in src/morphology/. 2016-03-07T17:11:09+00:00
715d685
< Updated the svn ignore property for recent changes in the infrastructure. 2016-02-16T22:31:44+00:00
721d690
< Updating svn:ignore’s. 2016-02-02T15:31:48+00:00
726,727d694
< Updated svn:ignore’s. 2016-02-02T10:35:45+00:00
< Updated svn:ignore’s. 2016-02-02T09:04:35+00:00
731d697
< Updated svn ignores. 2016-01-25T08:12:56+00:00
743d708
< Updated svn:ignore’s. 2015-11-18T23:09:59+00:00
758d722
< Updated svn ignores. 2015-10-20T07:52:23+00:00
783d746
< Ignore temporary files generated by the speller suggestion test script. 2015-09-02T20:01:50+00:00
827d789
< Ignore txt files in speller dirs. 2015-04-09T11:49:00+00:00
834c796,1030
< Moved rup from langs/ to startup-langs/. Nothing has happened since the initial commit, and core source files are missing. It seems to be clearly in the startup phase. If the move is not motivated, it is easy to move back or revert. 2015-04-01T10:26:42+00:00
---
> [Template merge - langs/und] Require new version of the core, updated comments about Err tags. 2015-03-30T10:05:11+00:00
> +Err/Sub -> +Err/Orth, according to the meeting 19.3., with notes in http://divvun.no/doc/lang/common/ErrorTags.html. Commandos used for the tag change: 2015-03-29T09:36:09+00:00
> [Template merge - langs/und] Removed CmpNP tags from downcase-derived-proper-strings.xfscript.in. 2015-03-19T07:33:07+00:00
> [Template merge - langs/und] When doing 'make clean', remove generated html files in the root dir. 2015-03-14T10:23:58+00:00
> [Template merge - langs/und] Removed multichar definition of superfluous flag diacritics. 2015-03-13T13:47:40+00:00
> [Template merge - langs/und] Added a new directory named devtools/ to each language, with the idea that it should contain tools useful for development, but not necessarily suitable for automake testing. Initially it contains shell scripts to generate a table of generated word forms for each continuation lexicon. 2015-03-13T10:24:25+00:00
> [Template merge - langs/und] Removed corpus names from tools/spellcheckers/fstbased/hfst/data/Makefile.am. It caused the build to stop with an error for all languages except FIN. 2015-03-12T08:44:38+00:00
> [Template merge - langs/und] Make building the abbr.txt configurable (default=no), check for the existence of src/morphology/stems/abbreviations.lexc, and error out if not found. 2015-03-12T06:26:03+00:00
> [Template merge - langs/und] Forgot to include the new Makefile (r109076) in configure.ac. 2015-03-12T05:42:59+00:00
> [Template merge - langs/und] Preparations for supporting corpus-based frequency weights, as per TommiP. 2015-03-11T17:28:31+00:00
> [Template merge - langs/und] Enabled weighting of speller fst's. Adjust weights and tags as needed. 2015-03-11T13:21:18+00:00
> [Template merge - langs/und] Added support for all languages to generate the abbr.txt file used by $GTCORE/scripts/preprocess. At the same time added initial support for compiling pmatch scripts into fst's for hfst-proc2, which is the future alternative to preprocess. 2015-03-09T14:52:03+00:00
> [Template merge - langs/und] Forgot to remove some debug statements from the yaml test runner. Now cleaned. 2015-03-06T16:21:47+00:00
> [Template merge - langs/und] Moved MWE tag processing into the core - we want this for many languages. 2015-03-06T15:24:28+00:00
> [Template merge - langs/und] Added support for a new type of yaml tests: speller acceptance testing. The basic idea is to just give a list of words and word constructions (compounds, derivations, etc) the speller should accept or reject, and let the yaml test bench verify whether this is actually the case. 2015-03-06T13:07:55+00:00
> [Template merge - langs/und] Several changes to properly support all position-based +CmpN/XX tags: * moved tag path splitting and tag-to-flag conversion into separate regex files in the core. * added support for compiling and using the new regexes * added support for a new type +CmpN/Suff * added the required multichar symbols to the root.lexc files * increased required core version number 2015-03-05T15:13:56+00:00
> [Template merge - langs/und] Fixed a bug in the yaml test bench when both hfst and xfst was enabled, but where only one type is built, e.g. for Apertium. 2015-03-05T14:08:15+00:00
> [Template merge - langs/und] Added build support for alternate orthographies: default fst's, dicts and oapha. 2015-03-04T15:22:16+00:00
> [Template merge - langs/und] Fixed a bug that caused the wrong fst to be picked in certain cases, which caused the test script to fail. 2015-03-04T10:51:23+00:00
> [Template merge - langs/und] A couple of changes related to testing: * require Python 3.3+ * require new gtcore * update YAML test runner to make SMS testing work as intended also with Xerox 2015-03-03T16:31:36+00:00
> [Template merge - langs/und] Added support for country/region specific proofing tools in configure.ac. 2015-03-02T12:50:54+00:00
> [Template merge - langs/und] We do not support anything but the latest/newest Voikko now. 2015-02-27T12:40:01+00:00
> [Template merge - langs/und] Finalised the basic multiple writing system support, by adding support for Oahpa and dictionary fst's. 2015-02-27T11:12:23+00:00
> [Template merge - langs/und] Added a configuration flag to enable two-step compose-intersect. In most cases this will not make any difference, but for some languages it will correct a bug in compose-intersect that would otherwise create a bad fst, and for other languages it will make the operation much slower without changing the fst. Disabled by default, whether it is useful must be tested in each case / language. 2015-02-27T09:11:18+00:00
> [Template merge - langs/und] Corrected errors in hfst compilation of alternative writing system fst's. 2015-02-26T17:54:48+00:00
> [Template merge - langs/und] Added test runners for genation and analysis tests only for the descriptive fst. 2015-02-26T12:28:20+00:00
> [Template merge - langs/und] Compilation of the default set of fst's with alternate writing systems working. 2015-02-26T08:30:12+00:00
> [Template merge - langs/und] First step in adding support for alternate writing systems and orthographies: adding variables to configure.ac. Removed the variable LO_min_version, it isn't used. 2015-02-25T06:44:28+00:00
> +Mal -> +Sem/Mal 2015-02-24T17:59:54+00:00
> +Fem -> +Sem/Fem 2015-02-24T17:59:24+00:00
> +Clth -> +Sem/Clth 2015-02-24T17:58:39+00:00
> +Veh -> +Sem/Veh 2015-02-24T17:58:13+00:00
> +Edu -> +Sem/Edu 2015-02-24T17:57:46+00:00
> +Build -> +Sem/Build 2015-02-24T17:57:16+00:00
> +Wthr -> +Sem/Wthr 2015-02-24T17:56:41+00:00
> +Measr -> +Sem/Measr 2015-02-24T17:56:09+00:00
> +Route -> +Sem/Route 2015-02-24T17:55:45+00:00
> +Txt -> +Sem/Txt 2015-02-24T17:55:17+00:00
> +Time -> +Sem/Time 2015-02-24T17:54:43+00:00
> +Plant -> +Sem/Plant 2015-02-24T17:53:59+00:00
> +Group -> +Sem/Group 2015-02-24T17:53:26+00:00
> +Hum -> +Sem/Hum 2015-02-24T17:52:19+00:00
> +Ani -> +Sem/Ani 2015-02-24T17:49:52+00:00
> +Org -> +Sem/Org 2015-02-24T17:47:53+00:00
> +Obj -> +Sem/Obj 2015-02-24T17:46:34+00:00
> +Sur -> +Sem/Sur 2015-02-24T17:45:58+00:00
> +Plc -> +Sem/Plc 2015-02-24T17:44:44+00:00
> [Template merge - langs/und] Split the m4/ax_python_module.m4 file, it contained mostly java autotools stuff. Improved the message to update the gtcore. 2015-02-24T15:14:55+00:00
> [Template merge - langs/und] Added the make-optional-hyph-tags filter to the generators. Fixes bug #1914. 2015-02-12T22:25:31+00:00
> [Template merge - langs/und] Make use of the new remove-adv_comp filter. Require new core and newest hfst. 2015-02-12T15:05:06+00:00
> [Template merge - langs/und] Put to use the make-optional-adv_comp filter. 2015-02-12T11:31:19+00:00
> [Template merge - langs/und] Don't build xerox fst's within the Apertium dir tree - no need for it. 2015-02-12T11:19:12+00:00
> [Template merge - langs/und] Require new core because of new filters. Use hfst-optimized-lookup in the yaml testing (but only if possible), this should speed up hfst testing quite a bit. 2015-02-11T21:35:12+00:00
> [Template merge - langs/und] Put the new optional minip filter to use, and increased required gtcore version. 2015-02-10T23:03:18+00:00
> [Template merge - langs/und] Replaced all instances of sub and lexsub filters with the new, generated error filters. 2015-02-10T20:36:01+00:00
> [Template merge - langs/und] Added support for extracting error tags and constructing filters for manipulating error strings and tags. Updated required version of gtcore. 2015-02-10T14:38:16+00:00
> [Template merge - langs/und] Remove variant tags in disamb analyser. 2015-02-10T13:47:44+00:00
> [Template merge - langs/und] Xerox fst's are irrelevant to Apertium, don't even try to build them. 2015-02-10T09:13:18+00:00
> [Template merge - langs/und] Use the new make-optional-v1-tags filter for apertium generators. 2015-02-09T19:36:54+00:00
> [Template merge - langs/und] Forgot to include the new regex in the src file listing in the previous commit. 2015-02-09T17:56:22+00:00
> [Template merge - langs/und] Corrected dictionary generators to require a variant tag except for +v1, which is optional. 2015-02-09T17:40:34+00:00
> [Template merge - langs/und] Removed 'invert net' from a couple of more instances. 2015-02-09T16:59:36+00:00
> [Template merge - langs/und] Treat Hfst and Xerox the same during *tmp.Xfst and *.Xfst build - invert both only in the last step when going from tmp to non-tmp fst (invert the analyser for hfst, the generator for xfst). This should remove one more confusing difference between the two. 2015-02-09T15:16:34+00:00
> [Template merge - langs/und] Check that we have at least Python3.1 when enabling Apertium, error out if not. Also add AM check for hfst-optimized-lookup. 2015-02-09T14:42:48+00:00
> [Template merge - langs/und] A small, functionally equivalent change: from suffix rule to pattern rule. 2015-01-30T16:39:05+00:00
> [Template merge - langs/und] Now +CmpN/Pref is correctly supported (earlier it was treated as +CmpN/First). 2015-01-30T12:38:49+00:00
> [Template merge - langs/und] Corrected fst file reference in test shell script. 2015-01-29T20:00:45+00:00
> [Template merge - langs/und] Corrected source file reference in test shell script. 2015-01-29T18:23:24+00:00
> [Template merge - langs/und] Changes to a couple of Makefile.am files to fix issues with 'make dist'. 2015-01-29T09:44:45+00:00
> [Template merge - langs/und] The last part of the CmpN location restriction flag diacritics added. 2015-01-29T07:45:04+00:00
> Makefile.in does not belong in svn. 2015-01-28T08:43:20+00:00
> [Template merge - langs/und] Code cleanup: no use for the M4 part - the null alternative did not work. 2015-01-27T19:51:12+00:00
> [Template merge - langs/und] Finally nailed all combinations of fst compilator and lexicon minimisation - now downcasing of derived proper nouns is working as it should again for both Xerox, Hfst hyperminimised and Hfst normal lexc compilation. 2015-01-27T15:17:43+00:00
> [Template merge - langs/und] +CmpN/Only supported, first steps in tag splitting taken. 2015-01-26T20:43:50+00:00
> [Template merge - langs/und] Moved code common to all yaml testrunner shell scripts to an include file in GTCORE to avoid code duplication and reduce the risk for introducing bugs. This requires the newest version of the CORE. Because of the inclusion, I had to rename the test runner to .sh.in, and added autoconf processing of it. Also added a test file for testing the base speller fst (it must be tailored to each language of course). 2015-01-26T13:52:29+00:00
> [Template merge - langs/und] Last change to get hyperminimisation to produce the correct output: made the derived-proper downcase script being processed by autoconf, so that we can require a symbol in a certain context, and at the same time in the end let the symbol be empty if not needed. 2015-01-22T10:38:50+00:00
> [Template merge - langs/und] Added optional flag diacritic inserted by Hfst hyperminimisation. This resolves the remaining cases of errors after the hfst team fixed a bug in lexc compilation with hyperminimisation turned on. Since it is optional, it does not make any harm when using Xerox or when not using hyperminimisation. 2015-01-22T08:36:09+00:00
> [Template merge - langs/und] Added xerox variable flag-is-epsilon to the tag reorder regex. This fixes most of the cases of errors after the hyperminimisation bug was fixed in hfst-lexc. The remaining errors must be fixed in the downcase-derived-proper regex. 2015-01-22T07:15:43+00:00
> [Template merge - langs/und] Added more silent builds for hfst tools. 2015-01-20T19:52:00+00:00
> [Template merge - langs/und] Added conversion of tags to flag diacritica for position-restricting tags. These are currently used in sma, sme, smj and sje. Together with some additions to the R lexicon, the tags will finally do what they are meant to do for hfst-based spellers. 2015-01-20T09:03:18+00:00
> [Template merge - langs/und] Added Multichar symbol definitions for flag diacritica controlling compounding based on position tags. Done for most langs, the symbols will be ignored if not used. 2015-01-19T12:05:06+00:00
> [Template merge - langs/und] New: added example test file for the fstspeller fst file (starting point for foma and hfst spellers). 2015-01-14T18:13:34+00:00
> [Template merge - langs/und] Fixed: errors in the yaml test runner when the fst has a suffix 'hfst'. 2015-01-14T18:02:57+00:00
> [Template merge - langs/und] Fixed: directory and fst names in the yaml runner shell script. 2015-01-14T17:25:24+00:00
> [Template merge - langs/und] Added support for yaml tests for speller fst's. 2015-01-14T16:59:34+00:00
> [Template merge - langs/und] Added support for Xerox fst's in tools/spellcheckers/fstbased, mainly to help in debugging hfst. Turned out to be very useful. 2015-01-13T23:02:08+00:00
> [Template merge - langs/und] Improved comments to make the lemma generation script easier to adapt. 2015-01-13T15:39:49+00:00
> [Template merge - langs/und] Additions to generate the inverted fst's, to enable symmetric yaml testing. 2015-01-13T06:07:54+00:00
> [Template merge - langs/und] Fixed: order of filter application was wrong, causing all Use/-Spell forms to be included in the spellers. 2015-01-12T21:29:54+00:00
> [Template merge - langs/und] Fixed: error in easter egg building after the previous commit. 2015-01-09T10:04:41+00:00
> [Template merge - langs/und] Make sure the easter egg is rebuilt every time the fst is rebuilt. 2015-01-09T09:20:40+00:00
> [Template merge - langs/und] Fixed: The MacVoikko target contained one subtarget that built even when spellers were not enabled, and thus failed because of a missing dependency. 2015-01-08T09:25:45+00:00
> [Template merge - langs/und] A number of changes to make the MacVoikko.service build cleanly with proper dependency tracking. Also a bit safer cleaning. 2015-01-07T11:03:29+00:00
> [Template merge - langs/und] Fixed: The MacVoikko target was missing from noinst_DATA, thus it was not built. 2015-01-07T09:30:48+00:00
> [Template merge - langs/und] Two template merges: * Added initial support for building language-specific macosx systemwide spellers. [r105104] * Added strip function to get rid of extra spaces, resolves bug in abbr.txt build. [r104185] 2015-01-05T15:48:16+00:00
> [Template merge - und] Included lexc files in src/morphology/ in the abbr file making. 2014-11-19T11:40:02+00:00
> [Template merge - und] Expanded the source file base for building the abbr file, more like the old infra. 2014-11-19T11:27:57+00:00
> [Template merge - und] Only delete (aka 'make clean') generated corpus files used for weighting if such files exist. Removes a very dangerous 'rm -rf .*' command. 2014-11-18T07:26:09+00:00
> [Template merge - und] Fixed bug in the phonology building that caused extra source files not to be compiled. 2014-11-14T09:48:27+00:00
> [Template merge - und] Removing Use/LexSub strings from all normative fst's. Fixes bug #1904. 2014-11-11T09:35:01+00:00
> [Template merge - und] Added support for turning off building of vislcg3/syntactic tools. 2014-11-04T10:29:56+00:00
> [Template merge - und] Improvements and corrections in the README file. 2014-10-28T22:59:14+00:00
> [Template merge - und] Changed Hfst configuration: * moved xerox check before hfst check to ... * automatically enable hfst if the Xerox tools are not found * moved minimum version requirement definition to configure.ac * removed hfst-foma requirement, instead checking for all required tools * removed path check for obsolete hfst tools * improved hfst configuration messages * updated the summary text to reflect that hfst is automatically enabled These changes should ease configuration on systems without Xerox. 2014-10-28T22:08:01+00:00
> [Template merge - und] Corrected names of compiled twolc files in test/src/phonology/pair-test*.sh.in. We need to use the 'compose' fst because compiled twolc files are not treated the same as other fst's. We can't just skip the new lookup-friendly filenames either, because morphophonological rules can be written using xfscript, in which case the lookup renaming (and inversion) is essential. 2014-10-27T14:00:29+00:00
> [Template merge - und] Corrrected references to the new lookup style fst names. Fixes broken inituppercase tests. Updated config header in initcap yaml file correspondingly. 2014-10-23T09:08:36+00:00
> [Template merge - und] Now both general and language-pair specific relabelling using regexes are supported, in addition to using relabel files. The regexes allow context-dependent and multisymbol changes, whereas the relabel files only cover 1:1 mappings of single symbols. The actual change was to add support for regex files in the language-pair independent processing. The tools/mt/apertium/tagsets/README.txt file was more or less completely rewritten to better document the filenames being recognised, and how they should be used. 2014-10-23T06:03:36+00:00
> [Template merge - und] Retain the regular non-optimised hfst analyser for easy paradigm generation using a regex plus composition. 2014-10-21T10:55:38+00:00
> [Template merge - und] Fixed a bug in the Apertium build that blocked building of AP-tagged analysers. 2014-10-20T19:09:04+00:00
> [Template merge - und] Make sure there is always an apertium analyser for 'und' if nothing else. 2014-10-20T14:32:59+00:00
> [Template merge - und] Do not remove homonymy tags from the apertium fst's. Also simplified the automatic conversion by moving all non-automatic changes to a separate file, run as a sort of tag conversion postprocessing. Updated the tagset/README.txt file to contain info aobut the manually maintained postprocessing relabel file. Added an initial postprocessing relabel file containing word boundary and homonymy tag changes. 2014-10-14T15:45:50+00:00
> [Template merge - und] Do not remove homonymy tags from the regular analysers. 2014-10-14T14:15:14+00:00
> [Template merge - und] Fixed a bug in building Oahpa generators - orig-lang tags were not removed. Clean *.hfstol files in tools/mt/apertium/. 2014-10-14T11:07:16+00:00
> [Template merge - und] Moved Apertium tagset creation and relabeling from src/tagsets/ to tools/mt/apertium/tagsets/. This should fix building of apertium fst's for fin, smn. 2014-10-10T14:18:40+00:00
> [Template merge - und] Renamed AWK to GAWK in relevant places to get around another AWK test. Now gawk is found properly in all cases. 2014-10-09T10:17:56+00:00
> [Template merge - und] Improved test for gnu awk. 2014-10-09T07:49:46+00:00
> [Template merge - und] Require newest core to force people to upgrade to get an important bugfix. 2014-10-07T13:16:16+00:00
> [Template merge - und] Fixed a bug in the core for generated regexes - a reserved char was not escaped. Required core version bumped. 2014-10-06T07:26:13+00:00
> [Template merge - und] Hfst 3.8.0 is out, with a number of important bug fixes and improvements, including new options required to make our code build properly. 2014-10-03T19:50:14+00:00
> [Template merge - und] Several changes to accomodate a downcaseerror variant of the L2 error fst for Oahpa: * added configure.ac option --enable-downcaseerror (independent of the L2 opt) * a number of changes to the build instructions for Oahpa to support the new fst * made the error fst compilation independent of whether an L2 twolc/xfscript file is used - if not, it will just use the ordinary twolc/xfscript file. This way it is possible to build a downcaseerror fst without starting L2 development. * svn-copied regexes from the old to the new infra, including to the core * increased gtcore version number and required version number due to new regexes 2014-10-02T14:10:42+00:00
> [Template merge - und] Corrected wrong filenames and file references that blocked the oahpa L2 build. 2014-10-02T06:33:23+00:00
> [Template merge - und] Tagset relabeling didn't work for xfst files, now it does. Also generalised the use of relabel files (for use with hfst-relabel). 2014-10-02T06:18:41+00:00
> [Template merge - und] Simplified the building of hfst's with alternative tagsets, now that the *.hfst files are not in optimised lookup format. Silenced regex compilation. 2014-10-01T14:38:45+00:00
> Removed all instances of 'invert net' - it is just wrong and confuses the use of the fst's. The inversion needed for proper behavior of Xerox' lookup tool is automatically taken care of when building *.lookup.xfst. This way all fst's should behave correctly in all situations with all tools and operations - just make sure you pick the right fst! (see the end of the filename) 2014-10-01T05:39:54+00:00
> [Template merge - und] Last part of the lookup & composition cleanup: phonetics and phonology now covered. Now all non-lexical and non-filter files have a suffix .compose.* or .lookup.* depending on their intended use, and they are all properly inverted where needed (i.e. only for Xerox' lookup tool). There might still be source files to clean, but that is a separate step. 2014-09-30T22:43:32+00:00
> [Template merge - und] Corrected a couple of cases where old filenames were still used, and thus broke compilation. Also improved filtering of transcriptors, and constructed transcriptor target names dynamically based on the source files. 2014-09-30T21:22:42+00:00
> [Template merge - und] Xfscript and lookup cleanup: now we explicitly build files made for lookup and composition marked in the filenames. This is done for hyphenation and for orthography, phonology and phonetics still to be done. From now on there should be no need to use invert as part of the xfscript code - DON'T DO IT! All targets updated to use the new filenames. Removed inversion from the hyphenation xfscript. 2014-09-30T19:35:01+00:00
> [Template merge - und] Use explicit pipe mode with hfst-xfst. 2014-09-30T10:28:14+00:00
> [Template merge - und] Moved Apertium target language specification from configure.ac to tools/mt/apertium/Makefile.am. Changed the target filename construction to better follow the Apertium naming scheme. Fixed a bug introduced about four weeks ago that destroyed the dependency chain (due to a bug/fragileness in GNU make). 2014-09-30T09:42:32+00:00
> [Template merge - und] Cleaned up building of target fst's using the lookup-include.am file. Now all hfst transducers in optimised lookup format have the suffix .hfstol, and optimisation should not be hidden or implisit anymore. All test scripts should be updated as well. Also move all common targets from src/Makefile.am to am-shared/src-dir-include.am and sub-included AM files. This cleans up the src/ dir Makefile.am quite a lot. 2014-09-29T15:59:02+00:00
> [Template merge - und] Added support for additional local lexc files not part of the lexical fst. 2014-09-29T11:38:09+00:00
> [Template merge - und] Several changes to clean up the mess with the transcriptors: * moved transcriptor final builds from src/ to src/transcriptions/ * renamed transcriptor source files and targets * streamlined transcriptor compilation to use lexc-include and lookup-include * also silenced xfst in lookup-include.am 2014-09-26T08:25:41+00:00
> [Template merge - und] There were a couple of issues in the previous commit: * vpath directive didn't work reliably * L1 and L2 variabless were declared for easy merging, but in a way that AM didn't like * forgot to change the name of the lexical fst in the filter processing 2014-09-24T11:47:53+00:00
> Resolving L1 and L2 in src/morphology/Makefile.am 2014-09-23T19:33:36+00:00
> [Template merge - und] Several changes to accomodate L2 (language learner) analysers for Oahpa: * removed silent build instructions from twolc-include (they are taken from the   silent-build-include instead) * added support for compiling L2 phonology/twolc files when configured to * renamed $(GTLANG)-lexc.?fst to just lexicon.?fst. * added support for the error analyser in src_oahpa-include.am * added configure support for the L2 analyser (off by default) * added support for building the L2 lexical fst using L2 source files * added variables a.o. to support specifying L2 source files in src/morphology/ 2014-09-23T09:38:14+00:00
> [Template merge - und] Added support for filters written in lexc and xfscript. Renamed variables and added a lexc-include.am file to support general lexc compilation. 2014-09-22T12:02:22+00:00
> [Template merge - und] Fixed an unfortunate AM syntax error that blocked Automake, and thus all builds. 2014-09-19T20:36:50+00:00
> [Template merge - und] Three template updates at once: * Cleaned the filter build files even more. Now only local / language specific regex source files need to be listed in the local Makefile.am. * Fixed a problem with MT filter compilation that only revealed itself in sme. * Another filter build cleanup: all filter regexes in core are now built for all languages. One obsolete filter was removed. 2014-09-18T17:05:45+00:00
> [Template merge - und] Added a new filter to the filter compilation. Used the new filter to build correct fst's for dictionary analysis and generation. Increased the version number of the required gtd core version, due to the new and required filter in the core. 2014-09-18T07:33:48+00:00
> [Template merge - und] Major cleanup of filter and tagset compilation: * moved all non-local data and build instructions into am-shared/ * created dir-specific am-include files * clean use of regex-include.am * removed sme-specific source files from tools/mt/apertium/tagsets/Makefile.am * switched the apertium filter use to use the one built in src/filters/ instead of rebuilding it 2014-09-17T11:02:53+00:00
> [Template merge - und] analyser-oahpa-gt-desc should be analyser-oahpa-gt-norm. Now renamed. 2014-09-15T15:56:29+00:00
> [Template merge - und] The listbased speller fst is now generated properly using both Xerox and Hfst. 2014-09-15T12:55:36+00:00
> [Template merge - und] Fixed a logical error that turned off all hfst spellers. Renamed a variable. 2014-09-15T09:54:55+00:00
> [Template merge - und] Only build Apertium tagsets in tools/mt/ if Apertium is turned on. 2014-09-15T08:36:07+00:00
> [Template merge - und] Corrected a syntax error in the src_disamb-include.am file. Moved all fst trimming of general interest from tools-spellcheckers-listbased to tools-spellcheckers. Made the configuration so that list-based spellers will only compile if configured to build Hunspell. Also tried to make the configuration of other spellers such that they are automatically off when spellers are off. 2014-09-15T05:58:27+00:00
> [Template merge - und] Downcasing of the initial letter of derived proper nouns (Pariisi -> pariisilainen) is now finally working with Hfst. It requires Hfst svn rev. 4000. Batch two. 2014-09-12T09:44:24+00:00
> [Template merge - und] Downcasing of the initial letter of derived proper nouns (Pariisi -> pariisilainen) is now finally working with Hfst. It requires Hfst svn rev. 4000. 2014-09-11T22:02:11+00:00
> [Template merge - und] The first major step for adding support for generating list-based spellers such as Hunspell and the PLX (Polderland/MS Word) spellers. The conversion is not trivial, since we try to control compounding according to the linguistic specifiation in the lexicon (using tags). Although PLX is only for three Sámi languages, Hunspell conversion should be useful for all languages in our infrastructure. No real Hunspell or PLX files produced yet, only prerequisite fst's. - At the same time fixed a glitch in the version checking of VislCG3 that would turn off support for CG files now that the vislcg3 svn revision number has turned 10 000. 2014-09-08T20:31:14+00:00
> [Template merge - und] Added support for local overrides of the base speller fst. 2014-09-05T20:14:28+00:00
> [Template merge - und] Generalised and simplified the code for building oxt's - no more hard-coded filenames. Now the LO-voikko versions supported as well as the platforms are just defined in two variables, and the rest follows from there. The build code also handles cases of unsupported combinations of voikko versions and platforms. Also silenced the build quite a lot in non-verbose mode. 2014-09-04T07:58:01+00:00
> [Template merge - und] Switched to universal binary build for the LO41 voikko OXT. 2014-09-02T10:02:32+00:00
> [Template merge - und] Made the hfst optimised lookup file format explicit by using the .hfstol suffix, and by optimising files for lookup in a separate build step, instead of implicitly as before. So far only for tools/mt/apertium/, but more will come. Removed the removal of semantic tags - they are already optional, which should be more flexible and robust. 2014-08-27T11:52:53+00:00
> [Template merge - und] Made speller minimisation default to yes, specified where to push weights. 2014-08-26T07:02:38+00:00
> [Template merge - und] Added --encode-weights to determinise and minimise. This fixed the never-ending compilation of Finnish spellers. 2014-08-25T14:16:32+00:00
> [Template merge - und] The optimisations that worked for Greenlandic didn't work for Finnish, potentially due to Finnish being corpus-weighted and thus posing more challenges to determinisation and minimisation. Because of this the Greenlandic optimisation is now rolled into the configuration option --enable-minimised-spellers, OFF by default. 2014-08-22T17:31:28+00:00
> [Template merge - und] Added size and speed optimisations to the speller compilation process: remove-epsilons, push-weights, determinise and minimise. Together this made the KAL speller *much* smaller and *much* faster. It is now as fast and small as any other fst-based speller. 2014-08-22T09:09:54+00:00
> [Template merge - und] Hyperminimisation seems to be stable now, and I have added it as a standard configuration option. Also added autoconf support for the preliminary tool hfst-proc2, to facilitate easier testing of the tokeniser/analyser. 2014-08-21T08:45:06+00:00
> [Template merge - und] Updated the tagset targets to support Xerox fst's, and tagset replacement using regexes instead of the hfst-only relabel tool. Now all languages can get localised analysis and generation tags by adding a regex file and specifying a few targets. 2014-08-19T16:34:45+00:00
> [Template merge - und] Added build step to explicitly convert hfst transducers to optimised lookup format. Whitespace changes in the silent rule variables. Included the new lookup-include file in src-dir-include.am. 2014-08-19T11:26:55+00:00
> [Template merge - und] Preparations for better handling of lookup & testing of free-standing lexc and rewrite rule transducers: added build rules to do inversion of fst's intended for lookup. 2014-08-19T07:49:12+00:00
> [Template merge - und] Added a test dir for the upcoming hfst-based tokeniser. 2014-08-19T06:54:14+00:00
> [Template merge - und] Corrected some paths to enable VPATH building of spellers. Added support for retaining intermediate files when building using "make --debug". 2014-08-15T06:52:19+00:00
> [Template merge - und] Added support for building OXT for LO/OOo 3.6-4.0 for Mac. Language support is limited. 2014-08-11T09:05:30+00:00
> [Template merge - und] Properly clean src/morphology/. 2014-08-06T12:14:21+00:00
> [Template merge - und] Encapsulated most shell variable names in {} to handle hyphens etc in the variable names (after merge/update substitution of the __UND__ string). 2014-08-06T10:22:55+00:00
> [Template merge - und] Added a dir src/morphology/generated_files/ containing files generated during the build process. This is done to make a clear separation between files to be edited and files to be ignored. Also added a directory src/morphology/incoming/ to hold incoming lexical resources used to build the lexc or xml source files. Both dirs have a 00README.txt file explaining their use. 2014-08-06T08:10:56+00:00
> Remove white space at line endings 2014-07-10T14:39:41+00:00
> Added WANT_OAHPA option for analysers for all languages (until now only generator). Added the relevant line to the WANT_OAHPA targets in LANG/src/Makefile.am. 2014-07-01T14:20:22+00:00
> Added oahpa analyser as target (oahpa here meaning L2 transducer). This makes it possible to make alternative transducers modeling typical L2 errors/behaviour. 2014-07-01T14:09:10+00:00
> [Template merge - und] Fixed a bug in sigma extraction on certain Linux systems. 2014-06-22T06:51:06+00:00
> [Template merge - und] Better/more generalised handling of tag modifications for Apertium. 2014-06-20T20:34:21+00:00
> [Template merge - und] Added removal of lines marked '#RemoveFromApertium' from the apertium cg3 files. 2014-06-13T19:32:26+00:00
> [Template merge - und] Removed temporarily the downcasing of derived proper nouns from the hfst transducers - it causes them to become malfunctioning. 2014-06-11T11:54:50+00:00
> [Template merge - und] Fixed bug introduced yesterday that broke compilation of certain xfscript files. 2014-06-10T15:57:20+00:00
> [Template merge - und] Added support for testing the fst for initial upper casing of strings. This also includes yaml test support for non-analysing/-genereting fst's. 2014-06-09T18:14:55+00:00
> [Template merge - und] Properly handle downcasing of derived proper nouns as well as optional initial upper case. The optional initial upper-casing doesn't work for derived proper nouns when using Hfst because of an unimplemented featuare in hfst-xfst. It is reported to the hfst team. Increased gtd core version number due to new scripts and possible dependencies in the gtd core. 2014-06-06T11:19:13+00:00
> Use/Sub to Err/Sub Command used: cd $GTHOME perl -p -i -e 's/Use\/Sub/Err\/Sub/g' `grep -rl 'Use/Sub' * 2> /dev/null` 2014-06-04T08:13:54+00:00
> [Template merge - und] Extracting flag diacritics, to build regexes that can ignore them in certain cases (like optional initial upper case). Requires new version of the gtd core. At the same time split tag extraction in two - the first step extracts the whole sigma set, and from that we can extract tags, flag diacritics, etc. The sigma set extraction was greatly improved, removing a number of small errors due to handling of reserved symbols in Hfst and Xfst. 2014-05-26T07:42:09+00:00
> [Template merge - und] Added test summary for all yaml tests for a given fst. 2014-05-19T06:37:44+00:00
> suta sh- , not suta. 2014-05-18T10:26:43+00:00
> Added reference to the lexicon HUNDRED ; directly to the initial Root section, to get the numbers 100-999. Then added a 0 to the reference to HUNDRED  in the  UNDERTHOUSAND lexicon: :%0 HUNDRED ; Now 1-	   1000 gives 1000 analyses. 2014-05-18T08:00:05+00:00
> debug infos 2014-05-18T06:30:56+00:00
> corrected whitespace btw suti/suta and rest; corrected pl/sg of 100: number congruency 2014-05-18T06:16:15+00:00
> fugenvowel btw tens and ones 2014-05-18T06:01:50+00:00
> more corrections 2014-05-18T05:59:49+00:00
> commentet out sma ordinals 2014-05-18T05:19:16+00:00
> tiny correction 2014-05-17T21:37:14+00:00
> deleted by mistake: restored 2014-05-17T21:12:26+00:00
> clean up, corrections: watch for congruence in plural 'unã sutã' but 'dau suti', 'trei dau suti', etc. 2014-05-17T21:09:06+00:00
> joint work between Ciprian and Trond: modeling rup numbers 2014-05-17T21:05:13+00:00
> Dummy from nob. 2014-05-17T19:11:26+00:00
> [Template merge - und] With feedback from Brendan I finally got the number of tests passed and failed printed as part of the YAML testing. 2014-05-16T00:55:55+00:00
> [Template merge - und] Adaption to a new version of the morph-tester.py script by Brendan Molloy. Small adjustments to the yaml test printouts. 2014-05-15T21:40:20+00:00
> [Template merge - und] Major bug fix to the generate lemma test script. Now it actually checks that the generated lemmas correspond to the listed ones. Several of the languages had local modifications. Those are brought forward to the new version, to my best effort. 2014-05-12T20:18:44+00:00
> [Templage merge - und] Bugfix: no hardcoded language codes. 2014-05-12T11:25:23+00:00
> [Template merge - und] Now also (language pair independent) morphological generators for Apertium are installed with their correct Apertium file names. 2014-05-12T10:56:35+00:00
> [Template merge - und] Added renaming to Apertium style filenames, changed installation file list to only include files actually used by Apertium. With this change, everything should be in place for a fully automatic integration between the GT-Divvun infrastructure and the Apertium infrastructure through the use of pkg-config files, with one exception: morphological generators. 2014-05-12T07:19:57+00:00
> [Template merge - und] A rewritten pc file, with proper paths actually reflecting where things are installed, and with a shortened description to better fit the use of it. 2014-05-09T11:01:21+00:00
> [Template merge - und] We also need to install the pkg-config file... 2014-05-09T10:25:51+00:00
> [Template merge - und] After a long discussion, the moniker 'giella' was chosen instead of gtdivvun. Changed datadir from $(datadir)/gtdivvun/* to $(datadir)/giella/*. Added a pkg-config file so that all installed resources can be found automatically. 2014-05-09T10:03:49+00:00
> [Template merge - und] Changed datadir from $(datadir)/hfst/* to $(datadir)/gtdivvun/*, as it is the directory used to install the gtdivvun products, and not only hfst transducers are installed. 2014-05-08T08:13:33+00:00
> [Template merge - und] Require Automake 1.11.6 to avoid errors caused by older Automake's. 2014-05-07T12:05:03+00:00
> [Template merge - und] Make semantic tags optional also for dict and oahpa generators. Added support for hfst fst's for dict and oahpa. 2014-04-29T07:08:58+00:00
> [Template merge - und] Make semantic tags optional for all generators. Fixes bug http://giellatekno.uit.no/bugzilla/show_bug.cgi?id=1854. 2014-04-29T05:58:17+00:00
> [Template merge - und] Uncommented the cg3-with-apertium-tags targets, increased the gtcore version number. 2014-04-28T14:36:03+00:00
> [Template merge - und] Started work on adding hyphenators. No substantial changes, just Automake conditionals. 2014-04-25T20:12:28+00:00
> [Template merge - und] Actually made the options --disable-analysers and --disable-generators do what they should, earlier they had no effect. Also renamed those options. Wrapped the filter targets in mt/tools/apertium/filters/ in apertium conditionals, so that they will only be built if the apertium option is enabled. Added separate configure.ac option to disable the transcriptors (the num2text family). 2014-04-25T11:34:46+00:00
> [Template merge - und] Make sure all tests are within conditionals - only run them if the fst's have been built. 2014-04-24T16:19:26+00:00
> [Template merge - und] Added conversion of analysis tags from GTDivvun format to Apertium format for the vislcg3 files. The generated vislcg3 files are not valid, and the targets are thus commented out for now. 2014-04-23T07:53:44+00:00
> [Template merge - und] Added support for tmp files in the apertium target language specific analysers, to allow local processing of those analysers. Added more comments to explain the build process. 2014-04-17T06:30:25+00:00
> [Template update - und] Rewrote tag reordering of semantic tags to use a dynamically generated regex, and split tag reordering in three: reordering sub-POS tags, semantic tags, and language specific tags. The two first reordering operations are done on all languages. The reordering is done when building the raw file, to build a fixed tag order that other fst operations can rely on. The raw file build had to be split in two steps because of this. 2014-04-16T10:06:12+00:00
> [Template merge - und] Added support for target-language specific filtering for the Apertium analysers. 2014-04-15T16:06:41+00:00
> [Template merge - und] A major update to the Apertium fst building: * corrected broken logic when building the list of tags used by a language * build filter to remove derivation strings dynamically from the list of tags * added a new taglist2remove...strings-regex.sh file to the core * added a new dir filters/ within tools/mt/apertium/ for building apertium specific filters * added facility to modify locally remove...strings.regex files by using an exception file * build the remove-derivation-strings.regex dynamically also for regular fst's 2014-04-15T13:19:37+00:00
> Corrected path to yaml testfiles. 2014-04-13T06:57:05+00:00
> [Template merge - und] Now building the remove dialect tag filter dynamically, in the same way as done for the semantic tags. Requires a new version of the GTD core. 2014-04-11T09:48:05+00:00
> [Template merge - und] Dialect tags are now removed in the Apertium fst compilation. In addition, tags can now be custom changed and reordered on a language pair basis, see README.txt in tools/mt/apertium/tagsets/. 2014-04-10T15:06:42+00:00
> [Template merge - und] Corrected several errors in the MT Apertium fst builds: now removing semantic tags and tags for originating language. Silent hfst-invert. 2014-04-10T09:37:39+00:00
> [Template merge - und] Modified the gttags.txt target to produce output also in cases where no GTD tags are defined (this is the case in some of the experimental languages). Earlier the build would break in this case. 2014-04-10T07:37:13+00:00
> [Template merge - und] Commented out another debug echo statement. 2014-04-09T07:34:09+00:00
> [Template merge - und] Commented out a debug echo statement. 2014-04-08T19:07:17+00:00
> [Template merge - und] Fixed a bug with optional semantic tags: we built the regex, but not the fst's. 2014-04-08T11:09:21+00:00
> [Template merge - und] Corrected a bug in the lexc yaml testing. Fixed file refs in the dict fst tests. 2014-04-07T08:43:18+00:00
> [Template merge - und] Moved yaml test scripts for different transducer types up one level, to correspond to the parallel location of the fst files in the build tree. 2014-04-03T11:30:12+00:00
> [Template merge - und] Generalised the yaml test runner code, to automatically identify the relative paths of the test scripts and the fst's being tested, so that all sorts of fst's can be tested irrespective of where they are built. Added yaml testing for MT/Apertium. 2014-04-03T09:55:51+00:00
> what 2014-04-02T13:37:39+00:00
> docurup/doc/AromanianDocumentation.jspwiki 2014-04-02T13:32:59+00:00
> [Template merge - und] Moved some back-end scripts for yaml testing to the uppermost test directory, to ease sharing of the same code across test subdirectories. Updated paths to the new location. 2014-04-02T12:44:05+00:00
> [Template merge - und] Moved all silencing code to a separate include file (except in a few cases of double includes). Made the yaml testing a bit more verbose when rerunning individual tests (copy-paste testing). 2014-04-02T10:44:33+00:00
> [Template merge - und] Changed target language specifc analysers to be based off of analyser-mt-gt-desc.hfst, instead of the *.tmp.hfst file, to allow local post processing to be applied in the step from *.tmp.hfst to the *.hfst file. 2014-04-02T06:44:53+00:00
> [Template merge - und] Forgot to remove all the targets and build instructions in the old location. 2014-04-01T15:43:08+00:00
> [Template merge - und] Finalised moving the Apertium MT build code to the new location. All parts have been generalised, and the set of target languages to go with a specific source language (when analysing) is specified in configure.ac. That is, just list your target languages in configure.ac, and off you go. One feature still missing: target language derivation (and other) string filtering for the source language analyser. Coming soon. 2014-04-01T14:36:20+00:00
> symlinks 2014-04-01T10:00:28+00:00
> [Template merge - und] Reorganised MT fst building, moving it to a new dir in tools/. This is done to avoid too much stuff in one dir (src/), and to make it easier to extend the MT support without making the build files too large for one dir. 2014-03-31T14:08:46+00:00
> update: sma2rup 2014-03-30T11:15:52+00:00
> the Bitola alphabet 2014-03-30T08:54:06+00:00
> added Aromanian = rup 2014-03-29T19:35:59+00:00