Meeting setup
- Date: 13.8.2007
- Time: 09.30 Norw. time
- Place: Internet
- Tools: SubEthaEdit, iChat/Skype
Agenda
- Opening, agenda review
- Reviewing the task list from last week
- Documentation - divvun.no
- Corpus gathering
- Corpus infrastructure
- Infrastructure
- Linguistics
- name lexicon infrastructure
- Spellers
- Other issues
- Summary, task lists
- Closing
1. Opening, agenda review, participants
Opened at 10:00.
Present: Børre, Maaren, Per-Eric, Sjur, Thomas, Tomi
Absent: Saara, Trond
Agenda accepted as is.
2. Updated task status since last meeting
Børre
- add
smatexts to the corpus repository- not done
- run all known spelling errors in the prooftest corpus through the speller
- not done
- add extraction of all known spelling errors in the regular corpus (not the
prooftestcorpus), and check that they are properly corrected- not done
- update and fix our documentation and infrastructure
- not done
- follow-up contact with Davvi Girji
- Sent e-mail, no answer so far
- fix bugs!
- Fixed #454
- Other
- Trying to port sme from Xerox tools to SFST
- Trying to fix the G5, which has caused us some trouble lately.
Maaren
- lexicalise actio compounds
- not done
Per-Eric
- expand the smj typos list
- work and still working
- add missing smj words
- work and still working
- fix errors in
smj/src/typos.txt- working with it
- lexicalise words from the Olavi missing list
- work and still working
- test new speller for actios of 3-syllable verbs, adverbs of 3-s adjs and æ/ä
- done some tests
- add compounding tags to adjectives
- done
- ask SD/Sweden about
smjplace names (Per-Eric)- not done
Saara
- add new XSL/XML headers for proofing test docs
- Try to add files with Lars to the corpus interface.
- fix bugs!
Sjur
- run all known spelling errors in the corpus through the speller
- not done
- document the AppleScript testing tool
- not done
- integrate regression self tests with the make file
- not done
- improve speller test bench
- working on it
- integrate the ccat speller testing options in the make file
- working on it
- fix internet setup for Per-Eric’s satelite modem
- not done
- check with Sámi publishing houses whether support for CS2 is still needed
- one publisher still open, but I expect it won’t be a problem; will check this week.
- publish corpus contracts and project infra as open-source on NoDaLi-sta
- not done
- ask Trond whether Ilona can add the
smeplace names from Finland- not done
- fix bugs!
Thomas
- work with compounding
- still working
- fix stuorra-oslolaš lower case
o- not done
- test new speller for actios of 3-sybbable verbs, adverbs of 3-s adjs and æ/ä
- actios and adverbs are okey, æ/ä does not work, do not now why
- fix bugs!
- not done
Tomi
- make PLX conversion test sample; add conversion testing to the make file
- not done
- open up compounding for all actios
- done
- add Hunspell data generation/conversion
- not done
- add G5 bug to Bugzilla
- added
- make baseform extraction tool
- done
- fix bugs!
Trond
- Work on the web corpus issues
- update the
smjproper noun lexicon, and refine the morphological analysis, cf. the propernoun-smj-lex.txt - fix bugs!.
3. Documentation
TODO:
- write form to request corpus user account (Børre, Sjur, Trond)
- document how to apply for access to closed corpus, and details on the corpus and its use in general (Børre, Sjur, Trond)
- correct and improve documentation (infrastructure) (Børre)
4. Corpus gathering
Per-Eric will meet Sigga Tuolja-Sandström on his way home from Östersund.
Torkel Rasmussen will give texts to us, he will be in Tromsø in week 39-40, and transfer texts from his computer to ours then.
TODO:
- Go through the list of missing or errouneous
nobtexts, based upon Saara’s perfect list (Trond) - add
smatexts to the corpus repository (Børre)
5. Corpus infrastructure
Nothing this week either.
6. Infrastructure
The G5 is down/off the net after a system update, Børre is working on it.
TODO:
- update and fix our documentation and infrastructure (Børre)
- fix internet setup for Per-Eric’s satelite modem (Sjur)
7. Linguistics
North Sámi
The twol problems in sme were “solved” by revering the changes that caused
them. Thus, it works now, but with the flaws that started the whole process.
Thomas has finished compounding markup - very good!
The sme names from Finland is still not added to the lexicon (Børre has
them). This could be done by Ilona. Sjur will ask Trond.
TODO:
- lexicalise actio compounds. Example: vuolggasadji vs. vuolginsadji
(Maaren)
- possibly turn on free compounding as part of the PLX conversions (ie free
compounding in the spellers, but not in the analyzers/transducers)
- Tomi added this
- possibly turn on free compounding as part of the PLX conversions (ie free
compounding in the spellers, but not in the analyzers/transducers)
- fix stuorra-oslolaš lower case
o(Sjur, Thomas, Trond) - open up compounding for all actios in the PLX format (Tomi)
- done
- ask Trond whether Ilona can add the
smeplace names from Finland (Sjur)
Lule Sámi
The æ-ä alternation issue has turned into an interesting direction. With the latest speller (29.6.), it behaves like the following:
dæbbaga - ok
däbbaga - däbboga
dæbbaga
dæbbaga--
dibága
dibága--
dubága
dubága--
tubága
vællahit - ok
vällahit - vællahit
vellahit
gæhtjáj - ok
gähtjáj - ok
That is, ä works in some cases, but not in other. æ seems to work
everywhere it should.
Nils-Olof Sortelius at Sámediggi in Sweden (smj place names) is in
hospital - Per-Eric will talk to Kåre Tjikkom instead.
TODO:
- refine
smjproper noun lexica, cf. the propernoun-smj-lex.txt (Thomas, Trond) ä/æin speller (Thomas, Sjur)- does only work partially, see data above.
- lexicalise words from the Olavi missing list, but check against the pdf
original where in doubt (Per-Eric)
- still some to do
- actios of 3-syllable verbs must be checked in the next speller
(Thomas, Per-Eric)
- they are ok
- adverbs of 3-syllable adjectives must be tested in the next speller
(Thomas, Per-Eric)
- they are ok
- add compounding tags to (some weeks of work):
- nouns (Thomas)
- adjs (Per-Eric)
- ask SD/Sweden about
smjplace names (Per-Eric) - call Samuel Gælok about texts (Per-Eric)
- call Kåre Tjikkom about
smjplace names and correct-doc (Per-Eric)
8. Name lexicon infrastructure
Decisions made in Tromsø can be found in [this meeting memo.|/admin/physical_meetings/tromso-2006-08-propnoun.html]
TODO:
- fix bugs in lexc2xml; add comments to the log element (Saara)
- finish first version of the editing (Sjur)
- test editing of the xml files. If ok, then: (Sjur, Thomas, Trond)
- make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (Sjur, Saara)
- convert propernoun-($lang)-lex.txt to a derived file from common xml files (Sjur, Tomi, Saara)
- implement data synchronisation between risten.no and the cvs repo, and possibly other servers (ie the G5 as an alternative server to the public risten.no - it might be faster and better suited than the official one; also local installations could be treated the same way)
- start to use the xml file as source file
- clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (Thomas, Maaren, linguists)
- merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (linguists)
- publish the name lexicon on risten.no (Sjur)
- add missing parallel names for placenames (linguists)
- add informative links between first names like Niillas and Nils (linguists)
9. Spellers
OOo spellers
Tomi is working on the lexicon conversion to the Hunspell format. Some issues to solve. Software problems on the G5 - will be added to Bugzilla.
TODO:
- add Hunspell data conversion (Tomi)
- add G5 bug to Bugzilla (Tomi)
- done, solved - add note to the bug report about the actual problem (environment error on G5 account) (Tomi)
Testing
Spelling Error Markup
Steinar finished his job last week, and copied it all over to victorio. There is still some clean-up to do, mainly because the markup is added to the xml files, and not the originals. The present conversion process doesn’t handle error markup in this format.
TODO:
- Set up ways of adding meta-information (source info, used in testing or not, added to lexicon or not) (Saara)
- move the error markup in the xml files to (a copy of) the original (Børre)
Testing tools
TODO:
- document the AppleScript testing tool (Sjur)
- improve speller test bench (Sjur)
- integrate the
ccatspeller testing options in the Makefile (Sjur)
- integrate the
Regression tests
Tomi utilized oe of the java classes in the PLX conversion tool to
create a separate tool/command to extract the baseforms from the lexc
lexicons.
TODO:
- add extraction of all known spelling errors in the corpus (not the
prooftestcorpus), and check that they are properly corrected (Børre, Sjur) - how to do a regression self-test:
- extract all the base forms in the lexicon, and run them through the speller
- extract all SUB-marked entries, and run them through the speller
- integrate these in the make file (Sjur)
- make baseform extraction tool (Tomi)
- done
Lexicon conversion to the PLX format
It seems that we don’t need to get new licenses from either Polderland or Xerox
- Børre’s nice
sshsolution is working very well forsmj. There was a problem withsmethat needs to be investigated.
TODO:
- ask for mklex for Linux (victorio) from Polderland (Sjur)
- delayed until the present ssh solution is tested more
- ask Xerox for a commercial lisense for the xfst tools on the G5 (Sjur)
- delayed until the present ssh solution is tested more
Publicity follow-up
Per-Eric is being interviewed next week by Anna Sunna (SR Sámi Radio), and will be demonstrating installation and usage of the proofing tools.
It was cancelled by SR.
New public beta
Delayed till the majority of the present bugs are fixed.
10. Other
Corpus contracts
TODO:
- publish corpus contracts and project infra as open-source on NoDaLi-sta (Sjur)
Bug fixing
When fixing bugs, record the version number containing the fix in the Bugzilla bug report, such that for each bug, we know exactly when it should have been fixed, in what file(s) and what version.
56 open Divvun/Disamb bugs (21 of these 56 are speller bugs, 35 are general bugs), and 23 risten.no bugs
TODO:
- look over the Bugzilla status mails (Børre)
Board meeting
Sjur is going to Oslo on Thursday Aug. 16 to meet the board.
Project meeting
It would be good to have another project meeting sometime in September, to work on the hardest remaining issues. We’ll decide the date next week.
11. Next meeting, closing
The next meeting is 20.8.2007, 09:30 Norwegian time.
The meeting was closed at 10:57.
Appendix - task lists for the next week
Boerre
- add
smatexts to the corpus repository - run all known spelling errors in the prooftest corpus through the speller
- add extraction of all known spelling errors in the regular corpus (not the
prooftestcorpus), and check that they are properly corrected - update and fix our documentation and infrastructure
- follow-up contact with Davvi Girji
- fix bound corpus access for external users (documentation, application)
- move error markup in xml files to (a copy of) the originals
- fix bugs!
Maaren
- lexicalise actio compounds
Per-Eric
- expand the smj typos list
- add missing smj words
- fix errors in
smj/src/typos.txt - lexicalise words from the Olavi missing list
- test new speller for actios of 3-sybbable verbs, adverbs of 3-s adjs and æ/ä
- add compounding tags to adjectives
- ask SD/Sweden about
smjplace names - call Kåre Tjikkom about
smjplace names and correct-doc - call Samuel Gælok about texts
Saara
- add new XSL/XML headers for proofing test docs
- Try to add files with Lars to the corpus interface.
- fix bugs!
Sjur
- run all known spelling errors in the corpus through the speller
- document the AppleScript testing tool
- integrate regression self tests with the make file
- improve speller test bench
- integrate the ccat speller testing options in the make file
- fix internet setup for Per-Eric’s satelite modem
- check with Sámi publishing houses whether support for CS2 is still needed
- publish corpus contracts and project infra as open-source on NoDaLi-sta
- fix bound corpus access for external users (documentation, application)
- fix stuorra-oslolaš lower case
o ä/æinsmjspeller- restart work on the XML name editor/risten.no integration
- plan the rest of the project period
- fix bugs!
Thomas
- work with compounding
- fix stuorra-oslolaš lower case
o - test new speller for æ/ä
- fix bugs!
Tomi
- make PLX conversion test sample; add conversion testing to the make file
- add Hunspell data generation/conversion
- fix bugs!
Trond
- Work on the web corpus issues
- update the
smjproper noun lexicon, and refine the morphological analysis, cf. the propernoun-smj-lex.txt - fix bound corpus access for external users (documentation, application)
- fix stuorra-oslolaš lower case
o - fix bugs!.