Language Technology at UiT

The Divvun and Giellatekno teams build language technology aimed at minority and indigenous languages

View GiellaLT on GitHub divvungiellatekno/giellalt.uit.no

Meeting setup

Agenda

  1. Opening, agenda review
  2. Reviewing the task list from last week
  3. Documentation - divvun.no
  4. Corpus gathering
  5. Corpus infrastructure
  6. Infrastructure
  7. Linguistics
  8. name lexicon infrastructure
  9. Spellers
  10. Other issues
  11. Summary, task lists
  12. Closing

1. Opening, agenda review, participants

Opened at 09:37.

Present: Børre, Ilona, Sjur, Thomas, Tomi

Absent: Per-Eric, Trond

Agenda accepted as is.

2. Updated task status since last meeting

Børre

Ilona

Maaren

Per-Eric

Saara

Sjur

Thomas

Tomi

Trond

3. Documentation

We want to automatise as much as possible when releasing new public betas. Suggestion: we make a fully automatic (Bugzilla-RSS + Bugzilla filter) list of all fixed bugs, and a manual summary of the “highlights”. Only the highlights get translated into different languages.

TODO:

4. Corpus gathering

Nothing new.

TODO:

5. Corpus infrastructure

Nothing.

6. Infrastructure

Nothing.

7. Linguistics

North Sámi

TODO:

Lule Sámi

smj propernoun bug issue: procedure

  1. convert from common base (which means sme base)
    1. Words not convertable should be added to separate smj lexicon, and words that should not be converted from sme sme should be moved to non-convert lexicon in sme???
  2. send to smj morphology

The original todo was to correct the smj morphology. Current test shows weaknesses in both camps:

Testing procedures:

Suggestion: Let us first analyse the proper noun base forms for Lule Sámi, and thereafter look at the morphology.

TODO:

8. Name lexicon infrastructure

This sub-project needs to get up and running soon. Mainly Sjur’s task.

Decisions made in Tromsø can be found in [this meeting memo.|/admin/physical_meetings/tromso-2006-08-propnoun.html]

TODO:

  1. fix bugs in lexc2xml; add comments to the log element (Saara)
  2. finish first version of the editing (Sjur)
  3. test editing of the xml files. If ok, then: (Sjur, Thomas, Trond)
  4. make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (Sjur, Saara)
  5. convert propernoun-($lang)-lex.txt to a derived file from common xml files (Sjur, Tomi, Saara)
  6. implement data synchronisation between risten.no and the cvs repo, and possibly other servers (ie the G5 as an alternative server to the public risten.no - it might be faster and better suited than the official one; also local installations could be treated the same way)
  7. start to use the xml file as source file
  8. clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (Thomas, Maaren, linguists)
  9. merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (linguists)
  10. publish the name lexicon on risten.no (Sjur)
  11. add missing parallel names for placenames (linguists)
  12. add informative links between first names like Niillas and Nils (linguists)

9. Spellers

OOo spellers

Børre will try to help out on this, as there are quite a few existing bugs waiting to be solved by Tomi.

TODO:

Testing

Spelling Error Markup

TODO:

Automated testing

We need one more baseform test, one in which all baseforms are run through the nonrec transducer. Such a test will identify all/most errors in the lexicon and errors in the baseform extraction. The remaining errors are then related to speller conversion.

cat gt/doc/proof/spelling/testing/selftest-pl-sme-20070909.xml | grep \
'<original>' | cut -d">" -f2 | cut -d"<" -f1 | lookup -flags mbTT \
-utf8 gt/sme/bin/sme.fst | grep '\?' | cut -f1 | wc -l

Lex test of the lule-specific words. Three were not recognised:

Sálatduottar    Sálatduottar    +?
Várjjat Várjjat +?
Fatjatj Fatjatj +?

Abbreviations are currectly printed twice if they should be followed by a full stop - once with and once without the full stop. - Fixed during the meeting.

TODO:

Lexicon conversion to the PLX format

Several smj bugs discovered during last week, and added to Bugzilla: 495, 503, 509, 512. General bugs as well: 421, 486.

Clitics:

 +Clt+ge:#ge ENDLEX ;
 +Clt+ge:#k ENDLEX ;

 +Clt+gen:#gen ENDLEX ;
 +Clt+ga:#ga ENDLEX ;

The first two are variants of the same clitic, where the variation is governed by twolc rules. It needs to be generated as part of the PLX generation.

The two other ones are separate clitics, and do not vary according to any rules. They can thus be put in a separate PLX file to save space and time during compilation. This solution requires that we give them PLX codes that will prevent them from combining with word forms already outfitted with the first clitic. Tomi will look into this, and report back. The issue may require discussion with Polderland (ie a new subclass tag).

TODO:

New public beta

Delayed till the majority of the present bugs are fixed. - We will evaluate the situation on Wednesday 19. after lunch, and release a new speller on Friday if smj is good enough (sme already is).

10. Other

Corpus contracts

Delayed till after final release.

TODO:

Bug fixing

When fixing bugs, record the version number containing the fix in the Bugzilla bug report, such that for each bug, we know exactly when it should have been fixed, in what file(s) and what version.

62 open Divvun/Disamb bugs (32 of these 56 are speller-related bugs, 30 are general bugs), and 23 risten.no bugs

Project meeting

We’ll meet in September, 24-28, in Tromsø to work on the hardest remaining issues.

Hotel rooms: Ilona sun-wed, Tomi sun-fri.

TODO:

11. Next meeting, closing

The next meeting is 1.10.2007, 09:30 Norwegian time.

The meeting was closed at 10:49.

Appendix - task lists for the next week

Boerre

Ilona

Maaren

Per-Eric

Saara

Sjur

Thomas

Tomi

Trond