Meeting setup

• Date: 14.1.2008
• Time: 09.30 Norw. time
• Place: Internet
• Tools: SubEthaEdit, iChat/Skype

Agenda

Cf. one of the following, depending on context:

• the upper bar of the SEE window (provided you use the JSPWiki syntax mode)
• the TOC in Forrest-rendered output, like HTML and PDF

1. Opening, agenda review, participants

Opened at 09:57.

Present: Børre, Maaren, Per-Eric, Sjur, Thomas, Tomi, Trond

Absent: Risten

Agenda accepted as is.

2. Updated task status since last meeting

Børre

• start to reorganise the documentation
• check who signed the corpus contracts with Kurt Tore, and when
  • It hasn’t arrived at Samediggi. Could be it’s not signed, or that it has been sent to Iđut and is stuck there.
• gather sma texts
  • not done
• improve forrest stability with i18n, site look
  • not done
• set up the Leopard Server features for collaborative support
  • not done
• Hunspell lexicon conversion
  • Done some more testing
• move Steinar’s error markup in the xml files to (a copy of) the original
  • Done, but not moved to the corpus machine.
• InDesign documentation
  • Not done
• investigate the NSIS installer
  • Not done
• fix bugs!

Maaren

• lexicalise actio compounds

Per-Eric

• check some unusual and missing words from the last Olavi missing list
  • Done, but still words to analyse
• Corpus gathering, keep the contact with Kurt Tores family about his texts
  • Nothing this week
• Corpus gathering, try to call Sigga Tuolja Sandstrøm again, about her texts
  • Not done
• fix bugs!
  • Fixed 495, false positive and false negative

Saara

• add new XSL/XML headers for proofing test docs
• Set up ways of adding meta-information for proofing correct corpus docs (source info, used in testing or not, added to lexicon or not)
• add nested error markup to xml conversion
• discuss more parallel texts

Sjur

• document Windows CD installation work-around
  • not done
• start to reorganise the documentation
  • not done
• gather sma texts
  • not done
• improve forrest stability with i18n, site look
  • not done
• set up the Leopard Server features for collaborative support
  • not done
• check the present sma sources
  • not done
• name db/risten.no
  • not done
• improve hyphenation testing
  • not done
• investigate the NSIS installer
  • not done
• get the date straight for the Snåsa meeting
  • done, although there is an amount of uncertainty still
• get hotel rooms in Snåsa
  • not done
• make a first sma project plan
  • not done
• publish corpus contracts and project infra as open-source on NoDaLi-sta
  • not done
• fix bugs!
• other:
  • wrote article about minority language technology with Trond
  • some internal speller releases

Thomas

• sme->smj lexicon conversion to build bilingual lexicon resources
  • not anything this week
• look at test cases still not behaving properly
  • worked some
• fix bugs!
  • worked some

Tomi

• Hunspell lexicon conversion
  • not done
• fix bugs!
  • fixed

Trond

• sme->smj lexicon conversion to build bilingual lexicon resources
  • Worked a lot on this, reconverted all the POS again, cut down the set of inclusion candidates, rearranged the remaining ones.
• Reorganise documentation (with Børre and Sjur)
  • Not done
• Gather sma texts (with Børre and Sjur)
  • Not done
• Look at the sma source files (with Sjur)
  • I have set up a test suite for our verb twol problem, but have not discussed it with Sjur yet. It seems the verb problem is actually a cluster of different problems. Thus, not one big mystery, but rather a set of problems, each of them in itself known (one might hope).
• Name lexicon project: Test editing xml files (when they are ready for it)
  • Not done.
• Make a first sma project plan
  • Not done.
• fix bugs!.

3. Documentation

Our documentation needs a thorough clean-up and reorganisation to make it clearer and easier to use.

TODO:

• start to reorganise the documentation (Børre, Sjur, Trond)

4. Corpus gathering

Nothing has happened last week.

We need to start gathering sma texts right away. Some sources of sma texts:

• Nord-Trøndelag fylke
• Snåsa kommune
• Anna Jacobsen Don jih dan bijre I, II. This should be scanned.
• Lohkede Saemien, needs to be scanned
• other authors
• SD texts, specifically on Pia’s SD computer, teaching text books
• Bible
• Year book
• Sámi school books
• sma Davvin book(s?)
• some univ. texts: theses, student papers, teaching texts, Ove Lorentz does have a large number of web texts that can be used for vocabulary coverage checking.
• «Reindriftsbladet» and «Š» do publish some texts in sma
• Infonuorra, Samenet

TODO:

• follow-up on the smj texts from Kurt Tore (Per-Eric)
• check who signed the corpus contracts with Kurt Tore, and when (Børre)
• gather sma texts (Børre, Sjur, Trond)
• Put the list of possible corpus sources into a document gt/doc/lang/sma/sma-corpus-plan.jspwiki (Maaren)

5. Infrastructure

Forrest needs better handling of i18n, to help us get a more stable site. We should probably also look at a visual make-over of our site as well at the same time.

We now also have the time to start to explore the new collaboration features of the Leopard server: shared project calendars, possibly a wiki, permanent chat rooms for dedicated topics.

TODO:

• add Jabber account in iChat (all)
• improve forrest stability with i18n, site look (Børre, Sjur, Tomi)
• set up the Leopard Server features for collaborative support - permanent chat rooms, project calendar(s), wiki? (Børre, Sjur)

6. Linguistics

North Sámi

Hyphenation bugs still there, needs improved test bench.

Lule Sámi

Hyphenation: same as for sme.

TODO:

• sme->smj lexicon conversion to build bilingual lexicon resources, and increase smj coverage (Trond, Thomas, Svenne). Add the words when all words are ready.

South Sámi

Trond and Sjur needs to have a thourough look at the sma sources, and evaluate the present approach to its morphophonology.

TODO:

• check the present sources (Sjur, Trond)

7. Name lexicon infrastructure

Decisions made in Tromsø can be found in [this meeting memo.|/admin/physical_meetings/tromso-2006-08-propnoun.html]

TODO:

1. fix bugs in lexc2xml; add comments to the log element (Saara)
2. finish first version of the editing (Sjur)
3. test editing of the xml files. If ok, then: (Sjur, Thomas, Trond)
4. make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (Sjur, Saara)
5. convert propernoun-($lang)-lex.txt to a derived file from common xml files (Sjur, Tomi, Saara)
6. implement data synchronisation between risten.no and the cvs repo, and possibly other servers (ie the G5 as an alternative server to the public risten.no - it might be faster and better suited than the official one; also local installations could be treated the same way)
7. start to use the xml file as source file
8. clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (Thomas, Maaren, linguists)
9. merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (linguists)
10. publish the name lexicon on risten.no (Sjur)
11. add missing parallel names for placenames (linguists)
12. add informative links between first names like Niillas and Nils (linguists)

8. Proofing tools

Hunspell

Proper nouns not yet working, and they do not contain anything to clearly identify the end of the stem. This makes it harder to generate proper HUNSPELL entries.

It should be quite easy to introduce suffix/affix border markers in proper nouns

• it can be done in the -fem, -mal, etc. lexicons, before continuing to the proper inflection lexicons. This way we get the suffix marker where it should be, and the work for adding it is very limited. Please note that it requires changes to the twol rules, we need to test carefully.

TODO:

• Hunspell lexicon conversion (Tomi, Børre)
• add ‰>:‰> before every +Fem etc. tag (Thomas)
• add %>:0 alphabet entry in twolc, with a %> variant for HUNSPELL (Thomas)

Testing

Spelling Error Markup

This will wait till after the release.

TODO:

• Set up ways of adding meta-information (source info, used in testing or not, added to lexicon or not) (Saara)
• move Steinar’s error markup in the xml files to (a copy of) the original (Børre, Kimme)
  • done
• move Steinar’s correct files to the corpus repository (Børre)
• add nested error markup to xml conversion (Saara)
  • done
• test new and nested error markup (Sjur)

Automated testing

Paradigm testing is now fixed, and is working.

BUT: paradigms are not generated for smj verbs, in gt/smj/testing. Nouns, adjectives, propernouns, numbers all work fine.

TODO:

• improve hyphenation testing (Sjur)
• add verb paradigm generation bug to Bugzilla (Sjur)

Lexicon conversion to the PLX format

Open issues based on test results:

sme

• 425 - roman number - will not be fixed in 1.0 release
• 426 - REGRESSION: comp words from Divvun.no - guoktedássásaš accepted
  • numerals are now compounded, as in guokteduhát
• 536 - REGRESSION: speller accepts “impossible” compound-forms, geažideapmigárvu and giddesteapmisággi accepted again
• 542 - clitic -ge - FIXED
• 588 - regression - r. accepted as final part - FIXED
• 593 - missing words in beta2, still missing Nuppelohkái - not lexicalized
• 595 - prefix+name wihtout hyphen (ovdaLot instead of ovda-Lot)
• 597 - does not recognize nubbelohki - not lexicalized
• 599 - short attr -lot/-låk numerals (ok in smX.fst) - FIXED
• 603 - suomabealdi, norggabealdi accepted
• 604 - action as second compound part - FIXED
• 606 - speller accepts VUOHTA compound
• 609 - Anár-julggaštusa not recognized - FIXED, but still need to add a test case for the positive test
• 610 - missing duhát words - FIXED
• 611 - double hyphen sugg (missing test case)
• 613 - short gen. as second compound part
• 619 - numeral derivations - FIXED
• 622 - Words with clitic -ban not accepted - diehtiban not yet accepted
  • diehti not a word
• 625 - word+footnote - possibly Polderland or MS, or a consequence of allowing spell checking of words including digits

smj

20080114 version is missing proper nouns completely due to ssh error while transfering - Áilegas ?? weren’t those cases from sme -> smj - ok Ok, but the 575 bug was present also in the previous version Yes, but I want to discuss that data, it is disturbing to have test cases that constantly fails (or should fail). They test whether we get double hyphens in the suggestions. We should replace the test cases with the double hyphen strings we used to get.

• 575 - name+name compounds, three test cases not accepted
• 599 - numeral attr:s on lot, golmmalåk still not accepted - it is wrong and correctly sugg.
• 607 - acro + hyphen - FIXED
• 615 - actio and actor compounds - bargogáddjon still missing
• 617 - propnoun compounding - FIXED
• 618 - dipht. simpl.
• 619 - num. derivation - FIXED
• 624 - gen of ÅLLAGASJ-adj as first prt of cmp

TODO:

• look at test cases still not behaving properly (Thomas, Tomi)
• document how compounding is controlled in the PLX conversion (Tomi)

InDesign tools

The speller was delivered last week, but without documentation. Sjur could not get it to work. Documentation should be coming today.

We also need to communicate better (or at all, that is) with Davvi Girji and Min Áigi.

TODO:

• improve hyphenation testing (Sjur)
• contact DG and MÁ about proofing tools for InDesign (Sjur)

Hyphenators

Still issues to investigate.

Windows installer

New Windows installer NSIS

Benefits of using this:

• open source
• free / no cost
• can make installer packages on Mac/Linux
• truly multilingual
• we have full control over the installer sources

Drawbacks:

• untested
• extra work on our part

TODO:

• investigate the NSIS installer (Børre, Sjur)

Releases

We need to bring the Changes document up-to-date, and prepare for a 1.0.2 release later this month.

TODO:

• update the Changes document (Maaren)

9. Other

South Sámi project startup meeting

• in Snåsa
• end of January (from 28th of Jan - 1st of Feb)’ish
• Participants: SD (incl. Divvun), Nord-Trøndelag fylkeskommune, Snåsa kommune, UiT, “resource persons”

We extend the meeting on our part, to have this project’s first gathering.

TODO:

• get the date straight (Sjur)
• get hotel rooms (Sjur)
• make a first sma project plan (Sjur, Trond)

Corpus contracts + open source

TODO:

• publish corpus contracts and project infra as open-source on NoDaLi-sta (Sjur)

Bug fixing

When fixing bugs, record the version number containing the fix in the Bugzilla bug report, such that for each bug, we know exactly when it should have been fixed, in what file(s) and what version.

83 open Divvun/Disamb bugs (45 of these 83 are speller-related bugs, 38 are other bugs), and 23 risten.no bugs

10. Next meeting, closing

The next meeting is 21.1.2008, 09:30 Norwegian time.

The meeting was closed at 10:53.

Appendix - task lists for the next week

Boerre

• start to reorganise the documentation
• check who signed the corpus contracts with Kurt Tore, and when
• gather sma texts
• improve forrest stability with i18n, site look
• set up the Leopard Server features for collaborative support
• Hunspell lexicon conversion
• move Steinar’s correct files to the corpus repository
• InDesign documentation
• investigate the NSIS installer
• fix bugs!

Maaren

• Put the list of possible sma corpus sources into a document
• update the Changes document

Per-Eric

• check some unusual and missing words from the last Olavi missing list
• Corpus gathering, keep the contact with Kurt Tores family about his texts
• Corpus gathering, try to call Sigga Tuolja Sandstrøm again, about her texts
• fix bugs!

Saara

• add new XSL/XML headers for proofing test docs
• Set up ways of adding meta-information for proofing correct corpus docs (source info, used in testing or not, added to lexicon or not)
• discuss more parallel texts

Sjur

• document Windows CD installation work-around
• start to reorganise the documentation
• gather sma texts
• improve forrest stability with i18n, site look
• set up the Leopard Server features for collaborative support
• check the present sma sources
• name db/risten.no
• improve hyphenation testing
• investigate the NSIS installer
• get hotel rooms in Snåsa
• make a first sma project plan
• publish corpus contracts and project infra as open-source on NoDaLi-sta
• add verb paradigm generation bug to Bugzilla
• contact DG and MÁ about proofing tools for InDesign
• fix bugs!

Thomas

• sme->smj lexicon conversion to build bilingual lexicon resources
• look at test cases still not behaving properly
• add %>:0 alphabet entry in twolc, with a %> variant for HUNSPELL
• fix bugs!

Tomi

• Hunspell lexicon conversion
• document how compounding is controlled in the PLX conversion
• fix bugs!

Trond

• sme->smj lexicon conversion to build bilingual lexicon resources
• Reorganise documentation (with Børre and Sjur)
• Gather sma texts (with Børre and Sjur)
• Look at the sma source files (with Sjur)
• Name lexicon project: Test editing xml files (when they are ready for it)
• Make a first sma project plan
• fix bugs!.