| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
user:zeman:interset:to-do [2009/02/18 09:40]
zeman Web interface. |
user:zeman:interset:to-do [2014/07/25 13:58] (current)
zeman Autoři u cs::ajka. |
| ====== To do ====== | ====== To do ====== |
| | |
| | ===== Interset 2.0 ===== |
| | |
| | * Opravit češtinu (UTF8) v komentáři ABSTRACT pro Pod::Weaver. Jinde je kódování v pořádku, ale tuhle větu Pod::Weaver rozebírá a čte ji špatně. |
| | * A další věc v POD: U CS::Ajka mám dvě sekce AUTHOR. Jednu vloženou ručně, kde je navíc Petr Pořízka a Markus Schäfer, a druhou vloženou automaticky, kde jsem jenom já. Chtělo by to zařídit, aby v tomto konkrétním případě se ta automatická vynechala. |
| | * Nějak lépe vyřešit stažené tvary. Aspoň nový rys, aby to nebylo v other. České příklonné "-s" už má rys ve dvou českých sadách (multext a ajka). Taky mu nastavit 2. osobu, singulár a aux. Podobně je na tom "proň", "zač", "-ť". V němčině "zum", "zur"... Ve francouzštině "du", "aux"... Naopak arabské "al-" sem asi nepatří, to lze prostě chápat jako morfém určitosti, který se nikdy nevyskytuje jako samostatné slovo. |
| | * Projít ''###!!!'', zejména ve FeatureStructure: |
| | * Pokud možno zrušit synpos. Mám spočítáno, že se používá v 21 sadách, ale nevyplývá z toho, zda by se bez něj tyto sady obešly. |
| | * Číslovky více méně předělat po vzoru zájmen, pouze základní číslovky si ponechají slovní druh ''num''. |
| | * Přenést všechny ovladače, které vznikly v Treexu a do SVN Intersetu se zatím nedostaly. |
| | * Oprášit pravidelné kopírování wiki stránek Intersetu do repozitáře SVN (resp. Redmine). Nyní je tam zastaralá kopie. |
| | * Uspořádat seminář a brainstorming pro zájemce o Interset. |
| | * Do té doby prozkoumat Martinovy návrhy a udělat si na ně názor. |
| | * Sepsat manuál pro nové autory ovladačů. |
| | * Jak prezentovat Interset neperlovému světu? Např. tabulky pro decode(). Jde o to, aby lidé mohli Interset snadno použít i v Pythonu, Javě, C a jinde. |
| | * Sem patří i Tagzplorer! |
| |
| ===== Infrastructure ===== | ===== Infrastructure ===== |
| |
| * 14.3.2008: During the last two weeks I made substantial changes to all the drivers due to extensive driver testing and new strict encoding procedures. After all tests pass successfully, the whole thing should be committed to SVN. Besides, it should be re-tested with the parser adaptation project. | |
| * Clean up ''tagset::common'' w.r.t. the various algorithms of enforcing permitted values. | |
| * Query feature value: a shared function detects array and if it is array, searches it for a given value. | * Query feature value: a shared function detects array and if it is array, searches it for a given value. |
| * New test in driver-test.pl: does a driver decode into arrays? If so, what features are affected? If not, is it capable of encoding arrays (i.e. does it call the function that gets rid of arrays)? | * New test in driver-test.pl: does a driver decode into arrays? If so, what features are affected? If not, is it capable of encoding arrays (i.e. does it call the function that gets rid of arrays)? |
| * New test in driver-test.pl: how many tags / what proportion of the tagset are decoded using the ''other'' feature? Enumerate them. Maybe, tags that set the ''other'' feature should not be considered when building the list of possible feature structures? (Sometimes it happens that a feature-value combination is permitted only in an ''other'' tag. Although the set of set/unset features could be used as an additional clue leading to this tag, such tags are often marginal stuff that one does not want to use as default.) | |
| * Extend the ''print_permitted_fs.pl'' tool so that it can search all available drivers. If it is called that way, it prints the name of the driver in front of every feature structure generated by that driver. Print to STDOUT (now it prints to STDERR). Let the user query and filter the feature structure directly: '$_->{mood} eq "ind" && $_->{other} =~ m/^X/'. Grepping the textual output is less powerful. | * Extend the ''print_permitted_fs.pl'' tool so that it can search all available drivers. If it is called that way, it prints the name of the driver in front of every feature structure generated by that driver. Print to STDOUT (now it prints to STDERR). Let the user query and filter the feature structure directly: '$_->{mood} eq "ind" && $_->{other} =~ m/^X/'. Grepping the textual output is less powerful. |
| * Include the text of the GNU GPL in the released package. Create a //License// page at this website. | |
| * Enable checking out the latest development version directly from the Subversion repository. | * Enable checking out the latest development version directly from the Subversion repository. |
| * Create a web interface to DZ Interset. The user will be able to enter a tag from an arbitrary set (the interface will be able to recognize the set or sets) and get a detailed description in English or Czech of the feature values. Features and values could link to Wikipedia explanations. Conversions to other tagsets (including round-trip conversions) will be provided and all tags will link to similar descriptions of their own. Example words will be provided for every tag. The interface will also be able to read whole files in common corpus formats (CoNLL, PML, CSTS, RDT, TEI, Penn, Brill...), highlight word forms / lemmas / tags and link to tag descriptions. Conversion of tags in whole files will be provided. | * Create a web interface to DZ Interset. The user will be able to enter a tag from an arbitrary set (the interface will be able to recognize the set or sets) and get a detailed description in English or Czech of the feature values. Features and values could link to Wikipedia explanations. Conversions to other tagsets (including round-trip conversions) will be provided and all tags will link to similar descriptions of their own. Example words will be provided for every tag. The interface will also be able to read whole files in common corpus formats (CoNLL, PML, CSTS, RDT, TEI, Penn, Brill...), highlight word forms / lemmas / tags and link to tag descriptions. Conversion of tags in whole files will be provided. |
| ===== Features and values ===== | ===== Features and values ===== |
| |
| * Make the form of numbers (digit / roman / word) orthogonal to the cardinal-ordinal distinction, as it is in Multext-East. When tokenization permits, we can have both "5" (cardinal) and "5." (ordinal). | |
| * Normalize processing of pronouns, determiners, interrogative adverbs etc. Old drivers use a different approach from the new ones (beginning with Bulgarian). Pronoun as an independent part of speech will cease to exist. | * Normalize processing of pronouns, determiners, interrogative adverbs etc. Old drivers use a different approach from the new ones (beginning with Bulgarian). Pronoun as an independent part of speech will cease to exist. |
| * Remove ''pos="pron"''. Distribute pronouns to nouns, adjectives and adverbs. When encoding into a tagset that distinguishes pronouns, detect pronouns by non-empty ''prontype''. Remove subposes of pronouns (''pers'', ''clit''...) | * Remove ''pos="pron"''. Distribute pronouns to nouns, adjectives and adverbs. When encoding into a tagset that distinguishes pronouns, detect pronouns by non-empty ''prontype''. Remove subposes of pronouns (''pers'', ''clit''...) |
| * Create overview of common elements that do not have their own part of speech. E.g. infinitive particles. Specifically, make a global classification of "small parts of speech" (pavilón malých šelem :-)), especially particles, adpositions and conjunctions, punctuation and foreign words, hyphenated prefixes and abbreviations. (Pronouns, determiners and numerals are excluded as a separate chapter.) | * Create overview of common elements that do not have their own part of speech. E.g. infinitive particles. Specifically, make a global classification of "small parts of speech" (pavilón malých šelem :-)), especially particles, adpositions and conjunctions, punctuation and foreign words, hyphenated prefixes and abbreviations. (Pronouns, determiners and numerals are excluded as a separate chapter.) |
| * Create an overview of verb forms, moods and tenses in various languages, and their representation in DZ Interset. | * Create an overview of verb forms, moods and tenses in various languages, and their representation in DZ Interset. |
| * Design methodology for contracted word forms where two different parts of speech have merged in one word. Currently this is solved at different places in different ways. Examples: Czech "proň", "bylť", German "zum" and probably other phenomena in other languages. Enclitical pronouns are special case. In some languages they can or must join a verb or other words (Czech "byls", "sis", "tys", "žes"; Spanish "despiértate", "despertarse", "démelo"). Similar problem (but to much greater extent) arises in Arabic where the grammar often requires small words being glued to big ones (e.g. "wa" (conjunction), "al" (article) etc.) | * Design methodology for contracted word forms where two different parts of speech have merged in one word. Currently this is solved at different places in different ways. Examples: Czech "proň", "bylť", German "zum" and probably other phenomena in other languages. Enclitical pronouns are special case. In some languages they can or must join a verb or other words (Czech "byls", "sis", "tys", "žes"; Spanish "despiértate", "despertarse", "démelo"). Similar problem (but to much greater extent) arises in Arabic where the grammar often requires small words being glued to big ones (e.g. "wa" (conjunction), "al" (article) etc.) Warning! I believe that currently the Czech "proň" is tagged differently in cs::pdt and cs::multext. Both set ''subpos = preppron'' but the former under pronoun, the latter under preposition. |
| |
| ==== Things to think about (not sure that these changes should really be made): ==== | ==== Things to think about (not sure that these changes should really be made): ==== |
| * Zrušit ''subpos = voc''. Zatím se používá pro vokalizované tvary českých předložek v cs::pdt (a odvozeném cs::conll; nikde jinde). Místo toho by se ale dalo využít ''variant = long''. U tříd předložek to teď narušuje členění na předložky, záložky a "okololožky" (cirkumpozice). **Problém:** jak vokalizované, tak nevokalizované předložky se také vyskytují s ''variant = 1''. Nemůžu do jednoho rysu nacpat současně ''long'' a ''1'', a nemůžu ani říct, že z ''1'' taky plyne vokalizovanost. | * Zrušit ''subpos = voc''. Zatím se používá pro vokalizované tvary českých předložek v cs::pdt (a odvozeném cs::conll; nikde jinde). Místo toho by se ale dalo využít ''variant = long''. U tříd předložek to teď narušuje členění na předložky, záložky a "okololožky" (cirkumpozice). **Problém:** jak vokalizované, tak nevokalizované předložky se také vyskytují s ''variant = 1''. Nemůžu do jednoho rysu nacpat současně ''long'' a ''1'', a nemůžu ani říct, že z ''1'' taky plyne vokalizovanost. |
| * Define new value //pluralia tantum// (''ptan'') of ''number''? It is present in the Bulgarian CoNLL tagset and it could theoretically be present in other languages, including Czech. | * Define new value //pluralia tantum// (''ptan'') of ''number''? It is present in the Bulgarian CoNLL tagset and it could theoretically be present in other languages, including Czech. |
| | |
| |
| ===== Specific drivers ===== | ===== Specific drivers ===== |
| |
| * cs::pdt - reimplement "type L" pronouns as collective pronouns (introduced due to Bulgarian) | * cs::pdt - reimplement "type L" pronouns as collective pronouns (introduced due to Bulgarian) |
| * cs::pdt - use the new feature ''prepcase'' (introduced due to Portuguese) in distinguishing pronoun forms "jemu" vs. "němu" | |
