Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
user:hladka:playcoref [2009/02/25 22:30] hladka |
user:hladka:playcoref [2009/03/02 08:37] mirovsky |
||
---|---|---|---|
Line 21: | Line 21: | ||
* [[http:// | * [[http:// | ||
* Projekt anotace rozšířené textové koreference a bridging vztahů v PDT. (Anja Nedolužko: [[http:// | * Projekt anotace rozšířené textové koreference a bridging vztahů v PDT. (Anja Nedolužko: [[http:// | ||
+ | |||
Line 30: | Line 31: | ||
* Experiments with Czech so far | * Experiments with Czech so far | ||
- Nguy Giang Linh: Návrh souboru pravidel pro analýzu anafor v českém jazyce (A set of rules for anaphora resolution in Czech), MFF UK 2006. **Available: | - Nguy Giang Linh: Návrh souboru pravidel pro analýzu anafor v českém jazyce (A set of rules for anaphora resolution in Czech), MFF UK 2006. **Available: | ||
- | - Nguy Giang Linh; Žabokrtský, | + | - Nguy Giang Linh; Žabokrtský, |
* Linh's procedure | * Linh's procedure | ||
Line 76: | Line 77: | ||
====== Specification ====== | ====== Specification ====== | ||
- | |||
- | |||
- | |||
- | |||
- | |||
- | |||
- | |||
Line 92: | Line 86: | ||
* A game of two players. Players are paired randomly. Computer as a player: automatic coreference resolution **???????** | * A game of two players. Players are paired randomly. Computer as a player: automatic coreference resolution **???????** | ||
* Session time up to **???????** minutes. | * Session time up to **???????** minutes. | ||
- | * At the beginning, | + | * At the beginning |
- | * What my partner is doing? If (s)he hooks up the same pair of words as hooked up then the pair of words starts **??????? | + | * What my partner is doing? If (s)he hooks up the same pair of words as I hooked up then the pair of words starts **??????? |
* The players can re-hook up any word any time in the session. | * The players can re-hook up any word any time in the session. | ||
- | * To design the game for a particular language the following data and tools are needed (or are welcome): | + | * To design the game for a particular language the following data and tools are needed (or, better said, are welcome): |
- corpus of manually anotated coreference | - corpus of manually anotated coreference | ||
- POS tagger | - POS tagger | ||
- coreference resolution procedure | - coreference resolution procedure | ||
- | |||
- | |||
Line 112: | Line 104: | ||
=== Text Selection === | === Text Selection === | ||
- | * CS data ^JM^ | + | * CS data |
* Anja's data ## // PDT data that are currently being annotated for the extended coreference // | * Anja's data ## // PDT data that are currently being annotated for the extended coreference // | ||
- | * more 'user-friendly' | + | |
+ | * **---JM TO DO---** na datech od Anji zjistit pro nas zajimave statistiky typu | ||
+ | vety/ | ||
* **EN** | * **EN** | ||
* search the data that are available | * search the data that are available | ||
Line 130: | Line 125: | ||
* sentence by sentence | * sentence by sentence | ||
* supervised selection of documents for a session | * supervised selection of documents for a session | ||
- | |||
- | |||
===== Scoring ===== | ===== Scoring ===== | ||
- | * '' | + | * '' |
- | // w1 by mela byt nejvyssi; w2 by mela urcite nejak zohlednit uspesnost automaticke procedury - uspesnost merenou na jakych datech?; w3: kdyz hracum budeme zobrazovat i ta slova, ktera oznacil protihrac, a ja je neoznacila, nebudeme je tim tlacit | + | **JM**: |
+ | Já myslím, že do shody je tlačit chceme. Je žádoucí, aby anotace byla co nejúplnější. Když druhý hráč uvidí, že první hráč spojil nějaké slovo, vyvíjí | ||
+ | nepřehlédl a jestli by ho nemohl zapojit také. Neukazuje se mu kam, takže když nenajde žádný cíl, nezapojí ho a bude se radovat, že první hráč udělal nějakou chybu. Myslím, že ta funkce by měla brát **buď** automatickou anotaci **nebo** manuální, podle toho, co je k dispozici. Rovněž si teď myslím, že manuálně anotovaná data budeme používat minimálně - pouze pro změření úspěšnosti anotace pomocí hry - to ale nemusí být vůbec součástí skóre hry, to se udělá off-line. Manuálně anotovaných dat máme málo, jsou už anotovaná a nejsou zábavná. Z toho mi vyplývá, že bych manuální anotaci pro určování skóre nebral vůbec v úvahu a ze vzorečku nahoře bych první člen vyhodil. | ||
+ | **BH**: Jirka ma pravdu. Pocitani skore musi byt objektivni. Proto jsem vzorecek upravila tak, ze nebude pocitat shodu hrace vzhledem k rucni anotaci. | ||
===== Output Data Needed ===== | ===== Output Data Needed ===== | ||
* score list ## // | * score list ## // | ||
- | * documents after the '' | + | * documents after the '' |
+ | **JM**: Mluvil jsem kvůli měření mezianotátorské shody v anotování koreference se Zdeňkem a vyšlo z toho, že na měření shody na šipkách by použil prostě jen F-measure. Kappa je nevhodná kvůli tomu, že pravděpodobnost náhodné shody je poměrně nízká a těžko se určuje; kappa se hodí spíš pro klasifikační úlohy (proto ji použiju v Anjiině projektu na shodu v určování typu koreference, | ||
+ | - kappa measure | ||
+ | - G-theory - see [[http:// | ||
+ | Identifying Sources of Disagreement: | ||
+ | - the Pearson correlation - see (Snow et al., 2008) [[http:// | ||
* session | * session | ||
* player_A_id, | * player_A_id, | ||
* document(s) | * document(s) | ||
- | * number of corrections by player_A and by player_B | + | * number of corrections by player_A and by player_B |
- | * corrections by player_A and by player_B | + | * corrections by player_A and by player_B |
===== Design ===== | ===== Design ===== | ||
Line 156: | Line 155: | ||
* session time = elapsed time + remaining time | * session time = elapsed time + remaining time | ||
* how many sentences my partner has read so far | * how many sentences my partner has read so far | ||
- | * running pts **???????** | + | * running pts **??????? |
+ | * Format of the text | ||
+ | * **JM**: nouns and pronouns might be displayed slightly differently so that the user avoids other parts of speech easily; he should not be allowed to use other parts of speech at all | ||
* Visualization of the coreference pairs | * Visualization of the coreference pairs | ||
* colors | * colors | ||
- | * arrows | + | * arrows |
* ... | * ... | ||
+ | ===== Tools needed ===== | ||
+ | * tagger ## tool_chain (CAC2.0) | ||
+ | * Linh's coreference resolution procedure **---PS TO DO---** What type of input data the Linh's procedure works with? '' | ||
+ | * conversion: csts <-> pml m_coref scheme | ||
- | |||
- | ===== Tools needed ===== | ||
- | * tagger ^BH^ ## tool_chain (CAC2.0) | ||
- | * Linh's coreference resolution procedure ^PS^ ## What type of input data the Linh's procedure works with? '' | ||
- | * conversion: csts <-> pml m_coref scheme | ||
+ | ====== ACL - IJCNLP2009 ====== | ||
+ | * [[http:// | ||
+ | * [[http:// | ||
+ | * pracovni adresar ''/ |