[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
external:spr [2011/10/21 14:11]
smejkalova
external:spr [2012/01/19 10:47]
smejkalova
Line 5: Line 5:
 ===== People and contacts ===== ===== People and contacts =====
  
-    * Silvie Cinková cinkova@ufal.mff.cuni.cz +    * Silvie Cinková <cinkova (at) ufal.mff.cuni.cz> 
-    * Martin Holub holub@ufal.mff.cuni.cz +    * Martin Holub <holub (at) ufal.mff.cuni.cz> 
-    * Lenka Smejkalová smejkalova@ufal.mff.cuni.cz +    * Lenka Smejkalová <smejkalova (at) ufal.mff.cuni.cz> 
-    * Vincent Kríž vincent.kriz@gmail.com+    * Vincent Kríž <vincent.kriz (at) gmail.com>
  
 Institute of Formal and Applied Linguistics Institute of Formal and Applied Linguistics
Line 16: Line 16:
 Malostranské náměstí 25 Malostranské náměstí 25
 CZ-118 00 Praha CZ-118 00 Praha
 +
 +
 +
  
  
Line 21: Line 24:
  
 "CPA Verb Validation Sample 30 (En)" is a newly developed lexical resource. It contains descriptions of the following 30 English verbs:  "CPA Verb Validation Sample 30 (En)" is a newly developed lexical resource. It contains descriptions of the following 30 English verbs: 
 +
 +|//access // |//ally//   |//arrive//  |//breathe//  |//claim//   |
 +|//cool//    |//cry//    |//crush//   |//deny//     |//enlarge// |
 +|//enlist//  |//forge//  |//furnish// |//hail//     |//halt//    |
 +|//part//    |//plug//   |//plough//  |//pour//     |//say//     |
 +|//smash//   |//smell//  |//steer//   |//submit//   |//swell//   |
 +|//tell//    | //throw// |//trouble// |//wake//     |//yield//   |
 +
 +
 +
 +Here we present a small portion of the data just to illustrate its structure. For each verb we define a set of semantic patterns and provide a  sample of manually annotated corpus concordances. Then we do a detailed interannotator disagreement analysis, error detection, and adjudication.
  
  
-|//access // |//ally// |//analyse// |//arrive// |//breathe// | 
-|//claim//  |//cool// |//cry//     |//crush//  |//deny//    | 
-|//enlarge// |//enlist// |forge |frighten |furnish | 
-|hail |halt |part |plug |plough | 
-|pour |smash |smell |steer |submit | 
-|swell | throw |trouble |wake |yield | 
  
  
-Here we present a small portion of the data just to show its structure. 
  
  
 ==== Annotation Scheme Description ==== ==== Annotation Scheme Description ====
-     Specification of form for defining patterns - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/PDEV2.1-pattern-form.pdf|pdf]]+     Pattern Definition Form - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/PDEV2.1-pattern-form.pdf|pdf]]
      * Annotation Manual - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/annotation_manual.pdf|pdf]]      * Annotation Manual - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/annotation_manual.pdf|pdf]]
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +==== Pattern Definitions Preview ====
 +A few examples of revised PDEV entries:
 +     * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_patterns.html|cool]]
 +     * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_patterns.html|deny]]
 +     * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_patterns.html|yield]]
 +
 +An example of a pattern definition form in detail:
 +     * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_7.png|deny - pattern number 7]]
 +
 +
 +
 +
 +
 +
  
  
 ==== Annotated Data Preview ==== ==== Annotated Data Preview ====
 +Each of the examples below contains a multiple annotated set of 50 corpus concordances with manual disagreement analysis and final adjudication. The adjudicated data are used as a gold standard sample.
 +    * cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_analysis.pdf|pdf]]
 +    * deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_analysis.pdf|pdf]]
 +    * yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_analysis.pdf|pdf]]
  
  
-==== Disagreement Analysis Preview ==== 
  
  
-     * Pattern definitions - we have revised the pattern definitions for 30 verbs. Here is the sample of three of them (after revision): 
-          * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_patterns.html|cool]] 
-          * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_patterns.html|deny]] - detailed [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_7.png|view]] of pattern number 7 
-          * [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_patterns.html|yield]] 
-     * Annotation of 50 concordances per each of these verbs by three annotators. Here is a little sample of three verbs. There has been already finished the manual disagreements analysis and all instances were adjudicated and gold sample was created. 
-          * Manual disagreements analysis and adjudication: 
-                * cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_analysis.pdf|pdf]] 
-                * deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_analysis.pdf|pdf]] 
-                * yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_analysis.pdf|pdf]] 
-          * Automatic disagreements analysis - confusion matrix for each pair of the annotators in one file 
-                * cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_res.txt|txt]] 
-                * deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_res.txt|txt]] 
-                * yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_res.txt|txt]] 
-          * Inter-annotator agreement (Cohen's kappa for each pair, Fleiss' kappa for all together 
  
-Results of inter-annotator agreement + 
-^verb      size  ^ #N ^ Fleiss' kappa ^        Cohen's kappa            ^^^ +==== Disagreement Analysis Preview ==== 
-|                |    |               | A2 vs. A3 | A2 vs. A1 | A3 vs. A1 | +Confusion matrices for each pair of annotators and automatic disagreements analysis: 
-|cool    |      50|  16|    0.685|      0.743|      0.669|      0.646+    cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_res.txt|txt]] 
-|deny    |      50|  10|          0.524|      0.434|      0.571|      0.582+    deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_res.txt|txt]] 
-|yield         50|  10|    0.500|      0.489|      0.588|      0.429|+    yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_res.txt|txt]] 
 + 
  
  
  
  
-TODO: ne odlisna barva pozadi a rozsirit chlivky a kurziva (v tabulce sloves) + vycentrovat na strance 
-TODO: zkontrolovat obsah souboru (specifikace formulare) 
  

[ Back to the navigation ] [ Back to the content ]