[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Last revision Both sides next revision
external:spr [2011/10/20 15:29]
smejkalova vytvořeno
external:spr [2011/10/21 15:15]
smejkalova
Line 1: Line 1:
-====== Semantic Pattern Recognition ====== +====== Semantic Pattern Recognition ​(SPR)  --  project webpage ​====== 
-Doplnit strucny popis.+ 
 + 
 + 
 +===== People and contacts ===== 
 + 
 +    * Silvie Cinková <cinkova (at) ufal.mff.cuni.cz>​ 
 +    * Martin Holub <holub (at) ufal.mff.cuni.cz>​ 
 +    * Lenka Smejkalová <​smejkalova (at) ufal.mff.cuni.cz>​ 
 +    * Vincent Kríž <​vincent.kriz (at) gmail.com>​ 
 + 
 +Institute of Formal and Applied Linguistics 
 +Charles University in Prague 
 +Faculty of Mathematics and Physics 
 + 
 +Malostranské náměstí 25 
 +CZ-118 00 Praha 
 + 
 + 
 + 
 + 
 + 
 +===== CPA Verb Validation Sample 30 (En) ===== 
 + 
 +"CPA Verb Validation Sample 30 (En)" is a newly developed lexical resource. It contains descriptions of the following 30 English verbs:  
 + 
 +|//access // |//​ally// ​  ​|//​analyse//​ |//​arrive// ​  ​|//​breathe//​ | 
 +|//​claim// ​  ​|//​cool// ​  ​|//​cry// ​    ​|//​crush// ​   |//​deny// ​   | 
 +|//​enlarge//​ |//enlist// |//​forge// ​  ​|//​frighten//​ |//​furnish//​ | 
 +|//​hail// ​   |//​halt// ​  ​|//​part// ​   |//​plug// ​    ​|//​plough// ​ | 
 +|//​pour// ​   |//​smash// ​ |//​smell// ​  ​|//​steer// ​   |//​submit// ​ | 
 +|//​swell// ​  ​|//​throw// ​ |//​trouble//​ |//​wake// ​    ​|//​yield// ​  | 
 + 
 + 
 + 
 +Here we present a small portion of the data just to illustrate its structure. For each verb we define a set of semantic patterns and provide a  sample of manually annotated corpus concordances. Then we do a detailed interannotator disagreement analysis, error detection, and adjudication. 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Annotation Scheme Description ==== 
 +     * Pattern Definition Form - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​PDEV2.1-pattern-form.pdf|pdf]] 
 +     * Annotation Manual - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​annotation_manual.pdf|pdf]] 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Pattern Definitions Preview ==== 
 +A few examples of revised PDEV entries: 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_patterns.html|cool]] 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_patterns.html|deny]] 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_patterns.html|yield]] 
 + 
 +An example of a pattern definition form in detail: 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_7.png|deny - pattern number 7]] 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Annotated Data Preview ==== 
 +Each of the examples below contains a multiple annotated set of 50 corpus concordances with manual disagreement analysis and final adjudication. The adjudicated data are used as a gold standard sample. 
 +    * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_analysis.pdf|pdf]] 
 +    * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_analysis.pdf|pdf]] 
 +    * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_analysis.pdf|pdf]] 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Disagreement Analysis Preview ==== 
 +Confusion matrices for each pair of annotators and automatic disagreements analysis: 
 +    * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_res.txt|txt]] 
 +    * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_res.txt|txt]] 
 +    * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_res.txt|txt]]
  
  
  
  
-====== What we have already done ====== 
  
-* Manual for Annotators - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​annotation_manual.pdf|pdf]] 
-* Specification of form for defining patterns - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​PDEV2.1-pattern-form.pdf|pdf]] 
-* Pattern definitions - we have revised the pattern definitions for 30 verbs. Here is the sample of three of them (after revision): 
-    * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_patterns.html|cool]] 
-    * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_patterns.html|deny]] - detailed [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_7.png|view]] of pattern number 7 
-    * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_patterns.html|yield]] 
-* Annotation of 50 concordances per each of these verbs by three annotators. Here is a little sample of three verbs. There has been already finished the manual disagreements analysis and all instances were adjudicated and gold sample was created. 
-    * Manual disagreements analysis and adjudication:​ 
-          * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_analysis.pdf|pdf]] 
-          * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_analysis.pdf|pdf]] 
-          * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_analysis.pdf|pdf]] 
-    * Automatic disagreements analysis - confusion matrix for each pair of the annotators in one file 
-          * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_res.txt|txt]] 
-          * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_res.txt|txt]] 
-          * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_res.txt|txt]] 
-    * Inter-annotator agreement (Cohen'​s kappa for each pair, Fleiss'​ kappa for all together 
  
-Results of inter-annotator agreement 
-^verb     ​^ ​ size  ^ #N ^ Fleiss'​ kappa ^        Cohen'​s kappa            ^^^ 
-|         ​| ​       |    |               | A2 vs. A3 | A2 vs. A1 | A3 vs. A1 | 
-|cool   |      50|  16|    ​0.685| ​     0.743| ​     0.669| ​     0.646| 
-|deny   |      50|  10|          0.524| ​     0.434| ​     0.571| ​     0.582| 
-|yield ​   |      50|  10|    ​0.500| ​     0.489| ​     0.588| ​     0.429| 
  

[ Back to the navigation ] [ Back to the content ]