[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
external:spr [2011/10/20 15:32]
smejkalova
external:spr [2011/10/21 15:15]
smejkalova
Line 1: Line 1:
-====== Semantic Pattern Recognition ====== +====== Semantic Pattern Recognition ​(SPR)  --  project webpage ​====== 
-Doplnit strucny popis.+ 
 + 
 + 
 +===== People and contacts ===== 
 + 
 +    * Silvie Cinková <cinkova (at) ufal.mff.cuni.cz>​ 
 +    * Martin Holub <holub (at) ufal.mff.cuni.cz>​ 
 +    * Lenka Smejkalová <​smejkalova (at) ufal.mff.cuni.cz>​ 
 +    * Vincent Kríž <​vincent.kriz (at) gmail.com>​ 
 + 
 +Institute of Formal and Applied Linguistics 
 +Charles University in Prague 
 +Faculty of Mathematics and Physics 
 + 
 +Malostranské náměstí 25 
 +CZ-118 00 Praha 
 + 
 + 
 + 
 + 
 + 
 +===== CPA Verb Validation Sample 30 (En) ===== 
 + 
 +"CPA Verb Validation Sample 30 (En)" is a newly developed lexical resource. It contains descriptions of the following 30 English verbs:  
 + 
 +|//access // |//​ally// ​  ​|//​analyse//​ |//​arrive// ​  ​|//​breathe//​ | 
 +|//​claim// ​  ​|//​cool// ​  ​|//​cry// ​    ​|//​crush// ​   |//​deny// ​   | 
 +|//​enlarge//​ |//enlist// |//​forge// ​  ​|//​frighten//​ |//​furnish//​ | 
 +|//​hail// ​   |//​halt// ​  ​|//​part// ​   |//​plug// ​    ​|//​plough// ​ | 
 +|//​pour// ​   |//​smash// ​ |//​smell// ​  ​|//​steer// ​   |//​submit// ​ | 
 +|//​swell// ​  ​|//​throw// ​ |//​trouble//​ |//​wake// ​    ​|//​yield// ​  | 
 + 
 + 
 + 
 +Here we present a small portion of the data just to illustrate its structure. For each verb we define a set of semantic patterns and provide a  sample of manually annotated corpus concordances. Then we do a detailed interannotator disagreement analysis, error detection, and adjudication. 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Annotation Scheme Description ==== 
 +     * Pattern Definition Form - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​PDEV2.1-pattern-form.pdf|pdf]] 
 +     * Annotation Manual - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​annotation_manual.pdf|pdf]] 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Pattern Definitions Preview ==== 
 +A few examples of revised PDEV entries: 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_patterns.html|cool]] 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_patterns.html|deny]] 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_patterns.html|yield]] 
 + 
 +An example of a pattern definition form in detail: 
 +     * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_7.png|deny - pattern number 7]] 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Annotated Data Preview ==== 
 +Each of the examples below contains a multiple annotated set of 50 corpus concordances with manual disagreement analysis and final adjudication. The adjudicated data are used as a gold standard sample. 
 +    * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_analysis.pdf|pdf]] 
 +    * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_analysis.pdf|pdf]] 
 +    * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_analysis.pdf|pdf]] 
 + 
 + 
 + 
 + 
 + 
 + 
 +==== Disagreement Analysis Preview ==== 
 +Confusion matrices for each pair of annotators and automatic disagreements analysis: 
 +    * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_res.txt|txt]] 
 +    * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_res.txt|txt]] 
 +    * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_res.txt|txt]]
  
  
  
  
-====== Data ====== 
  
-     * Manual for Annotators - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​annotation_manual.pdf|pdf]] 
-     * Specification of form for defining patterns - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​PDEV2.1-pattern-form.pdf|pdf]] 
-     * Pattern definitions - we have revised the pattern definitions for 30 verbs. Here is the sample of three of them (after revision): 
-          * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_patterns.html|cool]] 
-          * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_patterns.html|deny]] - detailed [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_7.png|view]] of pattern number 7 
-          * [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_patterns.html|yield]] 
-     * Annotation of 50 concordances per each of these verbs by three annotators. Here is a little sample of three verbs. There has been already finished the manual disagreements analysis and all instances were adjudicated and gold sample was created. 
-          * Manual disagreements analysis and adjudication:​ 
-                * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_analysis.pdf|pdf]] 
-                * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_analysis.pdf|pdf]] 
-                * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_analysis.pdf|pdf]] 
-          * Automatic disagreements analysis - confusion matrix for each pair of the annotators in one file 
-                * cool - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​cool_res.txt|txt]] 
-                * deny - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​deny_res.txt|txt]] 
-                * yield - [[http://​ufal.mff.cuni.cz/​~smejkalova/​pdev/​yield_res.txt|txt]] 
-          * Inter-annotator agreement (Cohen'​s kappa for each pair, Fleiss'​ kappa for all together 
  
-Results of inter-annotator agreement 
-^verb     ​^ ​ size  ^ #N ^ Fleiss'​ kappa ^        Cohen'​s kappa            ^^^ 
-|         ​| ​       |    |               | A2 vs. A3 | A2 vs. A1 | A3 vs. A1 | 
-|cool   |      50|  16|    ​0.685| ​     0.743| ​     0.669| ​     0.646| 
-|deny   |      50|  10|          0.524| ​     0.434| ​     0.571| ​     0.582| 
-|yield ​   |      50|  10|    ​0.500| ​     0.489| ​     0.588| ​     0.429| 
  

[ Back to the navigation ] [ Back to the content ]