[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

• ÚFAL Wiki Home
• ÚFAL Home
• Sitemap
• RSS
• ÚFAL calendar
• Recent changes
• Login
• Hledat:

[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

View differences: Side by SideInline Go

Link to this comparison view

Both sides previous revision Previous revision 2012/01/19 10:47 smejkalova 2011/10/21 15:15 smejkalova 2011/10/21 15:14 smejkalova 2011/10/21 15:09 smejkalova 2011/10/21 14:32 smejkalova 2011/10/21 14:11 smejkalova 2011/10/21 14:09 smejkalova 2011/10/21 13:41 smejkalova 2011/10/20 15:32 smejkalova 2011/10/20 15:30 smejkalova 2011/10/20 15:29 smejkalova 2011/10/20 15:29 smejkalova vytvořeno Go		2012/01/19 10:47 smejkalova 2011/10/21 15:15 smejkalova 2011/10/21 15:14 smejkalova 2011/10/21 15:09 smejkalova 2011/10/21 14:32 smejkalova 2011/10/21 14:11 smejkalova 2011/10/21 14:09 smejkalova 2011/10/21 13:41 smejkalova 2011/10/20 15:32 smejkalova 2011/10/20 15:30 smejkalova 2011/10/20 15:29 smejkalova 2011/10/20 15:29 smejkalova vytvořeno Go Next revision Both sides next revision
external:spr [2011/10/21 14:11] smejkalova		external:spr [2011/10/21 14:32] smejkalova
Line 31:		Line 31:
-	Here we present a small portion of the data just to show its structure.	+	Here we present a small portion of the data just to show its structure. For each verb we define a set of semantic patterns and provide a  sample of manually annotated corpus concordances. Then we do a detailed interannotator disagreement analysis, error detection and adjudication.
	==== Annotation Scheme Description ====		==== Annotation Scheme Description ====
-	* Specification of form for defining patterns - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/PDEV2.1-pattern-form.pdf|pdf]]	+	* Pattern Definition Form - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/PDEV2.1-pattern-form.pdf|pdf]]
	* Annotation Manual - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/annotation_manual.pdf|pdf]]		* Annotation Manual - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/annotation_manual.pdf|pdf]]
		+
		+
		+	==== Pattern Definitions Preview ====
		+
		+	Pattern definitions - we have revised the pattern definitions for 30 verbs. Here is the sample of three of them (after revision):
		+	* [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_patterns.html|cool]]
		+	* [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_patterns.html|deny]] - detailed [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_7.png|view]] of pattern number 7
		+	* [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_patterns.html|yield]]
		+
	==== Annotated Data Preview ====		==== Annotated Data Preview ====
		+	Annotation of 50 concordances per each of these verbs by three annotators. Here is a little sample of three verbs. There has been already finished the manual disagreements analysis and all instances were adjudicated and gold sample was created. Manual disagreements analysis and adjudication:
		+	* cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_analysis.pdf|pdf]]
		+	* deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_analysis.pdf|pdf]]
		+	* yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_analysis.pdf|pdf]]
	==== Disagreement Analysis Preview ====		==== Disagreement Analysis Preview ====
		+
		+	Automatic disagreements analysis - confusion matrix for each pair of the annotators in one file
		+	* cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_res.txt|txt]]
		+	* deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_res.txt|txt]]
		+	* yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_res.txt|txt]]
-	* Pattern definitions - we have revised the pattern definitions for 30 verbs. Here is the sample of three of them (after revision):
-	* [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_patterns.html|cool]]
-	* [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_patterns.html|deny]] - detailed [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_7.png|view]] of pattern number 7
-	* [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_patterns.html|yield]]
-	* Annotation of 50 concordances per each of these verbs by three annotators. Here is a little sample of three verbs. There has been already finished the manual disagreements analysis and all instances were adjudicated and gold sample was created.
-	* Manual disagreements analysis and adjudication:
-	* cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_analysis.pdf|pdf]]
-	* deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_analysis.pdf|pdf]]
-	* yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_analysis.pdf|pdf]]
-	* Automatic disagreements analysis - confusion matrix for each pair of the annotators in one file
-	* cool - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/cool_res.txt|txt]]
-	* deny - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/deny_res.txt|txt]]
-	* yield - [[http://ufal.mff.cuni.cz/~smejkalova/pdev/yield_res.txt|txt]]
-	* Inter-annotator agreement (Cohen's kappa for each pair, Fleiss' kappa for all together
-	Results of inter-annotator agreement
-	^verb     ^  size  ^ #N ^ Fleiss' kappa ^        Cohen's kappa            ^^^
-	|         |        |    |               | A2 vs. A3 | A2 vs. A1 | A3 vs. A1 |
-	|cool   |      50|  16|    0.685|      0.743|      0.669|      0.646|
-	|deny   |      50|  10|          0.524|      0.434|      0.571|      0.582|
-	|yield    |      50|  10|    0.500|      0.489|      0.588|      0.429|

---

[ Back to the navigation ] [ Back to the content ]