[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]

You are here: start » user » zeman » treebanks » ja

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
user:zeman:treebanks:ja [2012/01/04 09:25]
zeman Domain and size. 		user:zeman:treebanks:ja [2012/01/04 09:45]
zeman Inside.
Line 42: Line 42:
 ==== Inside ==== ==== Inside ====
  
-The original morphosyntactic tags have been converted to fit into the three columns (CPOS, POS and FEAT) of the CoNLL format. There //should// be a 1-1 mapping between the [[http://www.buch-kromann.dk/matthias/treebank/PAROLE-manual.pdf|DDT positional tags]] and the CoNLL 2006 annotation. Use [[http://quest.ms.mff.cuni.cz/cgi-bin/interset/index.pl?tagset=da::conll|DZ Interset]] to inspect the CoNLL tagset.+The text has been romanized and the original characters (kanji + kana) are not available. There should be a 1-1 mapping between the romanized text (rōmaji) and the Japanese script of hiragana. There is no indication though where katakana or kanji are preferred over hiragana.
  
-The morphological analysis in the CoNLL 2006 version does not include lemmas (the original DTAG version does contain them). The morphosyntactic tags have been assigned (probably) manually. +The morphological analysis does not include lemmas. The part-of-speech tags have been assigned (probably) manually. Few morphosyntactic features are used.
- +
-Some multi-word expressions have been collapsed into one token, using underscore as the joining character. This includes adverbially used prepositional phrases (e.g. i_lørdags = on Saturdays) but not named entities.+
  
 ==== Sample ==== ==== Sample ====
  
-The first sentence of DDT 1.0 in the DTAG format: +The first three sentences of the CoNLL 2006 training data:
- +
-<code xml><tei.2> +
-  <teiHeader type=text> +
-    <fileDesc> +
-      <titleStmt> +
-        <title>Tagged sample of: 'Jeltsins skæbnetime'</title> +
-      </titleStmt> +
-      <extent words=158>158 running words</extent> +
-      <publicationStmt> +
-         <distributor>PAROLE-DK</distributor> +
-         <address><addrline>Christians Brygge 1,1., DK-1219 Copenhagen K.</address> +
-         <date>1998-06-02</date> +
-         <availability status=restricted><p>by agreement with distributor</availability> +
-      </publicationStmt> +
-      <sourceDesc> +
-        <biblStruct> +
-          <analytic> +
-            <title>Jeltsins skæbnetime</title> +
-            <author gender=m born=1925>Nikulin, Leon</author> +
-          </analytic> +
-          <monogr> +
-            <imprint><pubPlace>Denmark</pubPlace> +
-              <publisher>Det Fri Aktuelt</publisher> +
-              <date>1992-12-01</date> +
-            </imprint> +
-          </monogr> +
-        </biblStruct> +
-      </sourceDesc> +
-    </fileDesc> +
-    <profileDesc> +
-      <creation>1992-12-01</creation> +
-      <langUsage><language>Danish</langUsage> +
-      <textClass> +
-        <catRef target="P.M2"> +
-        <catRef target="P.G4.8"> +
-        <catRef target="P.T9.3"> +
-      </textClass> +
-    </profileDesc> +
-  </teiHeader> +
-<text id=AJK> +
-<body> +
-<div1 type=main> +
-<p> +
-<s> +
-<W lemma="to" msd="AC---U=--" in="9:subj" out="1:mod|2:mod|3:nobj|5:appr">To</W> +
-<W lemma="kendt" msd="ANP[CN]PU=[DI]U" in="-1:mod" out="">kendte</W> +
-<W lemma="russisk" msd="ANP[CN]PU=[DI]U" in="-2:mod" out="">russiske</W> +
-<W lemma="historiker" msd="NCCPU==I" in="-3:nobj" out="">historikere</W> +
-<W lemma="Andronik" msd="NP--U==-" in="1:namef" out="">Andronik</W> +
-<W lemma="Mirganjan" msd="NP--U==-" in="-5:appr" out="-1:namef|1:coord">Mirganjan</W> +
-<W lemma="og" msd="CC" in="-1:coord" out="2:conj">og</W> +
-<W lemma="Igor" msd="NP--U==-" in="1:namef" out="">Igor</W> +
-<W lemma="Klamkin" msd="NP--U==-" in="-2:conj" out="-1:namef">Klamkin</W> +
-<W lemma="tro" msd="VADR=----A-" in="" out="-9:subj|1:mod|2:pnct|3:dobj|12:pnct">tror</W> +
-<W lemma="ikke" msd="RGU" in="-1:mod" out="">ikke</W> +
-<W lemma="," msd="XP" in="-2:pnct" out="">,</W> +
-<W lemma="at" msd="CS" in="-3:dobj" out="2:vobj">at</W> +
-<W lemma="Rusland" msd="NP--U==-" in="1:subj|2:[subj]" out="">Rusland</W> +
-<W lemma="kunne" msd="VADR=----A-" in="-2:vobj" out="-1:subj|1:vobj|2:mod">kan</W> +
-<W lemma="udvikle" msd="VAF-=----P-" in="-1:vobj" out="-2:[subj]">udvikles</W> +
-<W lemma="uden" msd="SP" in="-2:mod" out="1:nobj">uden</W> +
-<W lemma="en" msd="PI-CSU--U" in="-1:nobj" out="2:nobj">en</W> +
-<W lemma="&quot;" msd="XP" in="1:pnct" out="">"</W> +
-<W lemma="jernnæve" msd="NCCSU==I" in="-2:nobj" out="-1:pnct|1:pnct">jernnæve</W> +
-<W lemma="&quot;" msd="XP" in="-1:pnct" out="">"</W> +
-<W lemma="." msd="XP" in="-12:pnct" out="">.</W> +
-</s></code> +
- +
-The first sentence of the CoNLL 2006 training data:+
  
-| 1 | Samme | _ | A | AN | degree=pos<nowiki>|</nowiki>gender=common/neuter<nowiki>|</nowiki>number=sing/plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=def/indef<nowiki>|</nowiki>transcat=unmarked | 0 | ROOT | _ | _ | +| 1 | kasahara | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD | 
-| 2 | cifre | _ | N | NC | gender=neuter<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=indef | 1 | nobj | _ | _ | +| 2 | arisa | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP | 
-| 3 | , | _ | X | XP | _ | 1 | pnct | _ | _ | +| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT | 
-| 4 | de | _ | P | PD | gender=common/neuter<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>register=unmarked | 7 | subj | _ | _ | +| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT | 
-| 5 | norske | _ | A | AN | degree=pos<nowiki>|</nowiki>gender=common/neuter<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=def/indef<nowiki>|</nowiki>transcat=unmarked | 4 | mod | _ | _ | +| |||||||||| 
-| 6 | piger | _ | N | NC | gender=common<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=indef | 4 | nobj | _ | _ | +| 1 | kadowaki | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD | 
-| 7 | tabte | _ | V | VA | mood=indic<nowiki>|</nowiki>tense=past<nowiki>|</nowiki>voice=active | 1 | rel | _ | _ | +| 2 | masakazu | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP | 
-| 8 | med | _ | SP | SP | _ | 7 | pobj | _ | _ | +| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT | 
-| 9 | i_lørdags | _ | RG | RG | degree=unmarked | 7 | mod | _ | _ | +| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT | 
-| 10 | mod | _ | SP | SP | _ | 7 | pobj | _ | _ | +| |||||||||| 
-| 11 | VMs | _ | N | NP | case=gen | 10 | nobj | _ | _ | +| 1 | kadowaki | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | COMP | 2 | COMP | 
-| 12 | værtsnation | _ | N | NC | gender=common<nowiki>|</nowiki>number=sing<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=indef | 11 | possd | _ | _ | +| 2 | saN | <nowiki>_</nowiki> | P | PNsf | <nowiki>_</nowiki> | 0 | ROOT | 0 | ROOT | 
-| 13 | . | _ | X | XP | _ | 1 | pnct | _ | _ |+| 3 | omatase | <nowiki>_</nowiki> | N | VN | <nowiki>_</nowiki> | 4 | COMP | 4 | COMP | 
 +| 4 | shimashita | <nowiki>_</nowiki> | VS | VSfin | ta | 0 | ROOT | 0 | ROOT | 
 +| 5 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 4 | PUNCT | 4 | PUNCT |
  
-The first sentence of the CoNLL 2006 test data:+The first three sentences of the CoNLL 2006 test data:
  
-| 1 | To | _ | A | AC | case=unmarked | 10 | subj | _ | _ | +| 1 | tashiro | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD | 
-| 2 | kendte | _ | A | AN | degree=pos<nowiki>|</nowiki>gender=common/neuter<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=def/indef<nowiki>|</nowiki>transcat=unmarked | 1 | mod | _ | _ | +| 2 | yasuko | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP | 
-| 3 | russiske | _ | A | AN | degree=pos<nowiki>|</nowiki>gender=common/neuter<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=def/indef<nowiki>|</nowiki>transcat=unmarked | 1 | mod | _ | _ | +| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT | 
-| 4 | historikere | _ | N | NC | gender=common<nowiki>|</nowiki>number=plur<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=indef | 1 | nobj | _ | _ | +| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT | 
-| 5 | Andronik | _ | N | NP | case=unmarked | 6 | namef | _ | _ | +| |||||||||| 
-| 6 | Mirganjan | _ | N | NP | case=unmarked | 1 | appr | _ | _ | +| 1 | hayakawa | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD | 
-| 7 | og | _ | C | CC | _ | 6 | coord | _ | _ | +| 2 | ryou | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP | 
-| 8 | Igor | _ | N | NP | case=unmarked | 9 | namef | _ | _ | +| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT | 
-| 9 | Klamkin | _ | N | NP | case=unmarked | 7 | conj | _ | _ | +| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT | 
-| 10 | tror | _ | V | VA | mood=indic<nowiki>|</nowiki>tense=present<nowiki>|</nowiki>voice=active | 0 | ROOT | _ | _ | +| |||||||||| 
-| 11 | ikke | _ | RG | RG | degree=unmarked | 10 | mod | _ | _ | +| 1 | hayakawa | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | COMP | 2 | COMP | 
-| 12 | , | _ | X | XP | _ | 10 | pnct | _ | _ | +| 2 | saN | <nowiki>_</nowiki> | P | PNsf | <nowiki>_</nowiki> | 6 | ADJ | 6 | ADJ | 
-| 13 | at | _ | C | CS | _ | 10 | dobj | _ | _ | +| 3 | ima | <nowiki>_</nowiki> | N | Ntmp | <nowiki>_</nowiki> | 6 | ADJ | 6 | ADJ | 
-| 14 | Rusland | _ | N | NP | case=unmarked | 15 | subj | _ | _ | +| 4 | chotto | <nowiki>_</nowiki> | ADV | ADV | <nowiki>_</nowiki> | 6 | ADJ | 6 | ADJ | 
-| 15 | kan | _ | V | VA | mood=indic<nowiki>|</nowiki>tense=present<nowiki>|</nowiki>voice=active | 13 | vobj | _ | _ | +| 5 | ojikaN | <nowiki>_</nowiki> | N | NN | <nowiki>_</nowiki> | 6 | SBJ | 6 | SBJ | 
-| 16 | udvikles | _ | V | VA | mood=infin<nowiki>|</nowiki>voice=passive | 15 | vobj | _ | _ | +| 6 | yoroshii | <nowiki>_</nowiki> | ADJ | ADJifin | <nowiki>_</nowiki> | 7 | COMP | 7 | COMP | 
-| 17 | uden | _ | SP | SP | _ | 15 | mod | _ | _ | +| 7 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT | 
-| 18 | en | _ | P | PI | gender=common<nowiki>|</nowiki>number=sing<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>register=unmarked | 17 | nobj | _ | _ | +| 8 | ka | <nowiki>_</nowiki> | PS | PSE | <nowiki>_</nowiki> | 7 | MRK | 7 | MRK | 
-| 19 | " | _ | X | XP | _ | 20 | pnct | _ | _ | +| 9 | chotto | <nowiki>_</nowiki> | ADV | ADV | <nowiki>_</nowiki> | 16 | ADJ | 16 | ADJ | 
-| 20 | jernnæve | _ | N | NC | gender=common<nowiki>|</nowiki>number=sing<nowiki>|</nowiki>case=unmarked<nowiki>|</nowiki>def=indef | 18 | nobj | _ | _ | +| 10 | doitsu | <nowiki>_</nowiki> | NAME | NAMEloc | <nowiki>_</nowiki> | 11 | COMP | 11 | COMP | 
-| 21 | " | _ | X | XP | _ | 20 | pnct | _ | _ | +| 11 | no | <nowiki>_</nowiki> | P | Pgen | <nowiki>_</nowiki> | 12 | ADJ | 12 | ADJ | 
-| 22 | . | _ | X | XP | _ | 10 | pnct | _ | _ |+| 12 | shucchou | <nowiki>_</nowiki> | N | VN | <nowiki>_</nowiki> | 13 | COMP | 13 | COMP | 
 +| 13 | no | <nowiki>_</nowiki> | P | Pgen | <nowiki>_</nowiki> | 14 | COMP | 14 | COMP | 
 +| 14 | koto | <nowiki>_</nowiki> | N | NF | <nowiki>_</nowiki> | 15 | COMP | 15 | COMP | 
 +| 15 | de | <nowiki>_</nowiki> | P | P | <nowiki>_</nowiki> | 16 | ADJ | 16 | ADJ | 
 +| 16 | gosoudaN | <nowiki>_</nowiki> | N | VN | <nowiki>_</nowiki> | 17 | COMP | 17 | COMP | 
 +| 17 | shitai | <nowiki>_</nowiki> | VADJ | VADJi | <nowiki>_</nowiki> | 18 | COMP | 18 | COMP | 
 +| 18 | no | <nowiki>_</nowiki> | N | NF | <nowiki>_</nowiki> | 19 | COMP | 19 | COMP | 
 +| 19 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 20 | COMP | 20 | COMP | 
 +| 20 | ga | <nowiki>_</nowiki> | PS | PSSb | <nowiki>_</nowiki> | 0 | ROOT | 0 | ROOT | 
 +| 21 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 20 | PUNCT | 20 | PUNCT |
  
 ==== Parsing ==== ==== Parsing ====

[ Back to the navigation ] [ Back to the content ]