===== Japanese (ja) =====

[[http://www.sfs.uni-tuebingen.de/en/ascl/resources/corpora/tueba-js.html|Tübingen Treebank of Spoken Japanese]] (TüBa-J/S, Verbmobil project)

==== Versions ====

  * Original TüBa-J/S in the Negra export format
  * CoNLL 2006

The original TüBa-J/S is HPSG-oriented, there is the lexical level, the phrasal level, the clausal level, and dependency edges between nodes. The CoNLL version contains only the dependency relations.

==== Obtaining and License ====

To obtain the treebank, download [[http://www.sfs.uni-tuebingen.de/resources/tuebajs-license.pdf|the license agreement]], print it, fill it out and sign it, scan and send it back to Kathrin Beck (kbeck (at) sfs (dot) uni-tuebingen (dot) de). She will send you the password for the download page. The license in short:

  * academic research usage
  * no redistribution
  * citation in publications not required (but it is common decency)

TüBa-J/S was created in the Verbmobil project by members of the [[http://www.sfs.uni-tuebingen.de/|Seminar für Sprachwissenschaft]], Eberhard Karls Universität Tübingen, Wilhelmstrasse 19, D-72074 Tübingen, Germany.

==== References ====

  * Website
    * http://www.sfs.uni-tuebingen.de/en/tuebajs.shtml
  * Data
    * //no separate citation//
  * Principal publications
    * Yasuhiro Kawata, Julia Bartels: Stylebook for the Japanese Treebank in Verbmobil, Report 240, September 29, 2000.
    * Sabine Buchholz, Erwin Marsi: [[http://acl.ldc.upenn.edu/W/W06/W06-29.pdf#page=165|CoNLL-X shared task on Multilingual Dependency Parsing]]. In: Proceedings of the 10th Conference on Computational Natural Language Learning (CoNLL-X), pp. 149-164, New York, USA, 2006.
  * Documentation
    * Yasuhiro Kawata, Julia Bartels: Stylebook for the Japanese Treebank in Verbmobil, {{:user:zeman:treebanks:report-240-00.pdf|Report 240}}, has been distributed together with the CoNLL 2006 version of the treebank (file ''doc/report-240-00.ps'').

==== Domain ====

Spoken dialogues, negotiations about time and place of business meetings. That is why many sentences are relatively short (a frequent single-word sentence is //hai// = “yes”).

==== Size ====

The CoNLL 2006 version contains 157,172 tokens in 17753 sentences, yielding 8.85 tokens per sentence on average (CoNLL 2006 data split: 151,461 tokens / 17044 sentences training, 5711 tokens / 709 sentences test).

==== Inside ====

The text has been romanized and the original characters (kanji + kana) are not available. There should be a 1-1 mapping between the romanized text (rōmaji) and the Japanese script of hiragana. There is no indication though where katakana or kanji are preferred over hiragana.

The morphological analysis does not include lemmas. The part-of-speech tags have been assigned (probably) manually. Few morphosyntactic features are used.

==== Sample ====

The first three sentences of the CoNLL 2006 training data:

| 1 | kasahara | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD |
| 2 | arisa | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP |
| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT |
| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT |
| ||||||||||
| 1 | kadowaki | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD |
| 2 | masakazu | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP |
| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT |
| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT |
| ||||||||||
| 1 | kadowaki | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | COMP | 2 | COMP |
| 2 | saN | <nowiki>_</nowiki> | P | PNsf | <nowiki>_</nowiki> | 0 | ROOT | 0 | ROOT |
| 3 | omatase | <nowiki>_</nowiki> | N | VN | <nowiki>_</nowiki> | 4 | COMP | 4 | COMP |
| 4 | shimashita | <nowiki>_</nowiki> | VS | VSfin | ta | 0 | ROOT | 0 | ROOT |
| 5 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 4 | PUNCT | 4 | PUNCT |

The first three sentences of the CoNLL 2006 test data:

| 1 | tashiro | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD |
| 2 | yasuko | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP |
| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT |
| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT |
| ||||||||||
| 1 | hayakawa | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | HD | 2 | HD |
| 2 | ryou | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 3 | COMP | 3 | COMP |
| 3 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT |
| 4 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 3 | PUNCT | 3 | PUNCT |
| ||||||||||
| 1 | hayakawa | <nowiki>_</nowiki> | NAME | NAMEper | <nowiki>_</nowiki> | 2 | COMP | 2 | COMP |
| 2 | saN | <nowiki>_</nowiki> | P | PNsf | <nowiki>_</nowiki> | 6 | ADJ | 6 | ADJ |
| 3 | ima | <nowiki>_</nowiki> | N | Ntmp | <nowiki>_</nowiki> | 6 | ADJ | 6 | ADJ |
| 4 | chotto | <nowiki>_</nowiki> | ADV | ADV | <nowiki>_</nowiki> | 6 | ADJ | 6 | ADJ |
| 5 | ojikaN | <nowiki>_</nowiki> | N | NN | <nowiki>_</nowiki> | 6 | SBJ | 6 | SBJ |
| 6 | yoroshii | <nowiki>_</nowiki> | ADJ | ADJifin | <nowiki>_</nowiki> | 7 | COMP | 7 | COMP |
| 7 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 0 | ROOT | 0 | ROOT |
| 8 | ka | <nowiki>_</nowiki> | PS | PSE | <nowiki>_</nowiki> | 7 | MRK | 7 | MRK |
| 9 | chotto | <nowiki>_</nowiki> | ADV | ADV | <nowiki>_</nowiki> | 16 | ADJ | 16 | ADJ |
| 10 | doitsu | <nowiki>_</nowiki> | NAME | NAMEloc | <nowiki>_</nowiki> | 11 | COMP | 11 | COMP |
| 11 | no | <nowiki>_</nowiki> | P | Pgen | <nowiki>_</nowiki> | 12 | ADJ | 12 | ADJ |
| 12 | shucchou | <nowiki>_</nowiki> | N | VN | <nowiki>_</nowiki> | 13 | COMP | 13 | COMP |
| 13 | no | <nowiki>_</nowiki> | P | Pgen | <nowiki>_</nowiki> | 14 | COMP | 14 | COMP |
| 14 | koto | <nowiki>_</nowiki> | N | NF | <nowiki>_</nowiki> | 15 | COMP | 15 | COMP |
| 15 | de | <nowiki>_</nowiki> | P | P | <nowiki>_</nowiki> | 16 | ADJ | 16 | ADJ |
| 16 | gosoudaN | <nowiki>_</nowiki> | N | VN | <nowiki>_</nowiki> | 17 | COMP | 17 | COMP |
| 17 | shitai | <nowiki>_</nowiki> | VADJ | VADJi | <nowiki>_</nowiki> | 18 | COMP | 18 | COMP |
| 18 | no | <nowiki>_</nowiki> | N | NF | <nowiki>_</nowiki> | 19 | COMP | 19 | COMP |
| 19 | desu | <nowiki>_</nowiki> | PV | PVfin | u | 20 | COMP | 20 | COMP |
| 20 | ga | <nowiki>_</nowiki> | PS | PSSb | <nowiki>_</nowiki> | 0 | ROOT | 0 | ROOT |
| 21 | <nowiki>.</nowiki> | <nowiki>_</nowiki> | <nowiki>.</nowiki> | <nowiki>.</nowiki> | <nowiki>_</nowiki> | 20 | PUNCT | 20 | PUNCT |

==== Parsing ====

Nonprojectivities in TüBa-J/S are not frequent. Only 1736 of the 157,172 tokens in the CoNLL 2006 version are attached nonprojectively (1.1%).

The results of the CoNLL 2006 shared task are [[http://ilk.uvt.nl/conll/results.html|available online]]. They have been published in [[http://aclweb.org/anthology-new/W/W06/W06-2920.pdf|(Buchholz and Marsi, 2006)]]. The evaluation procedure was non-standard because it excluded punctuation tokens. These are the best results for Japanese:

^ Parser (Authors) ^ LAS ^ UAS ^
| Basis (John O'Neil) | 90.57 | 93.16 |
| Nara (Yuchang Cheng) | 89.91 | 93.12 |
| Malt (Nivre et al.) | 91.65 | 93.10 |