[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]

You are here: start » ufal » tasks

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision Both sides next revision
ufal:tasks [2012/01/18 14:49]
ufal 		ufal:tasks [2012/01/18 14:53]
ufal
Line 5: Line 5:
  
 === Europarl tokenizer === === Europarl tokenizer ===
-  * **info:** A sample tokenizer, distributed as a part of the Europarl tools+  * **info:** A sample rule-based tokenizer, can use a list of prefixes which are usually followed by a dot but don't break a sentence. Distributed as a part of the Europarl tools.
   * **version:** v6 (Jan 2012)    * **version:** v6 (Jan 2012) 
   * **author:** Philipp Koehn and Josh Schroeder   * **author:** Philipp Koehn and Josh Schroeder

[ Back to the navigation ] [ Back to the content ]