This page is an add-on to the following paper:

Ondřej Bojar, Pavel Straňák, Daniel Zeman, Gaurav Jain, Michal Hrušecký, Michal Richter, Jan Hajič: English-Hindi Translation – Obtaining Mediocre Results with Bad Data and Fancy Models. In: Proceedings of ICON 2009, Hyderabad, December.

The purpose of the add-on page is to provide detailed documentation of the data, tools and settings used so that the results can be reproduced by other researchers.

• Data
  • IIIT-TIDES
  • Daniel Pipes
  • EMILLE
  • Agrocorpus
  • Shabdanjali
  • Wikipedia Named Entities
• Tools and their settings
  • Tokenization and normalization of the data
  • Hunalign
  • GIZA++
  • makecls
  • SRILM
  • Moses
  • Joshua
  • Mumbai Tagger
  • Affisix
  • Hindomor
  • HiTBSuf
  • počítadlo BLEU skóre
• Link tables from the paper to concrete settings
• Link to the PDF version of the paper; link to Biblio?

A dataset originally collected for the DARPA-TIDES surprise-language contest in 2002, later refined at IIIT Hyderabad and provided for the NLP Tools Contest at ICON 2008. For availability, enquire at the IIIT (http://ltrc.iiit.ac.in/). If you have the data, make sure that you have the same number of sentences and tokens. The Hindi side has been provided in two encodings, WX romanization and UTF-8. Although they should be equivalent, we always work with UTF-8.

The test data contain 1 reference translation per sentence.

No data have been lemmatised, so all the numbers mean forms.

- set1 = danielpipes-11+agrocorp-11+wikiner2008+wikiner2009+acl2005
- set2 = emille-11+danielpipes-11+agrocorp-11+wikiner2008+wikiner2009+acl2005
- set3 = emille-om+danielpipes-11+agrocorp-11+wikiner2008+wikiner2009+acl2005