[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]

You are here: start » courses » rg » 2012 » encouraging-consistent-translation

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
courses:rg:2012:encouraging-consistent-translation [2012/10/16 15:13]
dusek 		courses:rg:2012:encouraging-consistent-translation [2012/10/17 11:59]
dusek
Line 8: Line 8:
 The list of discussed topics follows the outline of the paper: The list of discussed topics follows the outline of the paper:
 ==== Sec. 2. Related Work ==== ==== Sec. 2. Related Work ====
- 
 **Differences from Carpuat 2009** **Differences from Carpuat 2009**
   * It is different: the decoder just gets additional features, but the decision is up to it -- Carpuat 2009 just post-edits the outputs and substitutes the most likely variant everywhere   * It is different: the decoder just gets additional features, but the decision is up to it -- Carpuat 2009 just post-edits the outputs and substitutes the most likely variant everywhere
Line 17: Line 16:
     * The authors do not state their evidence clearly.     * The authors do not state their evidence clearly.
     * One sense is not the same as one translation     * One sense is not the same as one translation
-==== Sec. 3. Exploratory analysis ==== 
  
 +==== Sec. 3. Exploratory analysis ====
 **Hiero** **Hiero**
   * The idea would most probably work the same in normal phrase-based SMT, but the authors use hierarchical phrase-based translation (Hiero)   * The idea would most probably work the same in normal phrase-based SMT, but the authors use hierarchical phrase-based translation (Hiero)
Line 33: Line 32:
 **The choice and filtering of "cases"** **The choice and filtering of "cases"**
   * The "cases" in Table 1 are selected according to the //possibility// of different translations (i.e. each case has at least two translations of the source seen in the training data; the translation counts are from the test data, so it is OK that e.g. "Korea" translates as "Korea" all the time)   * The "cases" in Table 1 are selected according to the //possibility// of different translations (i.e. each case has at least two translations of the source seen in the training data; the translation counts are from the test data, so it is OK that e.g. "Korea" translates as "Korea" all the time)
-  * Table 1 is unfiltered -- only some of the "cases" are then considered relevant +  * Table 1 is unfiltered -- only some of the "cases" are then considered relevant: 
-    * Cases that are too similar (less than 1/2 characters differ) are joined together +    * Cases that are //too similar// (less than 1/2 characters differ) are //joined together// 
-      * Beware, this notion of grouping is not well-defined, does not create equivalence classes: ''old hostages'' = ''new hostages'' = ''completely new hostages'' but ''old hostages'' != ''completely new hostages'' (we hope this didn't actually happen)+      * Beware, this notion of grouping is not well-defined, does not create equivalence classes: "old hostages" = "new hostages" = "completely new hostages" but "old hostages" != "completely new hostages" (we hope this didn't actually happen) 
 +    * Cases where //only one translation variant prevails// are //discarded// (this is the case of "Korea") 
 + 
 +==== Sec. 4. Approach ==== 
 +The actual experiments begin only now; the used data is different. 
 + 
 +**Choice of features** 
 +  * They define 3 features that are designed to be biased towrds consistency -- or are they? 
 +    * If e.g. two variants are used 2 times each, they will have roughly the same score 
 +  * The BM25 function is a refined version of the [[http://en.wikipedia.org/wiki/TF-IDF|TF-IDF]] score 
 +  * The exact parameter values are probably not tuned, left at a default value (and maybe they don't have much influence anyway) 
 +   * See NPFL103 for details on Information retrieval, it's largely black magic 
 + 
 +**Feature weights** 
 +  * The usual model in MT is scoring the hypotheses according to the feature values (''f'') and their weights (''lambda''):  
 +    * ''score(H) = exp( sum( lambda_i * f_i(H)) )'' 
 +  * The feature weights are trained on a heldout data set using [[http://acl.ldc.upenn.edu/acl2003/main/pdfs/Och.pdf|MERT]] (or, here: [[http://en.wikipedia.org/wiki/Margin_Infused_Relaxed_Algorithm|MIRA]]) 
 +  * The resulting weights are not mentioned, but if the weight is < 0, will this favor different translation choices? 
 + 
 +**Meaning of the individual features** 
 +  * C1 indicates that a certain Hiero rule was used frequently 
 +    * but rules are very similar, so we also need something less fine-grained 
 +  * C2 is a target-side feature, just counts the target side tokens (only the "most important" ones; in terms of TF-IDF) 
 +    * It may be compared to Language Model features, but is trained only on the target part of the bilingual tuning data. 
 +  * C3 counts occurrences of source-target token pairs (and uses the "most important" term pair for each rule, again) 
 + 
 +**Requirements of the new features** 
 +  * They need two passes through the data 
 +  * You need to have document segmentation 
 +    * Since the frequencies are trained on the tuning set (see Sec. 5), you can just translate one document at a time, no need to have full sets of documents 
 + 
 +==== Sec. 5. Evaluation and Discussion ==== 
 +**Choice of baseline** 
 +  * Baselines are quite nice and competitive, we believe this really is an improvement 
 +  * MIRA is very cutting-edge 
 + 
 +**Tuning the feature weights** 
 +  * For the 1st phase, "heuristically" probably means they just used some reasonable enough values, e.g. from earlier experiments 
 +    * This is in order to speed up the experiment, they don't want to wait for MIRA twice. 
 + 
 +**Different evaluation metrics** 
 +  * The BLEU variants do not differ that much, only in Brevity Penalty for multiple references 
 +    * IBM BLEU uses the reference that is closest to the MT output (in terms of length), NIST BLEU uses the shortest one 
 +  * This was probably just due to some technical reasons, e.g. they had their optimization software designed for one metric and not the other 

[ Back to the navigation ] [ Back to the content ]