Differences
This shows you the differences between two versions of the page.
| Both sides previous revision
Previous revision
|
|
courses:rg:2012:meant [2012/11/13 00:14]
rosa stlye |
courses:rg:2012:meant [2012/11/13 16:25] (current)
popel |
| |
| ===== 4 Meta-evaluation methodology ===== | ===== 4 Meta-evaluation methodology ===== |
| Here, we reminded the difference between Kendall's τ and Spearman's τ. Kendall's τ only takes the ranks into account, disregarding the actual scores, while Spearman's τ takes the scores into account. The formula for Kendall's τ is τ = (#same rank - #different rank) / #pairs. | Here, we reminded the difference between Kendall's τ and Spearman's τ. Kendall's τ only takes the ranks into account, disregarding the actual scores, while Spearman's τ takes the scores into account. The formula for [[http://en.wikipedia.org/wiki/Kendall%27s_tau|Kendall's τ]] is τ = (#same_ordered_pairs - #opposite_ordered_pairs) / #all_pairs. |
| |
| Martin also remarked that the authors use sentence-level BLEU to compute the correlation; however, BLEU was designed for whole documents, not for individual sentences, and therefore should preferably not be used on sentence level. | Martin also remarked that the authors use sentence-level BLEU to compute the correlation; however, BLEU was designed for whole documents, not for individual sentences, and therefore should preferably not be used on sentence level. |
