| Both sides previous revision
Previous revision
|
Next revision
Both sides next revision
|
courses:rg:wishlist [2013/04/22 23:39]
machys |
courses:rg:wishlist [2013/10/04 10:59]
popel |
| * Martin Popel would appreciate two RG meetings devoted to significance tests & MT evaluation. The two presenters should together read the following 4 papers (and related ones) and select two for presenting (one on bootstrap, one on approximate randomization). | * Martin Popel would appreciate two RG meetings devoted to significance tests & MT evaluation. The two presenters should together read the following 4 papers (and related ones) and select two for presenting (one on bootstrap, one on approximate randomization). |
| - <del>Stefan Riezler and John T. Maxwell III: [[http://acl.ldc.upenn.edu/W/W05/W05-0908.pdf|On Some Pitfalls in Automatic Evaluation and Significance Testing for MT]] (page 67) ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation, 2005.</del> | - <del>Stefan Riezler and John T. Maxwell III: [[http://acl.ldc.upenn.edu/W/W05/W05-0908.pdf|On Some Pitfalls in Automatic Evaluation and Significance Testing for MT]] (page 67) ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation, 2005.</del> |
| - Nicolas Stroppa, Karolina Owczarzak, Andy Way: [[http://doras.dcu.ie/15227/1/stroppa_owczarzak_07.pdf|A Cluster-Based Representation for Multi-System MT Evaluation]], 2007. | - <del>Nicolas Stroppa, Karolina Owczarzak, Andy Way: [[http://doras.dcu.ie/15227/1/stroppa_owczarzak_07.pdf|A Cluster-Based Representation for Multi-System MT Evaluation]], 2007</del>. |
| - <del>Philipp Koehn: [[http://acl.ldc.upenn.edu/acl2004/emnlp/pdf/Koehn.pdf|Statistical significance tests for machine translation evaluation]], EMNLP 2004.</del> | - <del>Philipp Koehn: [[http://acl.ldc.upenn.edu/acl2004/emnlp/pdf/Koehn.pdf|Statistical significance tests for machine translation evaluation]], EMNLP 2004.</del> |
| - Ying Zhang, Stephan Vogel, Alex Waibel: [[http://www.lrec-conf.org/proceedings/lrec2004/pdf/755.pdf|Interpreting BLEU/NIST Scores: How Much Improvement Do We Need to Have a Better System?]] | - Ying Zhang, Stephan Vogel, Alex Waibel: [[http://www.lrec-conf.org/proceedings/lrec2004/pdf/755.pdf|Interpreting BLEU/NIST Scores: How Much Improvement Do We Need to Have a Better System?]] |
| |
| * <del>Chi-kiu LO and Dekai WU: [[http://www.cs.ust.hk/~dekai/library/WU_Dekai/LoWu_Acl2011.pdf|MEANT: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility via semantic frames. ACL HLT 2011]]</del> | * <del>Chi-kiu LO and Dekai WU: [[http://www.cs.ust.hk/~dekai/library/WU_Dekai/LoWu_Acl2011.pdf|MEANT: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility via semantic frames. ACL HLT 2011]]</del> |
| * Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn: [[http://people.psych.cornell.edu/~jec7/pcd%20pubs/simmonsetal11.pdf|False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant]], Psychological Science, 2011. Yes, it is a psychological paper, but it is very valuable for anyone doing/reading any evaluation with significance tests. | * <del>Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn: [[http://people.psych.cornell.edu/~jec7/pcd%20pubs/simmonsetal11.pdf|False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant]], Psychological Science, 2011.</del> Yes, it is a psychological paper, but it is very valuable for anyone doing/reading any evaluation with significance tests. |
| |
| ==== Language and Vision ==== | ==== Language and Vision ==== |
