Both sides previous revision
Previous revision
Next revision
|
Previous revision
Next revision
Both sides next revision
|
external:addicter [2011/05/16 07:13] mphi |
external:addicter [2011/05/16 07:59] mphi |
| |
==== Word Alignment -- Progress and Results ==== | ==== Word Alignment -- Progress and Results ==== |
| |
| === Latest best results === |
| [[http://mtj.ut.ee/addicter-best.txt|txt]] |
| |
=== Alternative model comparison === | === Alternative model comparison === |
| ^ Precision/Recall/F-score: ^^^^ | | ^ Precision/Recall/F-score: ^^^^ |
| ^ Lex ^ Order ^ Punct ^ Miss ^ | | ^ Lex ^ Order ^ Punct ^ Miss ^ |
^ meteor |0.092/0.251/**0.135** |0.047/0.229/**0.078** |0.248/0.665/**0.361** |0.020/0.382/**0.038** | | |
^ ter* |0.106/0.387/**0.167** |0.025/0.191/**0.044** |0.132/0.936/**0.232** |0.026/0.170/**0.046** | | ^ ter* |0.106/0.387/**0.167** |0.025/0.191/**0.044** |0.132/0.936/**0.232** |0.026/0.170/**0.046** | |
| ^ meteor |0.092/0.251/**0.135** |0.047/0.229/**0.078** |0.248/0.665/**0.361** |0.020/0.382/**0.038** | |
^ hmm |0.162/0.426/**0.234** |0.069/0.309/**0.112** |0.281/0.793/**0.415** |0.025/0.400/**0.047** | | ^ hmm |0.162/0.426/**0.234** |0.069/0.309/**0.112** |0.281/0.793/**0.415** |0.025/0.400/**0.047** | |
^ lcs |0.168/0.462/**0.247** |0.000/0.000/**0.000** |0.293/0.848/**0.435** |0.026/0.374/**0.049** | | ^ lcs |0.168/0.462/**0.247** |0.000/0.000/**0.000** |0.293/0.848/**0.435** |0.026/0.374/**0.049** | |
^ gizadiag* |0.183/0.512/**0.270** |0.044/0.250/**0.075** |0.285/0.784/**0.417** |0.038/0.224/**0.065** | | |
^ gizainter |0.170/0.483/**0.252** |0.049/0.137/**0.072** |0.284/0.878/**0.429** |0.029/0.409/**0.054** | | ^ gizainter |0.170/0.483/**0.252** |0.049/0.137/**0.072** |0.284/0.878/**0.429** |0.029/0.409/**0.054** | |
| ^ gizadiag* |0.183/0.512/**0.270** |0.044/0.250/**0.075** |0.285/0.784/**0.417** |0.038/0.224/**0.065** | |
^ berkeley* |0.200/0.540/**0.291** |0.050/0.330/**0.087** |0.292/0.844/**0.434** |0.039/0.267/**0.068** | | ^ berkeley* |0.200/0.540/**0.291** |0.050/0.330/**0.087** |0.292/0.844/**0.434** |0.039/0.267/**0.068** | |
| |
=== Explicit wrong lex choice detection === | === Explicit wrong lex choice detection === |
| * "Dan's method" |
* align input+czeng to reference+czeng and input+czeng to hypotheses+czeng | * align input+czeng to reference+czeng and input+czeng to hypotheses+czeng |
* extract hypothesis-to-reference alignments from there | * extract hypothesis-to-reference alignments from there |
| ^ Precision/Recall/F-score: ^^^^ | | ^ Precision/Recall/F-score: ^^^^ |
| ^ Lex ^ Order ^ Punct ^ Miss ^ | | ^ Lex ^ Order ^ Punct ^ Miss ^ |
^ meteor+hmm |0.162/0.426/**0.234** |0.068/0.309/**0.112** |0.286/0.794/**0.421** |0.025/0.400/**0.047** | | |
^ ter+hmm |0.116/0.402/**0.180** |0.030/0.184/**0.051** |0.145/0.912/**0.251** |0.026/0.181/**0.046** | | ^ ter+hmm |0.116/0.402/**0.180** |0.030/0.184/**0.051** |0.145/0.912/**0.251** |0.026/0.181/**0.046** | |
| ^ meteor+hmm |0.162/0.426/**0.234** |0.068/0.309/**0.112** |0.286/0.794/**0.421** |0.025/0.400/**0.047** | |
^ gizadiag+hmm |0.186/0.515/**0.273** |0.040/0.215/**0.067** |0.297/0.836/**0.438** |0.039/0.238/**0.067** | | ^ gizadiag+hmm |0.186/0.515/**0.273** |0.040/0.215/**0.067** |0.297/0.836/**0.438** |0.039/0.238/**0.067** | |
^ gizainter+hmm |0.194/0.505/**0.281** |0.062/0.282/**0.101** |0.299/0.806/**0.436** |0.033/0.382/**0.061** | | ^ gizainter+hmm |0.194/0.505/**0.281** |0.062/0.282/**0.101** |0.299/0.806/**0.436** |0.033/0.382/**0.061** | |
^ czenginter+hmm |0.214/0.545/**0.307** |0.093/0.525/**0.158** |0.304/0.818/**0.443** |0.038/0.363/**0.068** | | ^ czenginter+hmm |0.214/0.545/**0.307** |0.093/0.525/**0.158** |0.304/0.818/**0.443** |0.038/0.363/**0.068** | |
| ||||| | | ||||| |
^ berkeley+czenginter+hmm |0.219/0.568/**0.316** |0.070/0.432/**0.120** |0.298/0.817/**0.436**|0.048/0.290/**0.082** | | ^ berk+czengint+hmm |0.219/0.568/**0.316** |0.070/0.432/**0.120** |0.298/0.817/**0.436**|0.048/0.290/**0.082** | |
^ berkeley+czenginter+gizainter+hmm |0.220/0.569/**0.317** |0.068/0.420/**0.118** |0.298/0.812/**0.436**|0.048/0.290/**0.083** | | ^ berk+czengint+gizaint+hmm |0.220/0.569/**0.317** |0.068/0.420/**0.118** |0.298/0.812/**0.436**|0.048/0.290/**0.083** | |
^ berkeley+czenginter+meteor+hmm |0.220/0.569/**0.317** |0.070/0.440/**0.121** |0.295/0.810/**0.433**|0.048/0.290/**0.083** | | ^ berk+czengint+meteor+hmm |0.220/0.569/**0.317** |0.070/0.440/**0.121** |0.295/0.810/**0.433**|0.048/0.290/**0.083** | |
| ^ berk+czengint+meteor+gizaint+hmm |0.221/0.571/**0.318** |0.068/0.424/**0.118** |0.298/0.808/**0.436** |0.049/0.292/**0.084** | |
| |
==== TODOs ==== | ==== TODOs ==== |
* try domain adaptation for word alignment, EMNLP 2011 paper | * test alignment with synonym detection (cz_wn required) = separating ''lex::'' and ''disam::'' |
* test alignment with synonym detection (cz_wn required) = separating @@lex@@ and @@disam@@ | |
* order evaluation | * order evaluation |
* a lot of background research | * a lot of background research |
* if it (and only it) were to be moved in the original permutation, what would be the best place? | * if it (and only it) were to be moved in the original permutation, what would be the best place? |
* evaluate with nr. of intersections | * evaluate with nr. of intersections |
| * try domain adaptation for word alignment, EMNLP 2011 paper |
* comb and comment the code | * comb and comment the code |
* add help files | * add help files |
* integrate with the rest of Addicter | * integrate with the rest of Addicter |
* learner's corpus | * approach applicable to learner's corpora |
* see Anne Lüdelig, TLT9 | * see Anne Lüdelig, TLT9 |
* adapt to Sara's program | * adapt Addicter to Sara's program |
* alternative to reference-based evaluation: "Inconsistencies in Penn parsing", M. Dickinson | * alternative to reference-based evaluation: "Inconsistencies in Penn parsing", M. Dickinson |
| |