Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
khresmoi:start [2012/01/23 14:55] hlavacova |
khresmoi:start [2012/02/28 15:01] hlavacova |
||
---|---|---|---|
Line 12: | Line 12: | ||
* **JD** | * **JD** | ||
* **JB** | * **JB** | ||
+ | * **ZU** | ||
---- | ---- | ||
+ | ==== Poznámky k datům ==== | ||
+ | Píšu to rovnou sem nahoru, aby to bylo na ráně. | ||
+ | Data z **JRC-Aquis** se během posledních několika týdnů změnila, takže nejsou kompatibilní se skriptem, který je k nim dodáván, ten se nezměnil. Měla jsem zálohovaná data z páru fr-en, tak jsem je zpracovala, jsou na svém místě. V adresáři original jsou i ostatní data, tj. pro páry de-en a cs-en, ale nejsou zpracovaná, | ||
+ | Zpracování fr-en jsem provedla " | ||
+ | Z porovnání obou paralelních fr-en dat usuzuji, že na ně byl použit jiný aligner. Lepší? | ||
+ | |||
+ | |||
+ | |||
==== Data ==== | ==== Data ==== | ||
- | === MT training data available for KHRESMOI - actual situation | + | === MT training data available for KHRESMOI |
+ | |||
+ | ^ Corpus ^ Source ^ Domain ^ EN-FR ^ EN-DE ^ alignment | ||
+ | | TDA translation memory | ||
+ | | CESTA Evaluation Package | ||
+ | | EQueR Evaluation Package | ||
+ | | CESART Evaluation Package | ELRA | in | | | | | 9000 Kw | | DONE | | ||
+ | | French Gigaword | LDC | news | | | | | 863 Kw | | 8-) DVD | | ||
+ | | Acquis | JRC | law | 1,25 Ms | 1,33 Ms | sent | | | | ||
+ | | EMEA | European Medicines Agency | in | 373 Ks | 12 Mw | | 26.34 Mw | 14.9Mw | | ||
+ | | MESH | U.S. National Library of Medicine | in | | | | 838 kw | | ||
+ | | OrphaNet | ||
+ | | Europarl | WMT12 | parl | 1.8Ms | 1.7Ms | sent | | | | DONE | | ||
+ | | News Commentary | ||
+ | | News monolingual | WMT12 | news | | | | 181kw | 147kw | 162kw | DONE | | ||
+ | | United Nations | WMT12 | un | 12.3Ms | | | | | | DONE | | ||
+ | | French-English 109 corpus | WMT12 | web | 22.5Ms | | sent | | | | DONE | | ||
+ | | Medpedia wiki| Medpedia | in | | ||
+ | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | in | 24, | ||
+ | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | in | 33,5Mw | | par | | ||
+ | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | tech | 153, | ||
+ | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | tech | 178,8Mw | | par | | ||
+ | | MAREC | Wien TU | in | ? | ? | ? | | | ||
+ | | Springer Bilingual Corpus | much.more | in | | 1.09 Mw | sent | | | ||
+ | | Europarl3 | OPUS | | | 1.3 Ms | sent | | | ||
+ | | OpenSubtitles2011 | OPUS | | | 5 Ms | sent | | | ||
+ | |||
+ | |||
+ | k, M ... thousand, milion | ||
+ | w, s, f ... words, sentences, files (for parallel data only source (English) words are counted) | ||
+ | * viz podrobnější info v podsekcích | ||
+ | |||
+ | === Zdroje | ||
+ | __JRC Acquis__ | ||
+ | http:// | ||
+ | |||
+ | __MAREC__ | ||
+ | A61 (MEDICAL OR VETERINARY SCIENCE; HYGIENE): 1.589,849 files | ||
+ | Nevím, kolik slov, není to v jednolitém balíku. | ||
+ | Na žádost o přístup odpověděli: | ||
+ | //the IRF is not granting access to the MAREC collection anymore. However, the access for research purposes should be possible in a foreseeable future via the Vienna University of Technology - Allan will certainly come back to you when the legal status is cleared.// | ||
- | ^ Corpus ^ Source ^ Domain ^ EN-FR ^ EN-DE ^ EN ^ FR ^ DE ^ Note ^ | ||
- | | TMX | TDA | in | 13517 Kw | 6797 Kw | | | | downloaded | ||
- | | CESTA Evaluation Package | ||
- | | EQueR Evaluation Package | ||
- | | CESART Evaluation Package | ELRA | in | | | | 9000 Kw | | waiting | ||
+ | __Coppa__ | ||
+ | IPC: A61, C12N, C12P ... medical patents (doporučeno od WIPO) | ||
+ | Patenty členěny podle roku, ve dvou verzích: | ||
+ | * segmentované podle vět, ale menší - viz tabulka. Některé patenty chybí zcela, některé jsou zkráceny. | ||
+ | * nesegmentované - každý patent má 2 záznamy: jméno a abstract, obojí v EN i FR, tedy alignment podle paragrafu (odhad) | ||
- | === Zdroje (dle PP) === | ||
__Khresmoi wiki__ | __Khresmoi wiki__ | ||
http:// | http:// | ||
Line 32: | Line 80: | ||
__www stranka WMT workshopu__ | __www stranka WMT workshopu__ | ||
http:// | http:// | ||
+ | http:// | ||
__korpus OPUS__ | __korpus OPUS__ | ||
Line 63: | Line 112: | ||
---- | ---- | ||
- | ==== SVN ==== | + | ==== Úložiště |
- | Prosím PP o doplnění | + | / |
---- | ---- |