Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
khresmoi:start [2012/02/28 15:00] hlavacova |
khresmoi:start [2012/06/14 12:09] hlavacova |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| Medical Information Analysis & Retrieval | Medical Information Analysis & Retrieval | ||
| http:// | http:// | ||
| - | |||
| Line 15: | Line 14: | ||
| ---- | ---- | ||
| - | ==== Poznámky k datům ==== | ||
| - | Píšu to rovnou sem nahoru, aby to bylo na ráně. | ||
| - | Data z **JRC-Aquis** se během posledních několika týdnů změnila, takže nejsou kompatibilní se skriptem, který je k nim dodáván, ten se nezměnil. Měla jsem zálohovaná data z páru fr-en, tak jsem je zpracovala, jsou na svém místě. V adresáři original jsou i ostatní data, tj. pro páry de-en a cs-en, ale nejsou zpracovaná, | ||
| - | Zpracování fr-en jsem provedla " | ||
| - | Z porovnání obou paralelních fr-en dat usuzuji, že na ně byl použit jiný aligner. Lepší? | ||
| - | |||
| ==== Data ==== | ==== Data ==== | ||
| + | Jsou zde **/ | ||
| === MT training data available for KHRESMOI === | === MT training data available for KHRESMOI === | ||
| + | Kliknutím na korpus (první sloupec) se dostanete do sekce [[khresmoi: | ||
| + | ---- | ||
| ^ Corpus ^ Source ^ Domain ^ EN-FR ^ EN-DE ^ alignment | ^ Corpus ^ Source ^ Domain ^ EN-FR ^ EN-DE ^ alignment | ||
| - | | TDA translation memory | + | | [[khresmoi: |
| - | | CESTA Evaluation Package | + | | [[khresmoi: |
| - | | EQueR Evaluation Package | + | | [[khresmoi: |
| - | | CESART Evaluation Package | ELRA | in | | | | | 9000 Kw | | DONE | | + | | [[khresmoi: |
| - | | French Gigaword | LDC | news | | | | | 863 Kw | | | + | | [[khresmoi: |
| - | | Acquis | JRC | law | 1,25 Ms | 1,33 Ms | sent | | | | + | | [[khresmoi: |
| - | | EMEA | European Medicines Agency | in | 373 Ks | 12 Mw | | 26.34 Mw | 14.9Mw | | + | | [[khresmoi: |
| - | | MESH | U.S. National Library of Medicine | in | | | | 838 kw | | + | | [[khresmoi: |
| - | | OrphaNet | + | | [[khresmoi: |
| - | | Europarl | WMT12 | parl | 1.8Ms | 1.7Ms | sent | | | | DONE | | + | | [[khresmoi: |
| - | | News Commentary | + | | News Commentary |
| - | | News monolingual | WMT12 | news | | | | 181kw | 147kw | 162kw | DONE | | + | | News monolingual | WMT12 | news | | | | 181kw | 147kw | 162kw | DONE: |
| - | | United Nations | WMT12 | un | 12.3Ms | | | | | | DONE | | + | | United Nations | WMT12 | un | 12.3Ms | | | | | | DONE: |
| - | | French-English 109 corpus | WMT12 | web | 22.5Ms | | sent | | | | DONE | | + | | French-English 109 corpus | WMT12 | web | 22.5Ms | | sent | | | | DONE:giga | |
| | Medpedia wiki| Medpedia | in | | | Medpedia wiki| Medpedia | in | | ||
| - | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | in | 24,8Mw = 1,2Ms | | sent | | + | | [[khresmoi: |
| - | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | in | 33,5Mw | | par | | + | | [[khresmoi: |
| - | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | tech | 153,8Mw = 7,5Ms | | sent | | + | | [[khresmoi: |
| - | | Corpus Of Parallel Patent Applications (Coppa)| WIPO | tech | 178,8Mw | | par | | + | | [[khresmoi: |
| - | | MAREC | Wien TU | in | ? | ? | ? | | | + | | [[khresmoi: |
| - | | Springer Bilingual Corpus | much.more | in | | | + | | [[khresmoi: |
| | Europarl3 | OPUS | | | 1.3 Ms | sent | | | | Europarl3 | OPUS | | | 1.3 Ms | sent | | | ||
| | OpenSubtitles2011 | OPUS | | | 5 Ms | sent | | | | OpenSubtitles2011 | OPUS | | | 5 Ms | sent | | | ||
| + | | Czeng | UFAL | | | | ||
| + | | Drugbank | drugbank.ca | in | | | | ||
| + | | [[http:// | ||
| + | | PIL | The Patient Information Leaflet Corpus | in | | | | ||
| + | | [[www.nactem.ac.uk/ | ||
| + | | [[www.nactem.ac.uk/ | ||
| + | | [[https:// | ||
| + | | [[khresmoi: | ||
| + | | [[http:// | ||
| + | |||
| + | |||
| + | == Vysvětlivky == | ||
| k, M ... thousand, milion | k, M ... thousand, milion | ||
| w, s, f ... words, sentences, files (for parallel data only source (English) words are counted) | w, s, f ... words, sentences, files (for parallel data only source (English) words are counted) | ||
| * viz podrobnější info v podsekcích | * viz podrobnější info v podsekcích | ||
| + | Sloupec Note obsahuje název podadresáře / | ||
| + | 8-) stažená data, ale nezpracovaná | ||
| + | :?: nevíme, jestli chceme | ||
| + | :-? chceme stahovat, ale zatím nevíme, jak na to ... z různých příčin | ||
| + | 8-O čekáme na data | ||
| + | :!: nějaký problém, podrobněji v poznámkách - prokliknout z prvního sloupce | ||
| - | === Zdroje | ||
| - | __JRC Acquis__ | ||
| - | http:// | ||
| - | |||
| - | __MAREC__ | ||
| - | A61 (MEDICAL OR VETERINARY SCIENCE; HYGIENE): 1.589,849 files | ||
| - | Nevím, kolik slov, není to v jednolitém balíku. | ||
| - | Na žádost o přístup odpověděli: | ||
| - | //the IRF is not granting access to the MAREC collection anymore. However, the access for research purposes should be possible in a foreseeable future via the Vienna University of Technology - Allan will certainly come back to you when the legal status is cleared.// | ||
| - | |||
| - | |||
| - | __Coppa__ | ||
| - | IPC: A61, C12N, C12P ... medical patents (doporučeno od WIPO) | ||
| - | Patenty členěny podle roku, ve dvou verzích: | ||
| - | * segmentované podle vět, ale menší - viz tabulka. Některé patenty chybí zcela, některé jsou zkráceny. | ||
| - | * nesegmentované - každý patent má 2 záznamy: jméno a abstract, obojí v EN i FR, tedy alignment podle paragrafu (odhad) | ||
| + | === Další odkazy | ||
| __Khresmoi wiki__ | __Khresmoi wiki__ | ||
| http:// | http:// | ||
| Line 85: | Line 83: | ||
| http:// | http:// | ||
| - | __JRC Acquis__ | + | ==== Dokumenty |
| - | http:// | + | |
| - | + | ||
| - | __ELDA__ | + | |
| - | + | ||
| - | Objednali jsme několik balíčků s in-domain daty (EN-FR, FR) | + | |
| - | + | ||
| - | __TDA__ | + | |
| - | + | ||
| - | Máme kredit na stažení 1 mld. slov. Zatím stažena EN-FR, EN-DE in-domain data. | + | |
| - | + | ||
| - | __LDC__ | + | |
| - | + | ||
| - | === Paralelní data === | + | |
| - | | + | |
| - | | + | |
| - | + | ||
| - | === Mono data === | + | |
| - | [[FR]] | + | |
| - | [[DE]] | + | |
| - | [[EN]] | + | |
| ---- | ---- | ||
| - | ==== Dokumenty ==== | ||
| - | |||
| - | ---- | ||
| - | ==== Úložiště ==== | ||
| - | / | ||
| - | ---- | ||