Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
draft [2009/07/15 12:37] ptacek |
draft [2009/09/01 00:10] ufal |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Progress Report ====== | ====== Progress Report ====== | ||
+ | [[Progress Report]] - dal jsem to na zvlastni stranku, abysme si nelezli do zeli | ||
- | Hi Marc, | ||
- | ... | + | [[http://72.55.153.148/ |
- | Re: progress: there is progress in the following: | + | ====== Description of Czech Companion November Demonstrator ====== |
- | - speech re-training for the collected dialogue | + | The Czech version of the Companion deals with the Reminiscing about User's Photos scenario, taking advantage of data recorded in first phase of the project. The basic architecture |
- | - additional dialogue transription for ASR is ongoing (WP52.? T5.2.1) | + | |
- | - DM has been transferred from USFD to Prague | + | |
- | being extensively tested | + | |
- | - DAF editor transfer | + | |
- | - Sample dialogues (specifically aimed at the demo) | + | |
- | are ready - issues are being resolved between CU/ZCU | + | |
- | - DAFs are being prepared for the SC-CZ scenario AND | + | |
- | the sample dialogues | + | |
- | - DA set is being prepared, also based on the sample dialogues (WP5.2) | + | |
- | - preliminary DA tagger working (WP5.2) | + | |
- | - integration work is ongoing (CU/ZCU, internally at CU) | + | |
- | but no functioning full demo yet | + | |
- | I hope this is OK for the progress report. Pavel (I.) might add more specifics regarding | + | The dialog |
- | Best, | + | Our DAFs covering selected topics contain not only Companion replies mined from the corpora, but also new human-authored assessments, |
- | -- Jan | + | For a sample dialogue, see the Scenario Brief below. |
- | ====== Description of Czech Companion November Prototype ====== | + | {{user: |
- | The Czech version of Companion deals with the Reminiscing about User's Photos scenario. | ||
- | photopal domena, nahranej korpus, ze na to sou dafy (reusing SHEFF DM intergrated through Inamode Relayer (TID)) vhodny, moreover reusable for expected pomdp DM from UOX (reuse states, let pomdp' | ||
- | typy odpovedi a zpusob jejich implementace, | ||
- | NLP server s tectomt, ASR/TTS/SR client, connected over network | ||
- | XXX JPta | ||
- | advances in Czech NLU (on reconstructed spoken data): 300-500vet(? | + | ===== Automatic Speech Recognition |
- | pos ? analyzovat, generovat a kontrolovat ' | + | features: improved language models, real-time speaker adaptation |
+ | performance indicator: WER | ||
- | ===== Speech Reconstruction ===== | ||
- | features: omit filler phrases, irrelevant speech events, false starts, repetitions, | ||
- | imlementation(zahrnout tuhle info?): moses natrenovany na korpusu | ||
- | performance indicator: BLEU score (overall scoring of all features) to annotated corpora from T5.2.1., nejaka baseline | ||
- | XXX Mirek | ||
- | ===== Morphology Analyzer and POS tagging ===== | ||
- | features: XXX Mirek/ | ||
- | performance indicator: accuracy | ||
- | ===== Syntactic Parsing ===== | + | |
+ | ===== Speech Reconstruction (WP 5.2) ===== | ||
+ | features: omit filler phrases, remove irrelevant speech events, handle false starts, repetitions, | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ===== Morphology Analyzer and POS tagging (WP 5.2) ===== | ||
+ | features: coverage of photo-pal domain, domain adapted tagger | ||
+ | performance indicator: OOV rate, accuracy (Morce 95.1%) | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ===== Syntactic Parsing | ||
features: induce dependencies and labels | features: induce dependencies and labels | ||
- | performance indicator: | + | performance indicator: |
- | v tipu je natrenovat MacDonnalda na dialog datech, ten task je do M42, ted ne. | + | |
- | ===== Semantic Parsing ===== | ||
- | features: meaning representation with semantic roles (69 labels), coordinations, | ||
- | performance indicator: f-measure | ||
- | ===== Information Extraction ===== | + | |
+ | |||
+ | ===== Semantic Parsing (WP 5.2) ===== | ||
+ | features: assignment of semantic roles (69 roles), coordinations, | ||
+ | performance indicator: accuracy (correctly induced edges, labels) | ||
+ | |||
+ | |||
+ | ===== Information Extraction | ||
features: template based identification of predicates | features: template based identification of predicates | ||
covering predicates from before-mentioned set of DAFs. | covering predicates from before-mentioned set of DAFs. | ||
performance indicator: accuracy | performance indicator: accuracy | ||
- | ===== Named Entities Recognition ===== | + | |
- | features: detect person names, geographical locations | + | |
+ | ===== Named Entities Recognition | ||
+ | features: detect person names, geographical locations, organization names | ||
performance indicator: f-measure | performance indicator: f-measure | ||
- | ===== Dialog Act Tagging ===== | + | |
- | features: | + | |
+ | |||
+ | ===== Dialog Act Tagging | ||
+ | features: | ||
performance indicator: accuracy | performance indicator: accuracy | ||
- | ===== Sentiment Analysis ===== | ||
- | features: za tohle bych vydaval klasifikator, | ||
- | performance indicator: f-measure | ||
- | ===== Complete System Evaluation ===== | ||
- | T5.2.7 tohle zminuje, nick webb to pro nas asi neudela | ||
- | performance indicator: pocet slov ve vypovedich uzivatele(? | ||
- | ===== Dialog Manager ===== | ||
- | features: reply types, using (language independed) predicates (prakticky to znamena, ze pojmenuju testy na prechodech v dafech anglicky) | ||
- | performance indicator: rucni hodnoceni prijatelnosti vybrane akce | ||
- | ===== Natural Language Generation ===== | + | ===== Dialog Manager (WP 5.3) ===== |
- | features: | + | features: integrated DAF-based dialog manager from previous English prototype, |
+ | manual creation of DAFs covering following topics: Person_retired, | ||
+ | performance indicator: acceptability - manual evaluation of actions selected by DM | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ===== Natural Language Generation | ||
+ | features: | ||
performance indicator: BLEU score | performance indicator: BLEU score | ||
+ | |||
+ | |||
+ | |||
+ | ===== Emotional TTS (WP 5.5) ===== | ||
+ | features: emotions will be expressed implicitly, through the usage of communicative functions; new female voice database was recorded for this purposes | ||
+ | performance indicator: listening tests | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ===== Emotional Avatar Integration (WP 5.5) ===== | ||
+ | features: new Czech female voice with affective features will be integrated with the TID avatar | ||
+ | performance indicator: subjective evaluation of the naturalness and the ability to convey emotions (small-scale, | ||
+ | |||
+ | |||
+ | |||
+ | ====== Scenario Brief ====== | ||
+ | |||
+ | C1 Dobrý den, jak se jmenujete? (intro-daf-based) | ||
+ | H1 Marie. | ||
+ | |||
+ | C2 Těší mě. Já jsem Pavla a chtěla bych Vás blíž poznat. Jsem ráda, že mi ukážete Vaše fotografie. | ||
+ | |||
+ | C2 Co je tohle? (photo_maintenance) | ||
+ | H2 To je moje rodina, rodiče, sestra, její manžel, moje dvě děti a její dvě děti. | ||
+ | H2 Byli u mě na Vánoce na návštěvě. | ||
+ | |||
+ | C3 Trávíte Vánoce takhle pohromadě? | ||
+ | H3 Teď už ne. | ||
+ | H3 Teď už všechny moje dětičky vylétaly z hnízda. | ||
+ | |||
+ | C4 Jak se jmenuje váš manžel? | ||
+ | H4 Manžel už nežije, jmenoval se Karel Šístek. | ||
+ | |||
+ | C5 Máte ráda Vánoce? | ||
+ | H5 Teď už ne. | ||
+ | H5 Měla jsem je ráda, když byly děti malé. | ||
+ | H5 Teď už to ale není ono. | ||
+ | |||
+ | C6 Takže teď trávíte Vánoce sama? (ask_on_detail-daf-based) | ||
+ | H6 Nejsem úplně sama, chodím každé Vánoce k dceři, ale už to není, jak to bývalo, když byly doma malé děti. | ||
+ | |||
+ | C7 Jak děti vyrostou, tak má člověk najednou příliš mnoho času. | ||
+ | C7 Kdo je kdo na té fotce? | ||
+ | H7 U stromečku je můj synovec, ta holčička vedle je moje dcera, chlapeček s hračkou je můj syn, vedle je sestra, její manžel a nad ním je můj otec. | ||
+ | |||
+ | C8 To jste ale velká rodina. | ||
+ | C8 Jak se jmenuje vaše sestra? (companion commenting last utterance | ||
+ | H8 Sestra se jmenuje Marie Havlová, její manžel je Vladimír, ten bohužel nedávno zemřel. | ||
+ | |||
+ | C9 To je smutné. | ||
+ | C9 Znala jste ho dobře? | ||
+ | H9 Velmi dobře, trávili jsme obě rodiny společně hodně času. | ||
+ | |||
+ | C10 Máte už nějaká vnoučata? | ||
+ | H10 Mám vnučku Natálku. | ||
+ | H10 Ona je takové živé děťátko, taková neposeda. | ||
+ | |||
+ | C11 Takže Natálka je neposeda. repeat-rephrase | ||
+ | C11 Máte ještě nějaké další vnučky nebo vnuky? | ||
+ | H11 Čekáme ještě další přírustek. | ||
+ | |||
+ | C12 Hlídáte je někdy? | ||
+ | H12 Hlídám dost často. | ||
+ | H12 Já mám děti ráda a jsem s nimi moc ráda. | ||
+ | |||
+ | C13 To je fajn. | ||
+ | C13 S dětmi je legrace. | ||
+ | H13 Ano. | ||
+ | |||
+ | C Podíváme se na další fotku? | ||
+ | |||
+ |