[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

This is an old revision of the document!


Table of Contents

TectoMT Tutorial

Welcome at TectoMT Tutorial. This tutorial should take about 2 hours.

What is TectoMT

TectoMT is a highly modular NLP (Natural Language Processing) software system implemented in Perl programming language under Linux. It is primarily aimed at Machine Translation, making use of the ideas and technology created during the Prague Dependency Treebank project. At the same time, it is also hoped to significantly facilitate and accelerate development of software solutions of many other NLP tasks, especially due to re-usability of the numerous integrated processing modules (called blocks), which are equipped with uniform object-oriented interfaces.

Prerequisities

In this tutorial, we assume that TectoMT has been successfully installed on your machine. For installation details, see Getting started in TectoMT Developer's Guide.

Before running any experiments with TectoMT, you must set up your environment by running

source devel/config/init_shell_environ.sh

Layers of linguistic structures

TrEd

Architecture

In TectoMT, there is the following hierarchy of processing units (software components that process data):

First application

Once you have TectoMT installed on your machine, you can find this tutorial in devel/applications/tutorial/. After you cd in to this directory, you can see our plain text sample data in sample.txt.

Most applications are defined in Makefiles, which describe sequence of blocks to be applied on our data. In our particular Makefile, four blocks are going to be applied on our sample text: sentence segmentation, tokenization, tagging and lemmatization. Since we have our input text in plain text format, the file is going to be converted into tmt format beforehand (the in section).

We can run the application:

make all

Our plain text data sample.txt have been transformed into tmt, internal TectoMT format, and saved into sample.tmt. Then, all four blocks have been loaded and our data has been processed. We can now examine sample.tmt using a regular text editor. We'll now stop and describe data structure in TectoMT.

Changing the scenario

We'll now add syntax analysis to our scenario by adding four more blocks. Instead of

analyze:
        eval ${BRUNBLOCKS} -S -o \
                SEnglishW_to_SEnglishM::Sentence_segmentation_simple \
                SEnglishW_to_SEnglishM::Penn_style_tokenization \
                SEnglishW_to_SEnglishM::TagTnT \
                SEnglishW_to_SEnglishM::Lemmatize_mtree -- sample.tmt

we'll have:

analyze:
        eval ${BRUNBLOCKS} -S -o \
                SEnglishW_to_SEnglishM::Sentence_segmentation_simple \
                SEnglishW_to_SEnglishM::Penn_style_tokenization \
                SEnglishW_to_SEnglishM::TagTnT \
                SEnglishW_to_SEnglishM::Lemmatize_mtree  \
                SEnglishM_to_SEnglishA::McD_parser_local \
                SEnglishM_to_SEnglishA::Fix_McD_Tree \
                SEnglishM_to_SEnglishA::Fill_afun_after_McD -- sample.tmt

Note: Makefiles use tabulators to mark command lines. Make sure your lines start with tabulator (or two tabulators) and not, for example, with 4 spaces.

After running

make all

we can examine our sample.tmt again. Really, an analytical layer SEnglishA describing a dependency tree with analytical functions (<afun>) has been added to each bundle.

Adding a new block

The linguistic structures in TectoMT are represented using the following object-oriented interface/types:

We'll now examine an example of a new block in file devel/applications/tutorial/Tutorial.pm

This block illustrates the most common methods for accessing objects:

Attributes of documents, bundles or nodes can be accessed by attribute getters and setters, for example:

Our tutorial block Tutorial.pm is ready to use. We'll copy it to devel/libs/blocks/SEnglishA_to_SEnglishA/ where TectoMT will expect this block:

cp devel/applications/Tutorial.pm devel/libs/blocks/SEnglishA_to_SEnglishA/Tutorial.pm

We also have to add this block to our scenario which can be done by adding new Makefile target:

printfun:
        eval ${BRUNBLOCKS} -S -o \
                SEnglishA_to_SEnglishA::Tutorial -- sample.tmt

And we'll observe our new block's behaviour:

make printfun

More advanced block

In this application, we'll split the text into finite clauses.

Further information


[ Back to the navigation ] [ Back to the content ]