Both sides previous revision
Previous revision
Next revision
|
Previous revision
Next revision
Both sides next revision
|
external:tectomt:tutorial [2009/04/01 10:46] ptacek |
external:tectomt:tutorial [2010/04/14 01:30] popel použití tutorial.scen, element <bundle> není (jmenuje se <LM>) |
| |
===== TectoMT Architecture ===== | ===== TectoMT Architecture ===== |
| |
| |
| |
In TectoMT, there is the following hierarchy of processing units (software components that process data): | In TectoMT, there is the following hierarchy of processing units (software components that process data): |
| |
* The basic units are blocks. They serve for some very limited, well defined, and often linguistically interpretable tasks (e.g., tokenization, tagging, parsing). Technically, blocks are Perl classes inherited from ''TectoMT::Block'', each saved in a separate file. The blocks repository is in ''libs/blocks/''. | * The basic units are **blocks**. They serve for some very limited, well defined, and often linguistically interpretable tasks (e.g., tokenization, tagging, parsing). Technically, blocks are Perl classes inherited from ''TectoMT::Block'', each saved in a separate file. The blocks repository is in ''libs/blocks/''. |
* To solve a more complex task, selected blocks can be chained into a block sequence, called also a scenario. Technically, scenarios are instances of ''TectoMT::Scenario'' class, but in some situations (e.g. on the command line) it is sufficient to specify the scenario simply by listing block names separated by spaces. | * To solve a more complex task, selected blocks can be chained into a block sequence, called **scenario**. Scenarios are stored in ''*.scen'' files (alternatively, the block names separated by spaces can be simply listed on the command line) and at runtime the scenarios are represented by instances of ''TectoMT::Scenario'' class. |
* The highest unit is called application. Applications correspond to end-to-end tasks, be they real end-user applications (such as machine translation), or 'only' NLP-related experiments. Technically, applications are often implemented as ''Makefiles'', which only glue the components existing in TectoMT. Some demo applications can be found in ''applications''. | * The highest unit is called **application**. Applications correspond to end-to-end tasks, be they real end-user applications (such as machine translation), or 'only' NLP-related experiments. Technically, applications are often implemented as ''Makefiles'', which only glue the components existing in TectoMT. Some demo applications can be found in ''applications''. |
| |
This tutorial itself has its blocks in ''libs/blocks/Tutorial'' and the application in ''applications/tutorial''. | This tutorial itself has its blocks in ''libs/blocks/Tutorial'' and the application in ''applications/tutorial''. |
| |
There are also other directories for other purpose blocks, for example blocks which only print out some information go to ''libs/Print''. Our tutorial blocks are in ''libs/blocks/Tutorial/''. | There are also other directories for other purpose blocks, for example blocks which only print out some information go to ''libs/Print''. Our tutorial blocks are in ''libs/blocks/Tutorial/''. |
| |
| |
| |
Once you have TectoMT installed on your machine, you can find this tutorial in ''applications/tutorial/''. After you ''cd'' into this directory, you can see our plain text sample data in ''sample.txt''. | Once you have TectoMT installed on your machine, you can find this tutorial in ''applications/tutorial/''. After you ''cd'' into this directory, you can see our plain text sample data in ''sample.txt''. |
| |
Most applications are defined in ''Makefiles'', which describe sequence of blocks to be applied on our data. In our particular ''Makefile'', four blocks are going to be applied on our sample text: sentence segmentation, tokenization, tagging and lemmatization. Since we have our input text in plain text format, the file is going to be converted into ''tmt'' format beforehand (the ''in'' target in the ''Makefile''). | Most applications are defined in ''Makefiles'' and ''*.scen'' files, which describe sequence of blocks to be applied on our data. In our case, ''tutorial.scen'' lists four blocks to be applied on our sample text: sentence segmentation, tokenization, part-of-speech tagging and lemmatization. Since we have our input text in plain text format, the file is going to be converted into ''tmt'' format beforehand (the ''in'' target in the ''Makefile''). |
| |
We can run the application: | We can run the application: |
| |
* One physical ''tmt'' file corresponds to one document. | * One physical ''tmt'' file corresponds to one document. |
* A document consists of a sequence of bundles (''<bundle>''), mirroring a sequence of natural language sentences originating from the text. So, for one sentence we have one ''<bundle>''. | * A document consists of a sequence of bundles (element ''<bundles>''), mirroring a sequence of natural language sentences originating from the text. So, for each sentence we have one bundle. |
* Each bundle contains tree shaped sentence representations on various linguistic layers. In our example ''sample.tmt'' we have morphological tree (''SEnglishM'') in each bundle. Later on, also an analytical layer (''SEnglishA'') will appear in each bundle as we proceed with our analysis. | * Each bundle contains tree shaped sentence representations on various linguistic layers. In our example ''sample.tmt'' we have morphological tree (''SEnglishM'') in each bundle (actually, it is a flat tree: one technical root and its children are the tokens). Later on, also an analytical layer (''SEnglishA'') will appear in each bundle as we proceed with our analysis. |
* Trees are formed by nodes and edges. Attributes can be attached only to nodes. Edge's attributes must be stored as the lower node's attributes. Tree's attributes must be stored as attributes of the root node. | * Trees are formed by nodes and edges. Attributes can be attached only to nodes. Edge's attributes must be stored as the lower node's attributes. Tree's attributes must be stored as attributes of the root node. |
| |