[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]

You are here: start » user » zeman » treebanks » te

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision 		Previous revision
Next revision Both sides next revision
user:zeman:treebanks:te [2012/03/22 11:14]
zeman vytvořeno 		user:zeman:treebanks:te [2012/03/22 11:46]
zeman ICON 2009 Telugu data size.
Line 46: Line 46:
  
 ^ Part ^ Sentences ^ Chunks ^ Ratio ^ ^ Part ^ Sentences ^ Chunks ^ Ratio ^
-| Training | 980 | 6449 | 6.58 | +| Training    |  1456 |  5494 | 3.77 | 
-| Development | 150 | 811 | 5.41 | +| Development |   150 |   675 | 4.50 | 
-| Test | 150 | 961 | 6.41 | +| Test        |   150 |   583 | 3.89 | 
-| TOTAL | 1280 | 8221 | 6.42 |+| TOTAL       |  1756 |  6752 | 3.85 |
  
-The ICON 2010 version came with a data split into three parts: training, development and test:+The data distributed for ICON 2010 was slightly smaller, maybe it had been cleaned up? Note that the number of training words, 7602, is identical to the number published for ICON 2009. I cannot verify it because I only see chunks, not words in the CoNLL data format.
  
 ^ Part ^ Sentences ^ Chunks ^ Ratio ^ Words ^ Ratio ^ ^ Part ^ Sentences ^ Chunks ^ Ratio ^ Words ^ Ratio ^
-| Training | 979 | 6440 | 6.58 | 10305 | 10.52 | +| Training | 1400 | ? | ? | 7602 | 5.43 | 
-| Development | 150 | 812 | 5.41 | 1196 | 7.97 | +| Development | 150 | ? | ? | 839 | 5.59 | 
-| Test | 150 | 961 | 6.41 | 1350 | 9.00 | +| Test | 150 | ? | ? | 836 | 5.57 | 
-| TOTAL | 1279 | 8213 | 6.42 | 12851 | 10.04 |+| TOTAL | 1700 | ? | ? | 9277 | 5.46 |
  
-I have counted the sentences and chunks. The number of words comes from (Husain et al., 2010). Note that the paper gives the number of training sentences as 980 (instead of 979), which is a mistake. The last training sentence has the id 980 but there is no sentence with id 418.+We drew our training and test data from the ICON 2010 datasets but we have fewer sentences – why?
  
-Apparently the training-development-test data split was more or less identical in both years, except for the minor discrepancies (number of training sentences and development chunks).+^ Part ^ Sentences ^ Chunks ^ Ratio ^ 
 +| Training |  1300 |  5125 | 3.94 | 
 +| Test     |   150 |   597 | 3.98 | 
 +| TOTAL    |  1450 |  5722 | 3.95 |
  
 ==== Inside ==== ==== Inside ====

[ Back to the navigation ] [ Back to the content ]