[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]

You are here: start » courses » rg » 2012 » distributed-perceptron

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
courses:rg:2012:distributed-perceptron [2012/12/16 16:25]
machacek 		courses:rg:2012:distributed-perceptron [2012/12/16 21:59]
machacek
Line 1: Line 1:
 ====== Distributed Training Strategies for the Structured Perceptron - RG report ====== ====== Distributed Training Strategies for the Structured Perceptron - RG report ======
 +
 +===== Presentation =====
 +
 +==== 3 Structured Perceptron ====
 +
 +  * In unstructured perceptron, you are trying to separate two sets with hyperplane. See Question 1 for the algorithm. In training phase, you iterate your training data and adjust the hyperplane every time you make a mistake. [[http://www.youtube.com/watch?v=vGwemZhPlsA|Youtube Example]]
 +
 +  * Structured (or multiclass) perceptron is generalization of the unstructured perceptron. See figure 1 in the paper for the algorithm.
 +
 +==== 4 Distributed Structured Perceptron ====
 +
 +  * Motivation: There is no straightforward way to make the standard perceptron algorithm parallel. 
 +
 +=== 4.1 Parameter Mixing ===
 +
 +=== 4.2 Iterative Parameter Mixing ===
 +
 +==== 5 Experiments ====
 +
 +
  
 ===== Questions ===== ===== Questions =====
Line 38: Line 58:
  
 w = [0, 0.6] w = [0, 0.6]
 +
 +  
  
 ==== Question 2 ==== ==== Question 2 ====
Line 45: Line 67:
 w = [?, ?] w = [?, ?]
  
-Acording to English [[http://en.wikipedia.org/wiki/Perceptron#Multiclass_perceptron|Wikipedia]]: This multiclass formulation reduces to the original perceptron when **x** is a real-valued vector, y is chosen from {0,1}, and f(x,y) = y * **x**.+**Answer 1:** 
 + 
 +f(**x**,y) = (y == 0) ? (**x**, 0,0,...,0) : (0,0,...,0,**x**) 
 + 
 +**Answer 2:** 
 + 
 +Acording to English [[http://en.wikipedia.org/wiki/Perceptron#Multiclass_perceptron|Wikipedia]]: This multiclass formulation reduces to the original perceptron when **x** is a real-valued vector, y is chosen from {0,1}, and f(**x**,y) = y * **x**.
  
 However, I would say, that this holds only for activation treshold = 0. Therefore, this formula cannot be used to compute example from Question 1. However, I would say, that this holds only for activation treshold = 0. Therefore, this formula cannot be used to compute example from Question 1.
 +
  
 ==== Question 3 ==== ==== Question 3 ====
 In figure 4, why do you think that the F-measure for Regular Perceptron (first column) learned by the Serial (All Data) algorithm is worse than the Parallel (Iterative Parametere Mix)? In figure 4, why do you think that the F-measure for Regular Perceptron (first column) learned by the Serial (All Data) algorithm is worse than the Parallel (Iterative Parametere Mix)?
 +
 +
 +**Answer:**
 +
 +  * Iterative Parameter Mixing is just a form of parameter averaging, which has the same effect as the averaged perceptron.
 +    * F-measures for seral (All Data) and Paralel (Iterative Parameter Mix) are very similar in the second column. It is because the both methods are already averaged.
 +  * Bagging like effect
  
 ==== Question 4 ==== ==== Question 4 ====
Line 65: Line 101:
 N = argmax_N f(N, T, F, ...) N = argmax_N f(N, T, F, ...)
 f = ? f = ?
 +
 +**Answer:**
 +
 +We have not concluded on a particular formula.
 +  * It also depends on convergence criteria. 
 +  * With no time limitation, the serial algorithm would have the least energy consumption. 
 +  * With time limitation, we should use as least shards to meet the time limitation. 

[ Back to the navigation ] [ Back to the content ]