Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
spark:recipes:using-perl-via-pipes [2014/11/07 11:12] straka |
spark:recipes:using-perl-via-pipes [2014/11/07 14:11] straka |
||
---|---|---|---|
Line 43: | Line 43: | ||
</ | </ | ||
- | ==== Complete Example using Simple Perl Tokenizer ==== | + | ==== Complete Example using Simple Perl Tokenizer |
Suppose we want to write program which uses Perl Tokenizer and then produces token counts. | Suppose we want to write program which uses Perl Tokenizer and then produces token counts. | ||
Line 84: | Line 84: | ||
sc = SparkContext() | sc = SparkContext() | ||
(sc.textFile(input) | (sc.textFile(input) | ||
- | | + | |
| | ||
| | ||
Line 111: | Line 111: | ||
// let rdd be an RDD we want to process, creating '' | // let rdd be an RDD we want to process, creating '' | ||
- | rdd.map(encodeJson).pipe(" | + | rdd.map(encodeJson).pipe(" |
</ | </ | ||
+ |