[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
spark:using-python [2019/11/12 13:19]
straka
spark:using-python [2019/11/12 13:37] (current)
straka
Line 20: Line 20:
    ​.map(lambda word: (word, 1))    ​.map(lambda word: (word, 1))
    ​.reduceByKey(lambda c1,c2: c1+c2)    ​.reduceByKey(lambda c1,c2: c1+c2)
-   ​.sortBy(lambda ​(word,​count)count, ascending=False)+   ​.sortBy(lambda ​word_countword_count[1], ascending=False)
    ​.take(10))    ​.take(10))
 </​file>​ </​file>​
Line 60: Line 60:
    ​.map(lambda token: (token, 1))    ​.map(lambda token: (token, 1))
    ​.reduceByKey(lambda x,y: x + y)    ​.reduceByKey(lambda x,y: x + y)
-   ​.sortBy(lambda ​(word,​count)count, ascending=False)+   ​.sortBy(lambda ​word_countword_count[1], ascending=False)
    ​.saveAsTextFile(output))    ​.saveAsTextFile(output))
 sc.stop() sc.stop()

[ Back to the navigation ] [ Back to the content ]