Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision Next revision Both sides next revision | ||
spark:spark-introduction [2014/10/03 10:22] straka created |
spark:spark-introduction [2014/10/03 12:56] straka |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Spark Introduction ====== | ====== Spark Introduction ====== | ||
- | ===== Spark Introduction | + | This introduction shows several simple examples to give you an idea what programming |
- | ===== Spark Introduction | + | ===== Running |
+ | To run interactive Python shell in local Spark mode, run (on your local workstation or on cluster) | ||
+ | IPYSPARK=1 pyspark | ||
+ | The IPYSPARK=1 parameter instructs Spark to use '' | ||
+ | |||
+ | After a local Spark executor is started, the Python shell starts. | ||
+ | 14/10/03 10:54:35 INFO SparkUI: Started SparkUI at http:// | ||
+ | |||
+ | ==== Running Spark Shell in Scala ==== | ||
+ | |||
+ | To run interactive Scala shell in local Spark mode, run (on your local workstation or on cluster) | ||
+ | scala-shell | ||
+ | Once again, the SparkUI address is listed several lines above the shell prompt line. | ||
+ | |||
+ | |||
+ | ===== Word Count Example ===== | ||
+ | The central object of Spark is RDD -- resilient distributed dataset. It contains ordered sequence of items. |