ufal wiki spark https://wiki.ufal.ms.mff.cuni.cz/ 2026-04-18T18:09:33+00:00 ufal wiki https://wiki.ufal.ms.mff.cuni.cz/ https://wiki.ufal.ms.mff.cuni.cz/lib/tpl/ufal/images/favicon.ico text/html 2023-11-07T12:48:38+00:00 Anonymous (anonymous@undisclosed.example.com) spark:running-spark-on-single-machine-or-on-cluster https://wiki.ufal.ms.mff.cuni.cz/spark:running-spark-on-single-machine-or-on-cluster?rev=1699357718&do=diff Running Spark on Single Machine or on Cluster In order to use Spark, environment has to bee set up according to Using Spark in UFAL Environment. When Spark computation starts, it uses environment variable MASTER to determine the mode of computation. The following values are possible: text/html 2022-12-14T13:28:39+00:00 Anonymous (anonymous@undisclosed.example.com) spark:spark-introduction https://wiki.ufal.ms.mff.cuni.cz/spark:spark-introduction?rev=1671020919&do=diff Spark Introduction This introduction shows several simple examples to give you an idea what programming in Spark is like. See the official Quick Start or Spark Programming Guide or Python API Reference/Scala API Reference for more information. Running Spark Shell in Python To run interactive Python shell in local Spark mode, run (on your local workstation or on cluster using text/html 2022-12-14T13:25:29+00:00 Anonymous (anonymous@undisclosed.example.com) spark:using-python https://wiki.ufal.ms.mff.cuni.cz/spark:using-python?rev=1671020729&do=diff Using Python In order to use Spark in Python, environment has to bee set up according to Using Spark in UFAL Environment. Starting Interactive Shell Interactive shell can be started using: pyspark Better interactive shell with code completion using ipython3 (do `pip3 install text/html 2024-09-27T09:21:24+00:00 Anonymous (anonymous@undisclosed.example.com) spark:using-scala https://wiki.ufal.ms.mff.cuni.cz/spark:using-scala?rev=1727421684&do=diff Using Scala In order to use Spark in Scala, environment has to bee set up according to Using Spark in UFAL Environment (including the sbt). Starting Interactive Shell Interactive shell can be started using: spark-shell As described in Running Spark on Single Machine or on Cluster, environmental variable MASTER specifies which Spark master to use (or whether to start a local one).