Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | Next revision Both sides next revision | ||
spark:spark-introduction [2022/12/14 12:27] straka [Spark Introduction] |
spark:spark-introduction [2022/12/14 12:29] straka [Running Spark Shell in Python] |
||
---|---|---|---|
Line 6: | Line 6: | ||
To run interactive Python shell in local Spark mode, run (on your local workstation or on cluster using '' | To run interactive Python shell in local Spark mode, run (on your local workstation or on cluster using '' | ||
- | | + | |
- | The IPYTHON=1 parameter instructs Spark to use '' | + | The PYSPARK_DRIVER_PYTHON=ipython3 |
- | After a local Spark executor is started, the Python shell starts. | + | After a local Spark executor is started, the Python shell starts. |
- | the prompt line, the SparkUI | + | the prompt line, the Spark UI address is listed in the following format: |
- | | + | |
- | The SparkUI | + | The Spark UI is an HTML interface, which displays the state of the application -- whether |
==== Running Spark Shell in Scala ==== | ==== Running Spark Shell in Scala ==== |