Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
spark:using-python [2017/10/16 20:56] ufal [Starting Interactive Shell] |
spark:using-python [2019/11/12 13:37] straka |
||
---|---|---|---|
Line 8: | Line 8: | ||
< | < | ||
- | Better interactive shell with code completion using '' | + | Better interactive shell with code completion using '' |
- | < | + | < |
As described in [[running-spark-on-single-machine-or-on-cluster|Running Spark on Single Machine or on Cluster]], environmental variable '' | As described in [[running-spark-on-single-machine-or-on-cluster|Running Spark on Single Machine or on Cluster]], environmental variable '' | ||
Line 20: | Line 20: | ||
| | ||
| | ||
- | | + | |
| | ||
</ | </ | ||
* run interactive shell using existing Spark cluster (i.e., inside '' | * run interactive shell using existing Spark cluster (i.e., inside '' | ||
- | < | + | < |
* run interactive shell with local Spark cluster using one thread: | * run interactive shell with local Spark cluster using one thread: | ||
- | < | + | < |
* start Spark cluster (10 machines, 1GB RAM each) on SGE and run interactive shell: | * start Spark cluster (10 machines, 1GB RAM each) on SGE and run interactive shell: | ||
- | < | + | < |
- | Note that '' | + | Note that '' |
Line 60: | Line 60: | ||
| | ||
| | ||
- | | + | |
| | ||
sc.stop() | sc.stop() |