Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | Next revision Both sides next revision | ||
spark:running-spark-on-single-machine-or-on-cluster [2022/12/14 12:53] straka [Memory Specification] |
spark:running-spark-on-single-machine-or-on-cluster [2022/12/14 12:54] straka [Memory Specification] |
||
---|---|---|---|
Line 29: | Line 29: | ||
TL;DR: Good default is '' | TL;DR: Good default is '' | ||
- | The memory for each worker is specified using the following format: < | + | The memory for each worker is specified using the following format: |
+ | < | ||
The Spark memory limits the Java heap, and half of it is reserved for memory storage of cached RDDs. The second value sets a memory limit of every Python process and is by default set to '' | The Spark memory limits the Java heap, and half of it is reserved for memory storage of cached RDDs. The second value sets a memory limit of every Python process and is by default set to '' | ||