Use ipython , not python for your REPL. It has better code completion and introspection, and when you insert an embedded code, it can automatically "back off" from the inserted code.
This way you can put your experimental code in a test function and then insert the parts without worrying, and you will have to cancel your code.
If you insert large blocks that can be thought of as separate blocks, you will need to use the magic of %paste or %cpaste .
eg.
for i in range(3): i *= 2
With regular paste:
In [1]: for i in range(3): ...: i *= 2 ...: In [2]: print(i) 4
Using %paste
In [3]: %paste for i in range(10): i *= 2 print(i)
PySpark and IPython
You can also run PySpark in IPython, an advanced Python interpreter. PySpark works with IPython 1.0.0 and later. To use IPython, set the IPYTHON variable to 1 when bin / pyspark starts: 1
$ IPYTHON=1 ./bin/pyspark
Dunes source share