Hanging processes in post-processing

[mkaranasou]

Use: conf.set('spark.python.worker.reuse', 'true') to test if this resolves the spark issue.