how to get consolidated report at the task/job/stage in spark application

[ntallapa12]

Hi Jay,

I read some of the articles in your blog related to StatsReportListener, they are good. I am looking to log some stats after the spark application finishes.
Few metrics like

• Number of executors, jobs, stages, and tasks launched during entire application

I used event listeners like SparkListener but, on these, events are fired for every task/job/stage, I am looking for some consolidated report. Appreciate if you have any inputs around this.

Thanks,
Niranjan