Dmitry Tolpeko digs through Tez logs to figure out a performance issue:
Why did it take so long to run the job? Is there any way to improve its performance?
Tez Application Master Log
I am going to use the Tez AM log to investigate vertex performance and find possible bottlenecks.
Note that there is the Timeline Server REST API that you can use to get the statistics for Tez jobs, but the application master log is “event-driven”, shows the exact order of all events and contains much more details in general.
Click through for the process.