How to Read Spark DAGs, Rock the JVM
Written form to keep for later: Check out the Spark performance courses: Download Spark: Related video on reading Spark Query Plans: This video is for Spark programmers who know the essentials create a DataFrame, basic selects, joins) and wants a sneak peek at how Spark works internally and get an essential skill for performance analysis and improvement. In this video I ll show you how Spark creates computation dependencies before it can run anything, and I ll show you how to read and interpret Directed Acyclic Graphs (DAGs) and identify performance bottlenecks while the job is running. Follow Rock the JVM on: LinkedIn: Twitter: Blog:
|
|