Liliam Leme provides an answer to a common Synapse Spark pool question:
I’ve been reviewing customer questions centered around “Have I tried using MSSparkUtils to solve the problem?”
One of the questions asked was how to share results between notebooks. Every time you hit “run” in a notebook, it starts a new Spark cluster which means that each notebook would be using different sessions. Making it impossible to share results between executions of notebooks. MSSparkUtils offers a solution to handle this exact scenario.
Read on to see what MSSparkUtils is and how it helps in this case.