Gilbert Quevauvilliers builds a loop:
Continuing with my existing blog series on what I’m learning with notebooks and PySpark.
Today, I’m going to explain to you how I found a way to loop through data in a notebook.
In this example, I’m going to show you how I loop through a range of dates, which can then be used in a subsequent query to extract data by passing through each date into a DAX query.
Click through for Gilbert’s example. Here’s an alternative using something called a list comprehension. First, build a function that does what you want to do—that’d be the innards of Gilbert’s Python code, lines 31-54.
def perform_dax_query(row):
var_Date = row["Date"]
...
display(df_DAX_QueryResult)
Then, call that function for each row:
[perform_dax_query(row) for row in data_collect]
In this particular scenario, I’d personally stick with Gilbert’s composition, but in cases where you’re transforming a list of elements into a new list—for example, if you’re performing some data cleanup for each row in a list and you want the output to be a new list with cleaned-up data—then the list comprehension works really well.