Performance Test: Loading CSV Versus Loading Excel In Power Query

Chris Webb lays out a performance test which shows how quickly Power Query can read data from a CSV versus from an Excel spreadsheet:

The black line in the graph above is the amount of data read (actually the offset values showing where in the file the data is read from, which is the same thing as a running total when Power Query is reading all the data) from the Excel file; the green line is the amount of data read from the CSV file (the same data shown in the first graph above). A few things to mention:

  • Running Process Monitor while this second query was refreshing had a noticeable impact on its performance – in fact it was almost 20 seconds slower

  • The initial values of 80 million bytes seem to be where data is read from the end of the Excel file. Maybe this is Power Query reading some file metadata? Anyway, it seems as though it takes 5 seconds before it starts to read the data needed by the query.

  • There’s a plateau between the 10 and 20 second mark where not much is happening; this didn’t happen consistently and may have been connected to the fact that Process Monitor was running

The results were remarkable; check them out.

Related Posts

Storing Large Images in Power BI

Chris Webb shows us how to store a large image in Power BI: Jason Thomas and Gerhard Brueckl have both blogged on the subject of storing images as text inside a Power BI dataset:http://sqljason.com/2018/01/embedding-images-in-power-bi-using-base64.htmlhttps://blog.gbrueckl.at/2018/01/storing-images-powerbi-analysis-services-data-models/ Since they wrote those posts, however, Power BI has added the ability to set the Data Category property on measures as […]

Read More

Power BI Connection String Capitalization

Slava Murygin ran into a problem with capitalization on connection strings with Power BI Report Server: The message actually says:“Several errors occurred during data refresh. Please try again later or contact your administrator.”SessionID: 1b80301e-3898-417a-af9c-2e77ec490728[0] -1055784932: Credentials are required to connect to the SQL source. (Source at SQLServerName;DBA_Pro.). The exception was raised by the IDbCommand interface.[1] […]

Read More

Categories

August 2018
MTWTFSS
« Jul Sep »
 12345
6789101112
13141516171819
20212223242526
2728293031