Multi-Parameter Website Scraping With Power Query

Callum Green shows how to build up a URL based off of multiple parameters, scraping data from a page for each permutation of parameters:

The sections highlighted in red are the parameters and sit in between some of the hard-coded URL text

Code Breakdown:

–          Text = http://www.boxofficemojo.com/monthly/?page=

–          Parameter = [Page]

–          Text = &view=calendargross&yr=

–          Parameter = [Year]

–          Text = &month=

–          Parameter = [Month]

–          Text = &p=.htm

This is a rather clever solution, and if your parameters are functionally dependent (unlike this example, where it was a simple cross join of the three domains), you can still use the solution the same way; you just need to populate your parameter combination table differently.

Related Posts

Calculating Value Above Replacement

Rob Collie explains how to calculate Value Above Replacement: Let’s say you’re trying to improve the fuel efficiency of your car, and you have a budget of $100 to make those improvements.  Furthermore, let’s say that you can only spend your budget on three different kinds of improvements:  better tires, better spark plugs, and a better […]

Read More

Embedding Refresh Times in Power BI Reports

Marc Lelijveld shows how you can embed Power BI Dataflow refresh times in your Power BI reports: But maybe you want to visualize this as part of your report as well. With a really simple piece of Power Query code you can easily generate a date/time at the moment that your dataset is processed. Kasper […]

Read More

Categories

October 2017
MTWTFSS
« Sep Nov »
 1
2345678
9101112131415
16171819202122
23242526272829
3031