A Fine Slice Of SQL Server
Andy Leonard shows how you can incrementally develop Azure Data Factory pipelines:
A friend pinged me recently to ask about rolling back Azure Data Factory (ADF) pipeline versions. My response was a question: Are you using source control with ADF? That did not help the current situation.
I thought of the way I often build ADF pipelines and shared my methodology, which is relatively simple (it has to be simple for me to understand it!):
Click through for Andy’s approach.
Comments closedRab Saker and Bryan Smith hit on a topic close to my heart:
These patterns seem to indicate that KKBox could actually differentiate between customers based on their lifetime potential using information known at the time of acquisition. This information might help inform or steer specific discounts or promotions to customers as they register for a trial. This information might also inform KKBox of which offerings or capabilities to discontinue as some, e.g. Initial Payment Method 35 or the 7-day payment plan as shown in Figure 3, align with exceptionally high churn rates in the first 30-days with little long-term survivorship.
Of course, there are relationships between these factors so that we should be careful in viewing them in isolation. By deriving a baseline risk (hazard) of customer churn (Figure 4), we can calculate the influence of different factors on the baseline in such a manner that each factor may be considered an independent hazard multiplier. When combined (through simple multiplication) against the baseline, we can plot the a specific customer’s chances of abandoning a subscription by a given point in time (Table 1).
Click through for the story as well as a set of notebooks.
Comments closedStephanie Glen contrasts ANOVA against typical regression techniques using a picture:
If you scour the internet for “ANOVA vs Regression”, you might be confused by the results. Are they the same? Or aren’t they? The answer is that they can be the same procedure, if you set them up to be that way. But there are differences between the two methods. This one picture sums up those differences.
Click through to see that image.
Comments closedAlan Yu announces this month’s changes to Azure Data Studio:
The key highlights to cover this month include:
– Feature tour.
– Notebooks enhancements.
– Drag and Drop Tables in Query Editor.
– Azure Account icon added to Activity Bar.
– Insiders Build highlights.
– Bug fixes.
Read on for the set of changes.
Comments closedI’m not going to sugarcoat things – the person that sent me the request has more access than they rightly need. The “public” access worker did not need any of that access so I wasn’t going to just give her the same level.
Plus, we’re supposed to be a workforce that has embraced the DevOps spirit and DevOps is nothing if it doesn’t include Security in it.
So, if I could find a way to give the user enough permission to run the command and not a lot more, then the happier I would be.
Shane takes us through the process so we don’t have to.
Comments closedRob Farley takes us through the process of creating and using a Power BI streaming dataset:
Real-time Power BI sets are a really useful feature, and there’s a good description of them at https://docs.microsoft.com/en-us/power-bi/connect-data/service-real-time-streaming. I thought I’d do a quick walkthrough specifically around the Push side, and show you – including the odd gotcha that you might not have noticed.
To create a dataset that you want to push data into, you need to go to the Power BI service, go to your Workspace, and create a Streaming dataset. Even if you’re not wanting to use it with a streaming service, this is the one you need.
Rob has plenty of animated GIFs to walk you through the process, as well as a couple of caveats if you want to play along at home.
Comments closedMartin Schoombee gives us a warning around relying upon free custom visuals:
Before you get the impression that I’m against custom visuals, let me say this: I love custom visuals! I myself have used many custom visuals in the past and have been very quick to look for a custom visual when I couldn’t get something to display or work the way I needed it to in Power BI.
Custom visuals fill an important gap where the base product is not yet where it needs to be, and what better way for Microsoft to see what people need and where they need to invest more time from a visualization standpoint? It’s an awesome concept and I like it.
Unfortunately there are a few BUT’s to follow, but let me first tell you my story…
Read the whole thing. I like custom visuals a lot, but there are risks in a corporate world, and I don’t necessarily mean security.
Comments closedBen Miller takes us through a way to know your data:
You have many tables in your databases and you want to know how they are used. There are DMVs for index usage stats which will tell you about like sys.dm_db_index_usage_stats and querying them is insightful, but how do the stats change over time? These stats are reset when the instance is restarted and it is good to know that you have 2000 seeks and 500 scans of the index, but when did they happen? Was it on a common day? Common hour?
Ben has a way to help you figure that out.
Comments closedI’m not mad at 2019 or Batch Mode On Rowstore (BMOR) or anything.
But if I’m gonna get into it, I’m gonna document issues I run into so that hopefully they help you out, too.
One thing I ran into recently was where BMOR kicked in for a query and made it slow down.
Click through for the scenario, why it’s slower when using batch mode, and two ways you can improve the query.
Comments closedThe Hadoop in Real World team walks us through resource allocation for Spark applications:
In this post we will look at how to calculate resource allocation for Spark applications. Figuring out how to allocate resources for a Spark application requires a good understanding of resource allocation properties in YARN and also resource related properties in Spark. Let’s look at both.
This post covers the properties you want to keep an eye on when running Spark applications.
Comments closed