Press "Enter" to skip to content

Month: February 2024

The Death (and Life?) of Azure Synapse Analytics

Published 2024-02-07 by Kevin Feasel

Paul Andrew plays coroner:

I think it’s fair to say that Azure Synapse Analytics has had a hard life. It was announced in public preview as a surprise to most of the community, including Microsoft cloud solution architects. Ultimately meaning that very little private preview testing and feedback on the product was done before showing it to the world. This resulted in a lot of frustration in the subsequent year before it could be classified as generally available and more frustration after that while we battled with the missing production features. Even now, the product is lacking in a lot of functionality. Anyway, this is all in the past. Microsoft Fabric is the new kid on the block, and we need to address the unpopular question about the future of Synapse. And considering I’ve been very unpopular with the product teams before; I’ll take this one for the team. Sorry, but it needs to be addressed.

Read on for Paul’s thoughts. I tend to agree in general with his take, but do read Bogdan Crivat’s response. Bogdan is on the Synapse product team and shares some thoughts as well.

2 Comments

Translating Excel Date Values into R Dates

Published 2024-02-07 by Kevin Feasel

Steven Sanderson reads an Excel file:

Have you ever battled with Excel’s quirky date formats in your R projects? If so, you’re not alone! Those cryptic numbers can be a real headache, but fear not, fellow R warriors! Today, we’ll conquer this challenge and transform those numbers into beautiful, usable dates.

This is a common pain point in a lot of libraries and Steven shows how to solve it in R using a pair of functions.

Comments closed

New Power BI Course

Published 2024-02-07 by Kevin Feasel

Eugene Meidinger has gone indie:

The goal of this course is to orient you to the various pieces of Power BI, identify the source of problem, and give some general tips for solving them. If you are stuck and need help now, this should help.

Note! This is an early launch. Modules 1 and 2 are available now, and the remaining ones will be coming out weekly.

You can see the first module of Eugene’s course on YouTube and throw money at him to get the rest as he rolls it out. And if you do decide to throw money at him, load a bunch of pennies into a cannon and blast them at him, then tell him I sent you.

2 Comments

Batching Data Copy Operations in SQL Server

Published 2024-02-07 by Kevin Feasel

Jana Sattainathan performs a data migration:

The purpose of this post is to give you a skeleton of TSQL code on how to perform a copy of a large table (say from one database to another within the same SQL Server Instance) by breaking up the data into manageable chunks. I have blogged about how to break-up any Oracle or SQL Server table data into chunks in this blog post.

Read on to learn why you don’t want to perform the operation as a single query, and one alternative that’s available. The query is a good bit more complex, but that complexity is for a good cause.

Comments closed

Finally Blocks and Error Handling in Data Factory

Published 2024-02-07 by Kevin Feasel

Chen Hirsh doesn’t let failure get in the way of doing work:

Today I stumbled upon a weird behavior in Azure Data Factory (ADF) error handling.

ADF lets us add error handling in the flow control, In this example, I’m trying to copy some data, and if that fails go to on failure branch (red line). If the activity succeeded, go to on success branch (green line)

These work great (If you can call a failure great…).

Let’s take another step. What if I want to run another activity at the end, no matter if the copy succeeded or failed?

The behavior is a bit weird, as it doesn’t work quite the way you’d expect. Chen, however, shows us how to do it.

Comments closed

New Features in data.table

Published 2024-02-06 by Kevin Feasel

John MacKintosh takes us through what’s new in R’s data.table package:

The newest version of data.table has hit CRAN, and there are lots of great new features.

Among them, a %notin% function, a new let function that can be used instead of := ( I wasn’t too fussed about this originally but have tried it a few times today and I may well adopt it – although I do like that := really stands out in my code when assigning / updating variables).

Read on for the big changes. H/T R-Bloggers.

Comments closed

Exploring the Area under the ROC Curve

Published 2024-02-06 by Kevin Feasel

Aayush Srivastava takes us through one of the classics of classification:

In the realm of machine learning classification, model evaluation is an essential step to assess the performance and effectiveness of various algorithms. One widely-used tool for this purpose is the Area Under the Receiver Operating Characteristic Curve (AUC-ROC curve). In this blog, we will delve into the significance of the AUC-ROC curve, how it is calculated, and why it is an invaluable metric for evaluating classification models.

In this article, we will discuss the performance metrics used in the classification and also explore the implications of using two, namely AUC and ROC. Here is an overview of the important points that we will discuss in the article. 

The fun anecdote around ROC curves is that their name actually makes sense if you know the origin: it came out of the British army in World War II, where they tracked how their radar operators classified blips as German aircraft or noise (e.g., flocks of birds). The radar receiver operators had certain characteristics, where some were more effective at separating actual threats from noise, hence the Receiver Operating Characteristic curve.

Comments closed

Monitoring if an Azure Server Goes Offline

Published 2024-02-06 by Kevin Feasel

Paul Bergson builds an alert:

My miniature schnauzer, Raven, is a smart and lively dog who loves to hunt for rodents in the yard. She has a keen sense of smell and can detect the slightest movement of her prey. She barks loudly to alert me whenever she finds a potential target and chases after it with all her speed. However, the rodents are too cunning and often escape to a tree or a hole in the ground before she can catch them. She then returns to me with a disappointed look on her face, hoping for a treat or a pat on the head.

Azure Monitor is like Raven, but much more efficient and reliable. It can monitor your Azure servers and detect when they go offline in ~1 minute. It can also alert you via email, SMS, or webhook when something goes wrong, so you can take action to fix it. With Azure Monitor, you can stay on top of your server’s health and performance.

Read on to see how you can use Azure Monitor and build policies, with much less cleanup requirement than a dog.

Comments closed