Press "Enter" to skip to content

Day: January 16, 2023

Spark Structured Streaming with Synapse

Ryan Adams builds a demo:

In this post we are going to look at an example of streaming IoT temperature data in Synapse Spark.  I have an IoT device that will stream temperature data from two sensors to IoT hub. We’ll use Synapse Spark to process the data, and finally write the output to persistent storage. Here is what our architecture will look like: 

Click through for the architectural diagram and step-by-step on how to put the demo together.

Comments closed

The Problem with Tornado Graphs

Rita Fainshtein takes a beloved chart and gives it the beating it rightfully deserves:

In the picture above, you can see the execution of the budget on the right (blue), and the budget amount on the left (orange).

This interesting and special graph can be perfectly integrated into any business dashboard.

But does it let you know where the company stands in relation to its budget?

No, it is not. An optimistic view would be challenging, if not misleading.

Click through for more info and alternative ways of displaying this. In fairness, the area where I like seeing tornado charts is a categorical comparison, not a time series comparison, of two things. For example, during a head-to-head matchup in a sporting event, a tornado chart showing relevant stats about each side (or a star player on each side) tends to work reasonably well. Yeah, there are still clearer ways of showing the results in that case, but these have a soft spot in my heart for that specific use case.

Comments closed

The Cost of Ignoring GDPR

Matthew McGiffen looks at the fines:

The biggest fines (both individually and in total) have been levied for improper use of data, with Amazon, WhatsApp, Google and Facebook topping the list. After that though we have fines for insufficient protection of data. In most cases this is where companies have had some form of data breach and the safeguards in place weren’t deemed sufficient. The largest fine in this case was against British Airways who were hacked in 2018 and they received a fine of over 22 million euros for the lack of safeguards. That was calculated as 1.5% of the company’s turnover in 2017.

Read on for a summary of fines by violation type, more detail, and a link to the core data.

Comments closed

Setting CPU Affinity (Correctly)

Taiob Ali does something out of the ordinary:

Setting CPU affinity in SQL Server is not a task you do every day. Rarely are there use cases when you need to do that. I had a recent requirement to do it. We plan to replace a physical server with half of its current CPU. Primarily due to faster CPU and workload moved off of SQL Server to other cloud services. To test, we needed to set the CPU affinity mask in one of our non-production servers. In the research, I learned about the side effect of setting CPU affinity mask, which is nicely explained in this ( by Klaus Aschenbrenner) and this (by Adam Denby) blog post.

Click through to learn more about the process.

Comments closed

Avoiding Dynamic Data Sources Error with OData.Feed

Chris Webb avoids an error altogether:

In my last post I showed how, in many cases, you can avoid the “dynamic data sources” error with OData data sources by taking advantage of query folding. That’s not always possible though and in this post I’ll show you how you can use the Query option of the OData.Feed function to do so instead.

As always, Chris provides some nice detail and good examples.

Comments closed