Press "Enter" to skip to content

Month: January 2025

Near Real-Time Data Plotting in Python

Published 2025-01-16 by Kevin Feasel

Hristo Hristov wants to know where the International Space Station is:

Gathering data on events as they occur in real-time is a powerful and popular technique in scientific and industrial computing. If we can query an online REST API representing the position of the International Space Station’s (ISS), how can we visualize these data in real time? How do you plot the data points as soon as they arrive and observe changes in the station’s position immediately? Let’s look at using Python for a real time plot of data.

Click through for the solution and plenty of explanation along the way.

Leave a Comment

Speed Differences with Separating Data and Log Files

Published 2025-01-16 by Kevin Feasel

Brent Ozar performs a test:

I’ve already explained that no, it doesn’t make your database server more reliable – and in fact, it’s the exact opposite. But what about performance?

The answer is going to depend on your hardware and workload, but let’s work through an example. I’ll take the first lab workload from the Mastering Server Tuning class and set it up on an AWS i3en.2xlarge VM, which has 8 cores, 64GB RAM, and two 2.5TB NVMe SSDs. (This was one of the cheapest SQL-friendly VM types with two SSDs, but of course there are any number of ways you could run a test like this, including EBS volumes.)

I would expect cloud versus on-premises answers to be quite different, because cloud services tend to throttle you hard on how much storage throughput you’re allowed to have. For that reason, the results make perfect sense in AWS (or Azure or GCP for that matter), but unless your on-prem solution has hard throttles on IOPS or throughput because your sysadmins are monsters, the limits of performance would be in how hard you can push the drives or your storage controllers.

Ultimately, the most appropriate answer is to test your systems and not rely on expectations, especially if you’re shifting from on-premises to a cloud (or vice versa).

Leave a Comment

The Power of Consistency

Published 2025-01-16 by Kevin Feasel

Kevin Wilkie doesn’t like having four answers to the same problem:

I began reviewing the first SQL Server. Backups were done via a maintenance plan. No worries there. Many places rely on maintenance plans to do their backups each day.

On the second SQL Server, I found no maintenance plans. What I did find though, was a SSIS package creating backups each night for all of the databases on the server.

On the third SQL Server, I found an older version of Ola Hallengren’s Scripts.

And it goes on from there. Kevin makes a good point around how having a consistent solution is important, especially in troubleshooting scenarios. It’s easier to resolve problems if there’s a consistent pattern versus having to remember that, oh, yeah, this is the instance we tried using this one solution, but it didn’t work so we went with something else on another instance.

Leave a Comment

T-SQL Tuesday 182 Round-Up

Published 2025-01-16 by Kevin Feasel

Rob Farley gives us a roundup:

I was the host for T-SQL Tuesday this month, inviting people to write about integrity. And because I don’t believe in just providing a list of posts (although I’ve also done that at the end of the post), I have a completely fabricated story for you. And no, this wasn’t created using AI. I have more integrity than that (see what I did there?).

Read on for Rob’s summary and all of the links.

Leave a Comment

Creating a Microsoft Fabric Warehouse with Service Principal

Published 2025-01-16 by Kevin Feasel

Gilbert Quevauvilliers sets up a new warehouse:

In this blog post I am going to show you how to create a Microsoft Fabric Warehouse, where the owner will be the Service Principal.

As mentioned in the blog post here are some of the advantages of having the Service Principal as the Warehouse Owner.

  • Using a Service Principal to create the warehouse avoids issue where the person who created the warehouse leaves the organization and issues arise when the users account is deleted from Entra ID.
  • You avoid the painful logging in with the user account to ensure the password remains updated.
  • The organization now owns the warehouse and not an individual user.

I will show you how I created a Warehouse with the owner being a Service Principal this using a Microsoft Fabric Notebook

Click through for the notebook and additional commentary.

Leave a Comment

2025 Data Professional Salary Survey Results

Published 2025-01-15 by Kevin Feasel

Brent Ozar shares this year’s survey results:

We’ve been running our annual Data Professional Salary Survey for almost a decade, and I was really curious to see what the results would hold this year. How would inflation and layoffs impact the database world? Download the raw data here and slice & dice it to see what’s important to you. Here’s what I found.

Read on for Brent’s analysis and grab the data for yourself to try things out. I’ve used this dataset in the past for presentations and it usually goes over pretty well, especially because it includes quite a few real-life data quality challenges.

Leave a Comment

Working with DBCC OPTIMIZER_WHATIF

Published 2025-01-15 by Kevin Feasel

Yvonne Vanslageren shows off a very uncommon DBCC command:

DBCC OPTIMIZER_WHATIF is a powerful diagnostic command in SQL Server that allows database administrators and developers to simulate various hardware configurations. By doing so, it shows how the query optimizer would behave under different CPU and memory allocations—without requiring you to physically change server hardware. This makes it an invaluable tool for performance tuning, troubleshooting, and strategic planning in SQL Server environments.

This obviously won’t actually make things faster, but will help you answer questions like “would more CPU cores improve this query or do I need to dump more RAM into the server?”

Leave a Comment

Temp Table Performance in PostgreSQL

Published 2025-01-15 by Kevin Feasel

Jobin Augustine shares a warning:

PostgreSQL is one of the most powerful database systems in the world. I have always been passionate about its great power, especially its modern SQL language features.

However, that doesn’t mean everything is great. There are areas where it hurts. Novice users unaware of the problem might get into trouble, and I see such things happen very often. This is a note for those who design their solutions around PostgreSQL using temporary tables.

This is very interesting to read, especially in comparison to SQL Server. It’s another example of the adage that, just because something works in a specific way on one platform, it’s not necessarily going to work the same way on another.

Leave a Comment