Press "Enter" to skip to content

Day: June 11, 2026

Building a Type-6 Slowly Changing Dimension

Dinesh Asanka creates a dimenson:

In a data warehouse, one important concept is to retain historical data. This data is typically not available in operational systems. One approach in data warehouses is the use of Slowly Changing Dimensions (SCDs). What are the SCD options and are there any new approaches?

Click through for a quick depiction of Types 0 through 3, and then where 6 fits into the mix. I’m not 100% sure I’ve ever actually used a Type-6 slowly changing dimension in a production environment, though there are specific circumstances in which one could be quite useful.

Leave a Comment

Query Execution vs Query Processing

Louis Davidson disambiguates a pair of terms:

There have been a lot of posts on LinkedIn of late about the “logical execution order” of a query that all really miss some really big points. I was corrected myself in some terminology because I mistook the term “processing” to mean the same as “order” in these discussions when I was explaining why logical execution order is not what people expected.

Click through for a good explanation, as well as a plan to have your company pay for your Disney World trip.

I also appreciate how Louis still calls it SQL Sentry Plan Explorer because that’s what it always will be in my heart.

Leave a Comment

On-Prem Is Still On-Prem

Andy Brownsword reminds us that it’s still the same animal:

Cloud solutions have sliders which magically reduce downtime, performance issues, and account credit. When you build it, infrastructure and platforms are your challenge, and you get one shot to right-size it. Capacity, resilience, capabilities. They’re your challenges to solve. There is no cake slider.

It’s the capabilities I want to consider here. The capabilities we have available to us day-to-day. Potentially two opposing sides:

To summarize things, everything is awful, just as it always has been. Granted, that may not be how Andy would summarize things, but that’s why I get those big Curated SQL bucks.

Leave a Comment

Re-Migration and Data Engineering

Andy Leonard shares some thoughts:

Right-sizing didn’t always work out the way some clients were led to believe it would.

In nearly every instance, the right-sizing argument was presented (sold) as the solution to over-provisioning, or purchasing hardware to serve peak loads. The classic example was a US income tax service that needed more and faster compute available to meet increasing demand starting in late January and peaking in mid-April each calendar year. After mid-April, hardware that was beefy enough to handle that peak load sat mostly idle for the next 9 months.

I don’t think I’ve ever worked for a company where this scenario really made sense. Even in the e-commerce company where a sizable fraction of our total annual revenues happened over a 5-day period, the load was still significant enough the rest of the year that we made good use of our on-premises SQL Server hardware.

Leave a Comment

Identity Columns in Fabric Runtime 2.0

Jon Lunn is happy:

Fabric Runtime 2.0 is in public preview, and there is one small change for this that makes my life as a data engineer a little bit easier. Identity columns. I miss the good old T-SQL Identity column “Id INT IDENTITY(1,1)”, now we have something like it in Spark/Delta.

So with the the old spark/delta table runtime, you couldn’t have an default, automatically increasing column. But now we have these updates.

Click through to see how it works.

Leave a Comment