Press "Enter" to skip to content

Curated SQL Posts

Getting Started with Microsoft Fabric

Published 2023-11-21 by Kevin Feasel

Eugene Meidinger is ready to be at the intersection of Dunning-Kreuger and Imposter Syndrome:

I’ve written before about struggling to learn Azure Synapse, and I’ve struggled as well with getting excited about Microsoft Fabric. I think the pitch and the potential of Microsoft Fabric is real. The issue is that it solves problems I don’t have. In my work, I don’t deal with data so big that Power BI can’t handle it. I don’t deal with data so unstructured that Power Query can’t handle it.

But I know I need to learn Fabric. Power BI is a part of Fabric, the integrations are only going to continue to improve. If nothing else, I need to be able to tell customers if they should look into using Fabric or not. So what do you do when there is a technology you aren’t excited about, but have to learn?

Read on for Eugene’s scenario, which is certainly more interesting than Adventure Works.

Comments closed

Strategies for Filtering on Bit Columns

Published 2023-11-21 by Kevin Feasel

Aaron Bertrand answers a question:

Recently someone posted a question where they couldn’t quite figure out how to construct a predicate based on a bit parameter. They tried to write a procedure like this, which wouldn’t parse, of course:

At first, I thought Aaron meant querying integer bitmasks in T-SQL, in which case, the best strategy is “don’t.” But this is a different and much more useful scenario.

Comments closed

The State of Microsoft Fabric

Published 2023-11-21 by Kevin Feasel

Reitse Eskens shares some advice:

Last week the big announcement came at Microsoft Ignite, Fabric is GA.

Very cool, a lot of noise again for this shiny toolbox, but do we need to abandon everything and focus solely on the new toys?

Before I’ll answer that question, let’s look at a few moving parts of Fabric.

I think it’s still 1-2 years out from being fully baked. My hope is that there are (or will be) enough pieces in place to make it useful for enough scenarios that people don’t notice the gaps too much. There’s a lot of potential here and I don’t want Fabric to end up with a reputation of “too much stuff is missing to use it” because that reputation is hard to shake.

Comments closed

Displaying End-User-Defined Characters (EUDC) in SSMS

Published 2023-11-21 by Kevin Feasel

James Ferebee notes a problem:

A customer opened an interesting case stating SQL Server Management Studio (SSMS) would not display EUDC (End-User-Defined Characters) properly. More information on EUDC can be found here: End-User-Defined and Private Use Area Characters. 

After testing, it was identified by default, both SQL Server Management Studio and Visual Studio encountered this problem. In contrast, Notepad.exe displayed the desired character as intended. A shared aspect of SSMS and VS is the use of Windows Presentation Foundation (WPF).

Read on to see an example of how to create an EUDC and the workaround. I’d never even heard of EUDCs before, and the steps involved to get it to work probably explains why.

Comments closed

Lakehouse Management in Fabric via mssparkutils

Published 2023-11-21 by Kevin Feasel

Sandeep Pawar scripts out some lakehouse work:

At MS Ignite, Microsoft unveiled a variety of new APIs designed for working with Fabric items, such as workspaces, Spark jobs, lakehouses, warehouses, ML items, and more. You can find detailed information about these APIs here. These APIs will be critical in the automation and CI/CD of Fabric workloads.

With the release of these APIs, a new method has been added to the mssparkutils library to simplify working with lakehouses. In this blog, I will explore the available options and provide examples. Please note that at the time of writing this blog, the information has not been published on the official documentation page, so keep an eye on the documentation for changes.

This looks to be quite useful for CI/CD work.

Comments closed

Setting a Fixed Size for Transaction Log VLFs

Published 2023-11-21 by Kevin Feasel

Paul White was assured that there would be no math:

This is all very familiar, of course, but it is also dumb. Why on earth should we have to worry about internal formulas? It seems ridiculous to have to provision or grow a transaction log in pieces just to get a reasonable VLF outcome.

Wouldn’t it be better to be able to specify a fixed size for VLFs instead?

Starting with SQL Server 2022, there is now a way though it is undocumented and unsupported for the time being at least.

Read on to see what that option is, how it works, and what the limitations are. Looking at the side effects, I’d say this is probably not something you want to jump on right now.

Comments closed

New Features in healthyR.ts 0.3

Published 2023-11-20 by Kevin Feasel

Steven Sanderson lays out some updates:

One of the standout additions is the introduction of util_log_ts(). This function seems like a game-changer, providing a streamlined way to log time series data. This is incredibly useful, especially when dealing with extensive datasets, making the whole process more efficient and user-friendly. This is a helper function for auto_stationarize().

There’s a lot in this update and the blog post also includes several examples of automating stationarity and ARIMA.

Comments closed

Choosing the Right Technology in the Modern Azure Data Warehouse

Published 2023-11-20 by Kevin Feasel

Josephine Bush has some advice:

Here’s a quick description of the options we explored:

  • Azure Data Factory – Orchestrates and automates data movement and transformation. You can create workflows, pipelines, and ETL (Extract, Transform, Load) processes using it.
  • Databricks – A unified data science, engineering, and analytics platform based on Apache Spark. It simplifies data exploration, preparation, and machine learning workflows, allowing teams to collaborate efficiently. Interactive notebooks make Databricks a versatile tool for scalable data analysis and processing.
  • Synapse – Integration of big data and data warehousing in the cloud. It facilitates collaborative analytics and AI-driven insights using serverless and provisioned resources across various data sources. Integrated analytics, warehousing, and data integration are part of Synapse’s unified experience.
  • Fabric – An all-in-one analytics solution for enterprises that offers data movement, data lakes, data engineering, data integration, data science, and real-time analytics.

Read on for pros and cons of different options Josephine & crew reviewed, as well as the option they landed on and why.

Comments closed

Many-to-Many Power BI Relationships and Table Refreshes

Published 2023-11-20 by Kevin Feasel

Dany Hoter gives us a reason to minimize use of many-to-many relationships in Power BI:

I must admit that in the last two years I’ve told many Power BI/Kusto customers not to worry about relationships that are created as M:M.

I was pretty sure that with Direct Query, such relationships are fine,

Indeed, the generated queries looked fine and performed as expected.

I recently became aware that the number of queries generated for some visuals e.g. Matrix and tables can be affected by the type of relationships between the participating tables.

Read on for a description of why you shouldn’t load your Power BI semantic models with many-to-many relationships, especially once Kusto is involved.

Comments closed

A Pattern for DAX Time Intelligence Functions

Published 2023-11-20 by Kevin Feasel

Allison Kennedy checks the calendar:

Time Intelligence functions in DAX change the filter context on the Date table of your model. 

Step 0: Prepare 

Before using Time Intelligence functions, you should:

Read on for a four-step process covering how to apply a time intelligence function like DATEADD() or DATESYTD() in DAX.

Comments closed