Press "Enter" to skip to content

Curated SQL Posts

BotChat BiWeekly

Published 2023-09-05 by Kevin Feasel

Mala Mahadevan starts a newsletter:

I do my best to find trustworthy sources to learn from, but you know how it is – sometimes it’s tough to tell what’s legit. So, if you ever see me post something that seems a bit off, please cut me some slack. These aren’t necessarily my opinions, just things that caught my eye.

What I learn is just my take on what I heard or read. It might not always jive with what the original speaker or writer means, or understand. I don’t use any fancy AI bots like ChatGPT to help me out. I just quote stuff and break it down in my own words.

Mala focuses on a pair of videos. I snuck into the newsletter with a few bomb-throwing statements, particularly around anthropomorphism (the assignment of human or human-like qualities to non-humans). Anthropomorphism is extremely common in language. It’s all well and good as metaphor, but once you start to believe it for real, that’s when you end up in trouble.

Comments closed

A Currency Conversion Pattern for DAX

Published 2023-09-05 by Kevin Feasel

Teo Lachev pulls out the money clip:

Currency conversion is a common requirement, such as when implementing analytics on top of ERP sales data recorded in multiple currencies. The most flexible approach is to allow the user to select a currency and perform the conversion at runtime, ideally by centralizing the conversion code in Power BI calculation groups. However, this approach has several shortcomings:

Read on for those shortcomings, as well as an approach Teo has to improve things.

Comments closed

Row-Level Security Setup in SQL Server

Published 2023-09-05 by Kevin Feasel

Ben Johnston configures row-level security in SQL Server:

Implementing RLS is as much a business endeavour as technical. Without a business case to implement RLS, there is no reason for the extra effort and testing involved. This is where driving out business requirements and making sure the solution fits the problem is important. Non-technical members of the team or business partners likely won’t know what RLS is or if it should be used. Differential access to the same data, replacing systems (or proposed systems) with multiple reports based on user groups, and multi-tenant access are possible indicators that RLS may be a useful tool. There are always multiple ways to solve a problem. If RLS would simplify the design and make it more robust, that’s when I start to seriously consider it for a design. It does help if the business is aware of RLS and have used it in other projects or databases, but having the business essentially design the system is dangerous too. Use all of the information available during planning sessions and design the system that best fits the need of the business and the skills of the technical team.

Read the whole thing. I’m particularly interested in this series, as I’ve liked row-level security and didn’t find any trivial workarounds or exploits, but also hate how slow it is once you’re dealing with large datasets.

Comments closed

Australian Azure Downtime After-Action Report

Published 2023-09-05 by Kevin Feasel

Brent Ozar shares some thoughts:

Note that 11:34, the decision was made to shut down infrastructure without Microsoft failing your databases over elsewhere. If you were an Azure SQL DB or Cosmos DB user, and you weren’t paying for replicas in another data center, it was up to you to follow Microsoft’s disaster recovery guidance.

Controversial opinion: I actually love that and I think it’s great.

That is definitely a controversial opinion, but it’s also one I agree with. Read on for more of Brent’s thoughts.

Comments closed

Monitoring Azure SQL Transaction Log Usage with Powershell

Published 2023-09-05 by Kevin Feasel

Jose Manuel Jurado Diaz has a script for us:

Database administrators and IT professionals often need to be proactive in monitoring resources, especially when working in cloud environments like Azure SQL. One critical resource that requires monitoring is the transaction log of a SQL Server database. If it fills up, it can hinder database operations, leading to potential application downtime.

In this article, we’ll discuss a PowerShell script that monitors transaction log usage in Azure SQL databases and sends an email alert if a database exceeds a specific threshold.

There is a reference in the script to Check-LogUsage, which appears to be a different function, likely related to the linked T-SQL statement Jose posted.

Comments closed

The Value of KPIs and Cards in Power BI

Published 2023-09-05 by Kevin Feasel

Kurt Buhler and Stepan Resl give you a card:

When a user arrives at your report, they should be able to answer their most important questions in a few seconds. To do this, we typically put the most critical information in the top-left of the report (where we often look first). This information should provide a high-level overview, whereas additional details should be placed at the bottom of the report, behind interactions, or on later pages.

An effective and popular way to call attention to important numbers in Power BI is by using cards and KPI core visuals.

Read on for several examples and a breakdown of how they work best.

Comments closed

Enabling Staging for Microsoft Fabric Dataflows

Published 2023-09-05 by Kevin Feasel

Chris Webb shares some thoughts:

If you read this post that was published on the Fabric blog back in July, you’ll know that each Power Query query in a Fabric Gen2 dataflow has a property that determines whether its output is staged or not – where “staged” means that the output is written to the (soon-to-be hidden) Lakehouse linked to the dataflow, regardless of whether you have set a destination for the query output to be written to. Turning this on or off can have a big impact on your refresh times, making them a lot faster or a lot slower.

Chris shares a simple example of when staging might not be reasonable. This is going to be the less common scenario, however.

Comments closed

Tuning Kafka Connect Source Connectors

Published 2023-09-01 by Kevin Feasel

Catalin Pop makes things faster:

Kafka Connect is an open source data integration tool that simplifies the process of streaming data between Apache Kafka® and other systems. Kafka Connect has two types of connectors: source connectors and sink connectors. Source connectors allow you to read data from various sources and write it to Kafka topics. Sink connectors send data from the topics to another endpoint. This blog post discusses how to tune your source connectors to help you get the best throughput out of your compute resources. 

This includes which elements are tunable, metrics you’ll want to pay attention to along the way, and a detailed example.

Comments closed

Querying the Power BI REST API from Fabric Spark

Published 2023-09-01 by Kevin Feasel

Gerhard Brueckl makes the call:

Microsoft Fabric has a lot of different components which usually work very well together. However, even though Power BI is a fundamental part of Fabric, there is not really a tight integration between Data Engineering components and Power BI. In this blog post I will show you an easy and reusable way to query the Power BI REST API via Fabric SQL in a very straight forward way. The extracted data can then be stored in the data lake e.g. to create a history of your dataset refreshes, the state of your workspaces or any other information that is provided by the REST API.

Click through for a list of operations, followed by the code you’ll need to pull this off.

Comments closed

PostgreSQL 16 and Infinity

Published 2023-09-01 by Kevin Feasel

Ryan Lambert goes to infinity and beyond:

This month, Ryan Booz chose the topic: What Excites You About PostgreSQL 16? With the release of Postgres 16 expected in the near(ish) future, it’s starting to get real. It won’t be long until casual users are upgrading their Postgres instances. To decide what to write about I headed to the Postgres 16 release notes to scan through the documents. Through all of the items, I picked this item attributed to Vik Fearing.

  • Accept the spelling “+infinity” in datetime input

The rest of this post looks at what this means, and why I think this matters.

Read on to see what’s new about this and what it all means.

Comments closed