Press "Enter" to skip to content

Month: July 2023

Near-Real Time Reports with Power BI and KQL

Published 2023-07-28 by Kevin Feasel

Dany Hoter can’t wait:

Real time and near real time are subjective terms.

For some businesses, real time is up to 1 ms latency and for other cases 10 minutes latency is considered close to real time.

Lately I encountered a few cases in which Page refresh in PBI was used on a dataset using Direct Query against Kusto/ADX/RTA.

In this article I’ll cover a few best practices to ensure that such an implementation will be successful and conserve on resources.

I’m a bit of a stickler for the term “real-time” so I appreciate Dany’s preface here. The way I learned real-time versus online (versus batch) is, online is when you expect a result quickly but real-time is something you’d put in a fighter plane. And it turns out that, when you explain what the bill will look like, very few companies have the need for true real-time results.

Comments closed

Role Analysis: DBA, Data Engineer, DB Reliability Engineer

Published 2023-07-27 by Kevin Feasel

Kendra Little disambiguates roles:

Much has changed: development patterns transformed from Waterfall to Agile, DevOps drives automation and shared ownership of code, and cloud services have made many more kinds of PAAS databases, data lakes, and data lakehouses available to organizations of all sizes.

These changes have introduced new and varied career paths for data folks which have different emphases on skill sets. In this post, I talk through the commonalities and differences between DBAs, Database Reliability Engineers (DBREs), and Data Engineers (DEs). Whether you’re a hiring manager or data professional, it’s worth knowing about these roles.

And a fourth one I’d include is Database Engineer, which I’ve also called a development DBA (versus a production DBA): focus on one platform, like SQL Server, and specialize in code development and tuning on that platform. I’d expect a Data Engineer to be familiar with at least one or two non-SQL programming languages—Python, Scala, Java, C#, F#, pick your poison—but that wouldn’t necessarily hold for a DBE.

Comments closed

Testing Multi-User PBI Row-Level Security in Excel

Published 2023-07-27 by Kevin Feasel

Gilbert Quevauvilliers performs a test:

I have been doing a fair amount of work on Row Level Security (RLS) where the requirements were quite complex to implement.

Once I had completed the implementation of RLS there were two outstanding items I needed to test.

  1. Make sure that the users are only viewing the data they should see.
  2. Ensure that the performance of a selected user is still as fast as possible.

I would like to mention that this certainly can be done using DAX studio to get the query performance stats, I wanted to test this using Excel.

Gilbert had to do a fair amount here, but there’s a nice walkthrough indicating how to do it all.

Comments closed

Backup Buffers in SQL Server

Published 2023-07-27 by Kevin Feasel

Andy Yun continues a series on backup internals:

Welcome to Part 2 of SQL Server Backup Internals. This blog series is a companion piece to my How to Accelerate Your Database Backups for MSSQLTips.com. In Part 1 of this blog series, I introduced the parts of a BACKUP operation.

Now let’s start focusing on performance characteristics. We can impact the performance of a BACKUP operation by making changes to or more of the following:

Click through for that list, as well as an extended analogy on buffer count and max transfer size.

Comments closed

Variety in Power BI Bar Chart Designs

Published 2023-07-27 by Kevin Feasel

Kurt Buhler builds a series of charts from the Power BI bar chart visual:

These are only a few examples; there are many more. Knowing how to create these different bar chart variants is important to address these use-cases. While many of these visuals can be created custom in Deneb or from SVGs, you can also create them in the core visuals. However, it’s important to know the limitations of the core visuals in Power BI, and when this “Macguyvering” of the core visual formatting settings goes too far.

Kurt gives us a template (in pbib format) and explains various options we have around customizing visuals, not just taking what’s built in.

Comments closed

Ghost Records in SQL Server

Published 2023-07-27 by Kevin Feasel

Chad Callihan sees a g-g-ghost record:

Have you ever had to deal with ghost records? While I’ve never had a problem that required intervention, I’ve read about people who have.

Let’s talk about what ghost records are and how you should treat them if they’re giving you problems in your environment.

Click through to learn more about them, including some phone numbers for Catholic priests trained in performing database exorcisms.

Comments closed

The Lakehouse is (Still) Not Enough

Published 2023-07-26 by Kevin Feasel

Nikola Ilic needs more than a lakehouse:

In the previous parts of the Data Modeling for mere mortals series, we examined traditional approaches to data modeling, with focus on dimensional modeling and Star schema importance for business intelligence scenarios. Now, it’s time to introduce the concept of the modern data platform.

As usual, let’s take a more tool-agnostic approach and learn about some of the key characteristics of the modern data estate. Please, don’t mind if I use some of the latest buzzwords related to this topic, but I promise to reduce their usage as much as possible. 

Lakehouses are getting closer to being good enough, but the performance needs to be there, especially if you eventually have virtual data warehouses sitting on top of lakehouse data to deal with the need for structured fact-dimensional data for reporting tools.

Comments closed