Press "Enter" to skip to content

Category: Testing

Power Query Linting for Code Coverage

Published 2026-10-08 by Kevin Feasel

John Kerski has an update to pql-test:

Several years ago, I wrote a blog article introducing the concept of code coverage for semantic models: Part 8: Bringing DataOps to Power BI. With the state of Power BI technology at the time, the bridge was a little far and implementation was quite arduous.

That has changed. With Power BI Project files and User-Defined Functions becoming generally available in 2026, we now have detailed inspection possibilities with what tests exist, what specifically is being tested, and where the gaps in testing live.

I’m happy to announce that version 0.1.18 of pql-test introduces our first attempt at code coverage for semantic models: pql-test code-coverage.

Click through to see what’s new, as well as links to pql-test and more.

Leave a Comment

Automating SQL Server Benchmarking via HammerDB and Docker

Published 2026-09-14 by Kevin Feasel

Anthony Nocentino makes an announcement:

I’m excited to announce the release of a new open-source project that fully automates HammerDB benchmarking for SQL Server using Docker. If you’ve ever needed to run TPC-C or TPC-H benchmarks multiple times, you know how time-consuming the manual setup can be. This project removes the hassle and gets you up and running a single command: ./loadtest.sh.

Click through to learn more and to grab the code.

Comments closed

Test and Validate Azure SQL Database Migrations

Published 2026-07-27 by Kevin Feasel

Marlon Ribunal has a tool:

Introducing azsql-migration-test, a small open-source CLI that validates your Azure SQL Database migrations against a local Azure SQL Database Developer container — the same engine as the cloud, running on your machine.

The problem: proving a migration works shouldn’t require the cloud

The tool is AI-generated and it looks like the blog post is as well, but it does look to be useful.

Comments closed

Snapshot Testing in R

Published 2026-06-25 by Kevin Feasel

Jakub Sobolewski drills into a particular form of testing:

Snapshot testing is not about screenshots.

Most people meet it through UI regression tests: render a component, save a picture, fail the build when the picture changes. So the technique gets filed away as “the thing that compares images.” That is one use. But not the only one.

The mechanic underneath is general. Capture some output, save it to a file, and on every later run compare fresh output against the saved copy. The output can be a plot. It can also be console text, a log, a data frame, an error message, or a deeply nested list. Anything you can serialize, you can snapshot.

Read on to see how you can perform snapshot testing, using examples in R to demonstrate. H/T R-Bloggers.

Comments closed

Trusting Outputs from Fabric Data Agents

Published 2026-06-25 by Kevin Feasel

Jens Vestergaard says don’t trust, do verify:

In two previous posts I went down the path of getting a semantic model ready for AI: descriptions on every measure, an instructions file, the schema tidied up enough that a Fabric data agent has something real to read. That work has a satisfying endpoint. The model looks ready.

Ready is not the same as right.

The kind of evaluation Jens is talking about is fundamental to good business intelligence practices, regardless of whether you throw language models into the mix. Where language models do add complexity is the arbitrary scope of questions, how ambiguous people tend to be when writing, and the stochastic nature of answers. All of that makes the problem harder, though at least it isn’t an entirely different class of problem to solve.

Comments closed

User-Defined Functions and Power BI Testing

Published 2026-06-22 by Kevin Feasel

John Kerski is excited:

User Defined Functions (UDFs) are, in my opinion, the biggest update to Power BI Desktop since PBIP.

That may sound dramatic, but if you care about DataOps, semantic model quality, and reusable development patterns, UDFs fundamentally change what is possible with DAX.

Reuse is one of the core principles of DataOps. For years we have been able to build reusable patterns in Power Query, PowerShell, Python, YAML, and infrastructure automation. But DAX was always missing a key capability: reusable logic that could live inside the semantic model itself.

Until now.

Read on to learn more, as well as to get a link to John’s PQL.Assert DAX unit testing library.

Comments closed

Testing SQL Code in Python

Published 2026-06-09 by Kevin Feasel

Jamal Hansen writes some tests:

I once had a query that ran fine for months. Then someone added a column to the source table and a SELECT * downstream started returning unexpected data. The query didn’t error. It just silently gave wrong results. A test would have caught it immediately.

Schema changes break queries silently. Refactoring a CTE can shift results in ways you don’t notice. New data patterns expose assumptions you didn’t know you made. SQL deserves the same testing discipline as the rest of your code, and Python makes it straightforward.

PyTest, the library Jamal uses here, is one of my favorites for this kind of work. You can build up tests without a lot of ceremony and it’s pretty easy to deal with for most use cases.

Comments closed

Building Automated Tests in Power BI

Published 2026-06-03 by Kevin Feasel

John Kerski has a use for user-defined functions:

Reuse is a very important term in DataOps. It is defined as the practice of leveraging existing components, code, or processes across multiple projects to reduce redundancy and improve consistency.

However, when it comes to Power BI, reusing DAX measures across projects was a difficult ‘copy and paste’ job. For my teams, we used DAX measures to help with testing our semantic models, but ensuring consistent testing conventions (and standard schemas of the tests) required lots of manual review.

Thankfully, that changed in late 2025 when Microsoft introduced User Defined Functions (UDFs) for Power BI. In this article, I’ll demonstrate how to use UDFs for testing, plus how to standardize the way teams test their models.

Click through to see how.

Comments closed

Dimensional Testing in Kafka

Published 2026-05-22 by Kevin Feasel

Jack Vanlightly announces a new tool:

Most of my career in distributed systems has been as a tester, performance engineer and formal verification specialist. I’ve written performance benchmarking tools in the past, for RabbitMQ and Apache Pulsar but in recent years I’ve used OpenMessagingBenchmark (OMB) to run benchmarks against Apache Kafka and other messaging systems. But OMB is hard to deploy and has several limitations compared to more sophisticated benchmarking systems I’ve developed in the past. With Claude becoming so much better since Christmas I decided to write a Kafka-centric performance benchmarking tool, with a lot of inspiration from OMB. I took the bits I like about OMB and the things I like about the tooling I’ve built in the past, to make a performance testing tool for testing Apache Kafka.

Click through for an overview of the tool and how it works.

Comments closed

Migrating testthat to testit

Published 2026-05-18 by Kevin Feasel

Yihui Xie explains how to switch test frameworks in R:

Back in 2013, I wrote about testing R packages when I first released testit. Thirteen years later, I still believe that unit testing should be nothing more than “tell me if something unexpected happened.” Recently I converted a large testthat test suite to testit, and I thought I’d share a practical guide for anyone considering the same move.

Click through for that guide.

Comments closed