Press "Enter" to skip to content

Day: July 10, 2026

Merging Data into a Fabric Lakehouse via Python Notebook

Published 2026-07-10 by Kevin Feasel

GIlbert Quevauvilliers uses a pure Python notebook:

In this blog post I am going to show you how to use a Fabric Python runtime notebook (This is the notebook which only uses Pure Python functions and consumes significantly lower Capacity Units (CUs)).

The pattern is how to get new data and merge it into an existing Lakehouse table. This ensures that if the notebook is run again data will not be duplicated.

Why I am sharing this is I have found that there is not a lot of useful information about how to use a Python notebook to write to a lakehouse table easily. And then also how to use a Merge statement making it easier to insert or update your lakehouse tables. This simplifies the ingestion process, runs faster and consumes the least amount of CUs

Gilbert doesn’t mention it in the blog post but the notebook does use DuckDB to query the data using SQL.

Leave a Comment

Partition Pruning on Non-Partitioned Columns in PostgreSQL

Published 2026-07-10 by Kevin Feasel

Haki Benita does a bit of pruning:

One of the most valuable things about partitioned tables is pruning – the database’s ability to eliminate entire partitions based on a query predicate. Under conventional wisdom, pruning can only be achieved when querying by the partition key – this makes choosing the right key extremely difficult. However, if your data follows certain patterns, using some clever tricks you can achieve pruning even when filtering by non-partition key columns.

In this article, I demonstrate how to achieve partition pruning when filtering by non-partition key columns.

Click through to see how.

Leave a Comment

Two Pain Points in SQL Server Code

Published 2026-07-10 by Kevin Feasel

Steve Jones digs up a pair:

There are two things that immediately stand out to me when I see a query and create concern.

  1. cross joins
  2. functions in the where/on clause

While there are other things I might see, these two stand out and usually I can guess there will be issues.

Steve specifically calls out ANSI-89 style joins, which are awful. Before continuing, I agree with Steve’s points, but I figure I’d do a “yes, and” for this one.

When it comes to cross joins, there are specific circumstances in which I’ve written cross joins to great effect. The one that comes to mind most readily is when I need to create the raw data that will let you generate a matrix in a reporting tool.

Let’s say you want to know, for each sales territory and month, the number of new customers that month, the prior month, and the next month. The LAG() and LEAD() functions only work on rows, not intervals. Therefore, LAG() on the July row will show you the row prior to July. That might be June or it might be March, depending on how much data you have.

The solution, then, is to get the full set of months you care about from, say, a calendar table. Then, get the full set of sales territories. Cross join those two and you’ll have the entire range of relevant data. You can then left outer join to the actual data and fill in the gaps.

I have an example of this as a demo script on my Analyzing Business Data with T-SQL talk.

Leave a Comment