Press "Enter" to skip to content

Category: Architecture

Tips on When to Use Microsoft Fabric Shortcuts

Published 2026-08-10 by Kevin Feasel

James Serra offers up some guidance:

Imagine separate Sales, Finance, Shared Data, and Executive Analytics workspaces. Sales owns sales transactions, Finance owns budgets, and Shared Data owns common tables such as Customer, Product, and Date. Executive Analytics needs selected data from all three, but it does not want to copy everything into another lakehouse and maintain another set of pipelines. Instead, it creates shortcuts to the authoritative tables and presents them together in its own lakehouse.

This is where the beauty of Fabric OneLake shortcuts becomes obvious. To a report developer or analyst, the Executive Analytics lakehouse can look like one complete collection of tables. That person might not even know which tables are physically stored there and which are shortcuts—and usually should not need to know. Fabric resolves those paths behind the scenes, which is one reason I call Fabric “the great data unifier”: it can present one logical data estate without forcing all the data into one physical location.

Click through to learn more about what shortcuts are, how they work, and when you should (or should not) use them.

Leave a Comment

Drilling into Separate Fact Tables via Detail Rows in Power BI

Published 2026-08-10 by Kevin Feasel

Chris Webb crosses fact tables:

If you have a DirectQuery fact table in Power BI you can use user-defined aggregations to improve query performance; querying a smaller, summarised copy of your data in an Import mode aggregation table is always going to be faster than querying a large fact table containing all your detail data that is in DirectQuery mode. What’s more a composite model like this can have a much smaller footprint in memory than a model where all your tables are in Import or Direct Lake mode, which means you can use a smaller Fabric capacity SKU. However, in some cases you can take the same tables that you would use to create a composite model like this and solve the same problem slightly differently without using aggregations.

Click through for an example of this.

Leave a Comment

PostgreSQL HOT Tables and TPC-C Workloads

Published 2026-08-05 by Kevin Feasel

Avinash Vallarapu tunes a PostgreSQL implementation of the TPC-C workload:

Vacuum is almost always presented as a pain point, a culprit, and an over exaggerated source of performance problems in PostgreSQL. The MVCC implementation in PostgreSQL is different from Oracle, SQL Server, MySQL or MariaDB, and that implementation introduces two requirements of its own, freezing transaction IDs, which is largely seamless, and clearing dead tuples through the various forms of vacuum. At the same time the PostgreSQL community is far ahead in minimizing the impact of vacuum. Each release has introduced enhancements substantial enough that most users never realize vacuum is something they could tune at all, and the list of vacuum specific improvements is long enough to deserve an article of its own. Among all of those optimizations, one of the most often missed is how to avoid the need for vacuuming in the first place. That is achievable most of the time, and it is not new.

Pavan Deolasee worked on the idea through 2006 and 2007 and authored the concept of the Heap Only Tuple, or HOT. Simon Riggs, Heikki Linnakangas, Tom Lane and many other PostgreSQL core team members and contributors have written a great deal of enhancement around it since. In this article we look at what PostgreSQL HOT updates actually are, how fillfactor decides whether they succeed, and how we identify which tables benefit. We then put it to the test with a HammerDB benchmark using the HammerDB TPROC-C workload against PostgreSQL 18.4, six 60-minute runs across three dataset sizes, and the improvement from correctly applied PostgreSQL HOT updates is substantial.

Read on for an overview of HOT tables and the results of this experiment.

Leave a Comment

The Medallion Architecture’s Silver Layer: Physical or Virtual?

Published 2026-08-03 by Kevin Feasel

Nikola Ilic noodles a concept:

If you’ve been working with data platforms in recent years, chances are that you’ve implemented the medallion design pattern (or architecture:)) at least once. Bronze, silver, gold – raw, cleansed, curated – this design pattern has become so widely adopted that you’ll find it in every reference architecture, every certification exam, and every conference talk (including some of mine, I have to admit). And, in the vast majority of implementations I’ve seen (and built myself), all three layers are physically materialized as tables.

In this article, I’d like to challenge that habit. Not the medallion pattern itself – the logical separation of layers is, in my opinion, still of paramount importance. What I want to question is something much more specific: why is the silver layer a set of physical tables? And, with the recent announcement of GPU acceleration for the Fabric Data Warehouse, I believe this question deserves a serious answer, rather than “because that’s how the diagram looks”

Click through for Nikola’s thoughts, including when it might work for the silver layer to be virtual and when it doesn’t make sense.

Leave a Comment

Understanding the Agent2Agent Object Model

Published 2026-07-30 by Kevin Feasel

Paul Brebner digs into a protocol:

The Agent2Agent (A2A) object model defines the core building blocks that enable AI agents to discover one another, exchange messages, execute long-running work, and deliver durable outputs. The primary A2A objects are agent cards, messages, parts, tasks, and artifacts. Together, they provide a standardized foundation for AI agent interoperability across frameworks, platforms, and programming languages

This post focuses on A2A — specifically the nouns of the protocol: who participates, and what data objects carry meaning. Part 3 will cover how agents discover each other, send work, and deliver updates.

Click through to learn a bit more about the A2A protocol, as well as the major object-level components that make up a solution.

Leave a Comment

Partition Pruning on Non-Partitioned Columns in PostgreSQL

Published 2026-07-10 by Kevin Feasel

Haki Benita does a bit of pruning:

One of the most valuable things about partitioned tables is pruning – the database’s ability to eliminate entire partitions based on a query predicate. Under conventional wisdom, pruning can only be achieved when querying by the partition key – this makes choosing the right key extremely difficult. However, if your data follows certain patterns, using some clever tricks you can achieve pruning even when filtering by non-partition key columns.

In this article, I demonstrate how to achieve partition pruning when filtering by non-partition key columns.

Click through to see how.

Comments closed

The Muddy World of Hybrid Environments

Published 2026-06-10 by Kevin Feasel

Deborah Melkin muddies up the waters a bit:

My first reaction to this sentence was: Oh look, he assumes that everyone is in the cloud…

My next reaction was: Oh look, he acknowledged that maybe everyone isn’t in the clouds in his follow-up examples.

Since the cloud was introduced, the assumption was always going to be everyone will moving to the cloud. “You’ll be behind if you don’t learn the cloud,” they said. Then as people either moved or started looking into moving their databases, they realized they just couldn’t – whether it was due to missing features, higher than expected costs, etc. There was always some reason.

Click through for some of the challenges and realities of organizations where certain cloud-first or cloud-only services would be a major challenge, versus other services that are typically easier to deal with.

Comments closed

Skills for Cloud-to-On-Prem Migration

Published 2026-06-10 by Kevin Feasel

Reitse Eskens focuses on a set of skills:

This month, Koen Verbeeck invites the blogging community to write about their thoughts on returning to on-premises. What could be struggles, things we have to re-learn, etcetera.

When I read the invite, it immediately sparked inspiration, because there are increasing rumours around cloud exits. People musing about ‘what if’. Some clients reference these questions, but so far no one has directly asked me one with the intent of moving forward with it.

Click through for Reitse’s thoughts.

Comments closed

Choosing between ORMs vs Raw SQL

Published 2026-06-02 by Kevin Feasel

Jamal Hansen embraces the power of “and”:

At the beginning of this series, I promised that even if you know how to use an Object Relational Mapper (ORM) to interact with a database, knowing SQL would make you a better developer. Now that we have covered everything from SELECT to parameterized queries, it is time to answer the question that every Python developer eventually asks: when should I use an ORM, and when should I just write SQL?

I think there’s a reasonable split between the two, though the choice of ORM matters a lot as well. The heavier the ORM (nHibernate and Entity Framework, I’m looking at you), the more performance you’re trading off. By contrast, a micro-ORM like Dapper has a much slimmer profile, and if you combine that with judicious use of stored procedures for non-trivial work, the outcome is usually good.

Comments closed

Polymorphic Associations in Postgres

Published 2026-05-29 by Kevin Feasel

Andrei Lepikhov has multiple types:

Planning such a query efficiently is no easy task — and in my experience, this is confirmed by user reports from the 1C world, since PostgreSQL is currently not rich in LEFT JOIN optimisations. At the same time, the properties of this pattern enable the development of various techniques to improve execution efficiency. I’ve managed to implement several straightforward optimisations of this template. But first, let’s understand what polymorphic references actually are, where they come from, and how common they really are. That’s the gap I’m trying to fill with this post.

Click through for the explanation. This isn’t the easiest problem to solve in the relational world, though I do tend to prefer the subclass/superclass solution, myself.

Comments closed