Press "Enter" to skip to content

Category: Architecture

PostgreSQL HOT Tables and TPC-C Workloads

Published 2026-08-05 by Kevin Feasel

Avinash Vallarapu tunes a PostgreSQL implementation of the TPC-C workload:

Vacuum is almost always presented as a pain point, a culprit, and an over exaggerated source of performance problems in PostgreSQL. The MVCC implementation in PostgreSQL is different from Oracle, SQL Server, MySQL or MariaDB, and that implementation introduces two requirements of its own, freezing transaction IDs, which is largely seamless, and clearing dead tuples through the various forms of vacuum. At the same time the PostgreSQL community is far ahead in minimizing the impact of vacuum. Each release has introduced enhancements substantial enough that most users never realize vacuum is something they could tune at all, and the list of vacuum specific improvements is long enough to deserve an article of its own. Among all of those optimizations, one of the most often missed is how to avoid the need for vacuuming in the first place. That is achievable most of the time, and it is not new.

Pavan Deolasee worked on the idea through 2006 and 2007 and authored the concept of the Heap Only Tuple, or HOT. Simon Riggs, Heikki Linnakangas, Tom Lane and many other PostgreSQL core team members and contributors have written a great deal of enhancement around it since. In this article we look at what PostgreSQL HOT updates actually are, how fillfactor decides whether they succeed, and how we identify which tables benefit. We then put it to the test with a HammerDB benchmark using the HammerDB TPROC-C workload against PostgreSQL 18.4, six 60-minute runs across three dataset sizes, and the improvement from correctly applied PostgreSQL HOT updates is substantial.

Read on for an overview of HOT tables and the results of this experiment.

Comments closed

The Medallion Architecture’s Silver Layer: Physical or Virtual?

Published 2026-08-03 by Kevin Feasel

Nikola Ilic noodles a concept:

If you’ve been working with data platforms in recent years, chances are that you’ve implemented the medallion design pattern (or architecture:)) at least once. Bronze, silver, gold – raw, cleansed, curated – this design pattern has become so widely adopted that you’ll find it in every reference architecture, every certification exam, and every conference talk (including some of mine, I have to admit). And, in the vast majority of implementations I’ve seen (and built myself), all three layers are physically materialized as tables.

In this article, I’d like to challenge that habit. Not the medallion pattern itself – the logical separation of layers is, in my opinion, still of paramount importance. What I want to question is something much more specific: why is the silver layer a set of physical tables? And, with the recent announcement of GPU acceleration for the Fabric Data Warehouse, I believe this question deserves a serious answer, rather than “because that’s how the diagram looks”

Click through for Nikola’s thoughts, including when it might work for the silver layer to be virtual and when it doesn’t make sense.

Comments closed

Understanding the Agent2Agent Object Model

Published 2026-07-30 by Kevin Feasel

Paul Brebner digs into a protocol:

The Agent2Agent (A2A) object model defines the core building blocks that enable AI agents to discover one another, exchange messages, execute long-running work, and deliver durable outputs. The primary A2A objects are agent cards, messages, parts, tasks, and artifacts. Together, they provide a standardized foundation for AI agent interoperability across frameworks, platforms, and programming languages

This post focuses on A2A — specifically the nouns of the protocol: who participates, and what data objects carry meaning. Part 3 will cover how agents discover each other, send work, and deliver updates.

Click through to learn a bit more about the A2A protocol, as well as the major object-level components that make up a solution.

Comments closed

Partition Pruning on Non-Partitioned Columns in PostgreSQL

Published 2026-07-10 by Kevin Feasel

Haki Benita does a bit of pruning:

One of the most valuable things about partitioned tables is pruning – the database’s ability to eliminate entire partitions based on a query predicate. Under conventional wisdom, pruning can only be achieved when querying by the partition key – this makes choosing the right key extremely difficult. However, if your data follows certain patterns, using some clever tricks you can achieve pruning even when filtering by non-partition key columns.

In this article, I demonstrate how to achieve partition pruning when filtering by non-partition key columns.

Click through to see how.

Comments closed

The Muddy World of Hybrid Environments

Published 2026-06-10 by Kevin Feasel

Deborah Melkin muddies up the waters a bit:

My first reaction to this sentence was: Oh look, he assumes that everyone is in the cloud…

My next reaction was: Oh look, he acknowledged that maybe everyone isn’t in the clouds in his follow-up examples.

Since the cloud was introduced, the assumption was always going to be everyone will moving to the cloud. “You’ll be behind if you don’t learn the cloud,” they said. Then as people either moved or started looking into moving their databases, they realized they just couldn’t – whether it was due to missing features, higher than expected costs, etc. There was always some reason.

Click through for some of the challenges and realities of organizations where certain cloud-first or cloud-only services would be a major challenge, versus other services that are typically easier to deal with.

Comments closed

Skills for Cloud-to-On-Prem Migration

Published 2026-06-10 by Kevin Feasel

Reitse Eskens focuses on a set of skills:

This month, Koen Verbeeck invites the blogging community to write about their thoughts on returning to on-premises. What could be struggles, things we have to re-learn, etcetera.

When I read the invite, it immediately sparked inspiration, because there are increasing rumours around cloud exits. People musing about ‘what if’. Some clients reference these questions, but so far no one has directly asked me one with the intent of moving forward with it.

Click through for Reitse’s thoughts.

Comments closed

Choosing between ORMs vs Raw SQL

Published 2026-06-02 by Kevin Feasel

Jamal Hansen embraces the power of “and”:

At the beginning of this series, I promised that even if you know how to use an Object Relational Mapper (ORM) to interact with a database, knowing SQL would make you a better developer. Now that we have covered everything from SELECT to parameterized queries, it is time to answer the question that every Python developer eventually asks: when should I use an ORM, and when should I just write SQL?

I think there’s a reasonable split between the two, though the choice of ORM matters a lot as well. The heavier the ORM (nHibernate and Entity Framework, I’m looking at you), the more performance you’re trading off. By contrast, a micro-ORM like Dapper has a much slimmer profile, and if you combine that with judicious use of stored procedures for non-trivial work, the outcome is usually good.

Comments closed

Polymorphic Associations in Postgres

Published 2026-05-29 by Kevin Feasel

Andrei Lepikhov has multiple types:

Planning such a query efficiently is no easy task — and in my experience, this is confirmed by user reports from the 1C world, since PostgreSQL is currently not rich in LEFT JOIN optimisations. At the same time, the properties of this pattern enable the development of various techniques to improve execution efficiency. I’ve managed to implement several straightforward optimisations of this template. But first, let’s understand what polymorphic references actually are, where they come from, and how common they really are. That’s the gap I’m trying to fill with this post.

Click through for the explanation. This isn’t the easiest problem to solve in the relational world, though I do tend to prefer the subclass/superclass solution, myself.

Comments closed

Foreign Tables and Materialized Views in PostgreSQL

Published 2026-05-26 by Kevin Feasel

Richard Yen provides a write-back system for analysts:

I recently wrote a post about WAL log shipping and how a standby built on log shipping is a great way to give data analysts production data without putting the primary at risk. Having access to the production data in this way is great, but it’s read-only. How can we create views of this data for better analytics work? I want to make the case today that Foreign Data Wrappers and Materialized Views can make a great solution – not only in accessing production Postgres data, but also working with other data sources.

Click through for an architectural discussion of how they can work together.

Comments closed

Explaining the Fabric Ontology

Published 2026-05-21 by Kevin Feasel

James Serra takes us through a big word:

For years, most data conversations have started with tables. We ask where the data lives, what columns are available, how the joins work, and whether the data is in a warehouse, lakehouse, semantic model, or some other system. That makes sense, because tables are how most of us have worked with data for decades. But tables are not how the business thinks.

A business thinks in terms of customers, products, orders, shipments, assets, flights, runways, employees, policies, and actions. The problem is not usually a lack of data. The problem is a lack of shared meaning. Organizations often have the same business concept represented multiple ways across teams and systems, creating what I would call semantic drift. Sales may define a customer one way. Finance may define it another way. Operations may have yet another version in a different system with different keys, names, and assumptions. That is exactly where Fabric Ontology becomes important. It is designed to close the gap between physical data structures and business meaning.

Microsoft is a bit late to the ontology game and their current concept of an ontology shows. I can understand where they’re going but they still have a ways to go.

Comments closed