Press "Enter" to skip to content

Category: Internals

Read Efficiency in PostgreSQL Queries

Published 2026-02-18 by Kevin Feasel

Michael Christofides explains what’s happening under the covers:

A lot of the time in database land, our queries are I/O constrained. As such, performance work often involves reducing the number of page reads. Indexes are a prime example, but they don’t solve every issue (a couple of which we’ll now explore).

The way Postgres handles consistency while serving concurrent queries is by maintaining multiple row versions in both the main part of a table (the “heap”) as well as in the indexes (docs). Old row versions take up space, at least until they are no longer needed, and the space can be reused. This extra space is commonly referred to as “bloat”. Below we’ll look into both heap bloat and index bloat, how they can affect query performance, and what you can do to both prevent and respond to issues.

Read on for a detailed explanation.

Comments closed

Memory-Optimized Storage Structures in SQL Server

Published 2026-02-17 by Kevin Feasel

Hugo Kornelis digs into another storage structure:

After discussing traditional on-disk rowstore storage in part 1 and columnstores in part 2, it is now time to turn our eye towards memory-optimized storage structures in SQL Server.

Memory-optimized storage was introduced in SQL Server 2014, as part of a project that was codenamed “Hekaton” and later renamed to in-memory OLTP. Whereas columnstore indexes were specifically targeted towards large scale analytical work, Hekaton and memory-optimized tables are specifically geared towards high volume OLTP workloads. By fully eliminating locks and latches, and using precompiled machine code where possible, the processing time of transactions is significantly reduced, allowing for throughput numbers that were previously impossible to achieve.

Read on to learn much more about how SQL Server manages memory-optimized data and the types of operations that are permissible on this internal storage.

Comments closed

Transaction ID Locking

Published 2026-02-16 by Kevin Feasel

Hugo Kornelis disentangles two new features in SQL Server 2025:

One of these two features is Transaction ID (TID) Locking. Slated to end the memory waste of thousands of individual row locks, and the concurrency killer of lock escalation. What it is, how does it work, what are the limitations, and do we really get a free lunch?

Click through for the video, though I am firmly wedded to the idea that TANSTAAFL. I say this without spoiling any part of the video.

Comments closed

Identifying a Query in Oracle vs PostgreSQL

Published 2026-02-12 by Kevin Feasel

Kellyn Gorman brings in the usual suspects:

“How does the database identify this query and its execution plan?”

Both Oracle and PostgreSQL answer this question, but I find they do it in very different ways, reflecting fundamentally different design philosophies around optimization, observability, and stability.  As I dive into this rabbit hole once again, I’m going to reflect on how Oracle’s SQL_ID differs from the query_id in PostgreSQL and how two terms that sound so similar (PLAN_HASH_VALUE and query_hash) could be generated so differently, as well as misinterpreted.  I’m guilty of it myself, so it’s a good place to spend some time.

Read on for the answer.

Comments closed

Buffers in PostgreSQL

Published 2026-01-27 by Kevin Feasel

Radim Marek goes deep into buffers:

The work around RegreSQL led me to focus a lot on buffers. If you are a casual PostgreSQL user, you have probably heard about adjusting shared_buffers and followed the good old advice to set it to 1/4 of available RAM. But after we went a little bit too enthusiastic about them on a recent Postgres FM episode I’ve been asked what that’s all about.

Buffers are one of those topics that easily gets forgotten. And while they are a foundation block of PostgreSQL’s performance architecture, most of us treat them as a black box. This article is going to attempt to change that.

Read on to learn more about how PostgreSQL users buffers.

Comments closed

How Rowgoals Work in SQL Server

Published 2026-01-19 by Kevin Feasel

Hugo Kornelis has a new video:

For my second vlog, I decided to talk about rowgoals. First an explanation of what they are, then an overview of some obvious and some not so obvious cases where the optimizer will use a rowgoal, and finally a warning about cases where this normally beneficial feature might hurt instead of help.

Click through for part one of a new video series.

Comments closed

Investigating Hash Match Spills to tempdb

Published 2025-12-30 by Kevin Feasel

Hugo Kornelis digs in when you’re overdrawn at the memory bank:

Finding data in tempdb is hard. Not when the data is in objects we created ourselves, such as temporary tables or variables. They are stored in the internal system tables and reflected in various system dynamic management views. But that changes for internal objects. They are only used by the internal logic of, in this case, the Hash Match operator. There is no advantage to storing them in the system tables. When I explored the internals of tables used by Table Spool, Index Spool, and Window Spool, I found out that Microsoft has indeed not bothered to put anything in the system tables for such internal objects. The Hash Match operator is not different in this regard.

But I still found a way to locate this information.

Hugo explains how, though it does include some contrivances to make life a lot easier. I always love this sort of spelunking deep into the bowels of how things work, and Hugo is definitely on the top shelf when it comes to this kind of work.

Comments closed

Asynchronous Disk I/O in Postgres 18

Published 2025-12-16 by Kevin Feasel

Josef Machytka gives us the skinny:

PostgreSQL 17 introduced streaming I/O – grouping multiple page reads into a single system call and using smarter posix_fadvise() hints. That alone gave up to ~30% faster sequential scans in some workloads, but it was still strictly synchronous: each backend process would issue a read and then sit there waiting for the kernel to return data before proceeding. Before PG17, PostgreSQL typically read one 8kB page at a time.

PostgreSQL 18 takes the next logical step: a full asynchronous I/O (AIO) subsystem that can keep multiple reads in flight while backends keep doing useful work. Reads become overlapped instead of only serialized. The AIO subsystem is deliberately targeted at operations that know their future block numbers ahead of time and can issue multiple reads in advance:

Read on to see some of the consequences of this change, as well as more detail on how it works.

Comments closed

What VACUUM Really Does in Postgres

Published 2025-12-15 by Kevin Feasel

Radim Marek explains:

There is common misconception that troubles most developers using PostgreSQL: tune VACUUM or run VACUUM, and your database will stay healthy. Dead tuples will get cleaned up. Transaction IDs recycled. Space reclaimed. Your database will live happily ever after.

But there are couple of dirty “secrets” people are not aware of. First of them being VACUUM is lying to you about your indexes.

Click through to learn more.

Comments closed

When IQP Features Make Things Worse

Published 2025-12-08 by Kevin Feasel

Rebecca Lewis has a two-parter. First up is a post covering the guard rails available in IQP:

When Microsoft introduced Intelligent Query Processing in SQL Server 2019 and expanded it in SQL Server 2022 and 2025, the message was simple: upgrade, enable the right compatibility level, and the optimizer will quietly make your queries faster. Features like batch mode on rowstore, memory grant feedback, scalar UDF inlining, and parameter-sensitive plan (PSP) optimization all promise “automatic performance.”

But buried in Microsoft’s documentation is a reality worth understanding: Some IQP features can reduce or discontinue feedback when performance becomes unstable. This is intentional. IQP includes guard rails—safety mechanisms that change or stop certain feedback behaviors if they prove counterproductive.

Part two tells us how to figure out if an IQP feature got the works:

Memory Grant Feedback was introduced in SQL Server 2019 and enhanced in SQL Server 2022+. Microsoft documents several plan attributes that reveal how the engine adjusted or suspended feedback. These attributes appear under the MemoryGrantInfo node in the execution plan.

And stay tuned for part three.

Comments closed