Press "Enter" to skip to content

Category: Tools

The Shape of a Shape Profile

Jonathan Stewart continues a series on Shape:

Read it top down:

  • Title and row count. 1 table(s) · 5,000 rows profiled. Check this first. If it’s not the population you meant to profile, nothing below answers your question.
  • Capture line. captured safe (k=5). A safe capture shows statistics and formats only for a sensitive column, and lists a category only if at least 5 rows have it.
  • Table line. 5,000 rows · 8 columns · primary key: order_id. The primary key is detected from the data, not read from a schema.
  • Sampling. read whole: 5,000 rows; adequacy adequate. Shape read every row. Two analyses sample inside: pattern detection on 1,000 rows and distribution fitting on 2,000. The adequacy line says a value that appears in at least 0.06% of rows is seen with 95% probability.

Click through for the demo.

Leave a Comment

Updates to sp_Check Tools

Jeff Iannucci has a slew of updates:

We’ve been busy updating our suite of sp_Check tools, but…we forgot to tell you about that. And by we I mean me, so this post from me is to rectify that issue.

Speaking of issues, we have added a bunch of new issues to check for in sp_CheckSecurity recently. Fabiano Amorim and Andreas Wolter have been doing great work writing about SQL Server security issues, and we have tried to keep up with their findings by including new checks in sp_CheckSecurity. We would HIGHLY recommend updating your copy of that tool if you haven’t done so recently.

Click through for the full set of changes.

Leave a Comment

Shape: A Tool for Data Generation

Jonathan Stewart has a new open-source tool:

Wait, what? Didn’t you already build one of these?

I built a data generator, yes. But when I realized what profiling the data could open up, the ideas just kept coming. “With this data I could do this.” Then something else. Then something else.

The generator was only half the picture. Before I could make data behave the way I needed, I had to understand how it behaved in the first place.

Your schema tells you column names and types. It doesn’t tell you that December might bring in nearly three times February’s orders, that a small group of customers might place most of them, or that a field’s null rate might jump between two captures.

Click through for the intro article and check out the GitHub repo as well as a playground where you can try it in your browser.

Leave a Comment

Using Screencast Mode in VSCode

Mike Robbins shares some good advice:

If you’ve ever watched a live demo where the presenter says “I’ll just run this” and code appears out of nowhere, you know how easy it is to lose the audience. They can’t see which keys you pressed, which command you ran, or where you clicked. Visual Studio Code (VS Code) has a built-in feature that solves this problem: screencast mode.

I learned about screencast mode from Steven Judd while we were teaching PowerShell together in the OnRamp program.

It’s easy enough to turn on before demos and off after the presentation. Between that and Ctrl +/- to zoom in and out, Visual Studio Code is very convenient for demonstrations.

Leave a Comment

Connection Pooling with Microsoft.Data.SqlClient

Malcolm Daigle has an update:

You told us that applications using Microsoft.Data.SqlClient can take too long to establish the database connections they need. When many requests need connections at the same time, waiting for the connection pool to grow can delay application readiness and increase latency. This can happen even in common situations like querying several pieces of metadata during startup or receiving a burst of traffic after a quiet period. Microsoft.Data.SqlClient’s new connection pool (Pool V2) is designed to address this problem by establishing new connections concurrently, allowing the pool to respond more quickly when multiple requests need new connections at the same time.

Click through to see how this differs from prior behavior, and to see how well it performs.

Comments closed

Updates to the VSCode R Extension

Chris Brown notes some changes:

I was doing some analysis work with R today (don’t get to do that very often anymore) and R wasn’t playing with VS Code nicely.

Turns out there has recently been a major update to the VS Code R editor extension.

So if you are having issues, check out the Extensions page for setup instructions.

Click through for some of the major changes and for some thoughts on Positron versus base VSCode. H/T R-Bloggers.

Comments closed

Data API Builder 2.1.5 Updates

Carlos Robles, et al, announce a new version of Data API Builder:

Data API builder (DAB) 2.1.5 is now available as a stable release, and it is about meeting modern data where it lives: documents next to rows, embeddings next to both. This release brings native support for the SQL json and vector data types to the REST and GraphQL endpoints, so the same entities that serve your CRUD traffic can now store and expose document-shaped data and embeddings without custom code. It also introduces DAB as an embeddable NuGet library, hardens the MCP endpoint, moves the engine to .NET 10, and ships a set of security and reliability improvements.

Click through to see what’s new.

Comments closed

Queues in Postgres

Jobin Augustine builds a queue:

In many real production databases, we often see some tables receiving high-frequency updates (millions per day) and autovacuum running repeatedly (hundreds of times per day). Additionally, those tables can sometimes become heavily bloated. The end effect is poor performance, poor concurrency, and an unmanageable database.

In case we are using methods pg_gather for diagnosis, such anomalies are easy to spot by clicking on the table header to sort.

Read on to learn more about the PgQue extension and how it works.

Comments closed

Tracking Object History in Flyway Desktop

Steve Jones looks at an update to Flyway:


It’s a small change, but a handy one. Flyway Desktop (FWD) now includes the object history for different schema changes, so as you are evaluating how your changes might fit in with others, or you are trying to determine where something broke, you can see a list of historical changes. This post looks at checking history quickly in FWD.

Click through for a demo.

Comments closed

Building an Azure DevOps Extension for Microsoft Fabric

Kevin Chant gives us the rundown:

When I first released the extension, I decided to keep the source code private. Having invested significant time developing and testing it, I wanted to first establish the extension.

Now that the source code is available in a GitHub repository I decided to share my experiences to help others create their own Azure DevOps extensions. Because there is more to creating an extension then just modifying the source code.

Click through for Kevin’s lessons learned.

Comments closed