Press "Enter" to skip to content

Category: Tools

The Design of DataDiluvium

Published 2025-03-24 by Kevin Feasel

Adron Hall wraps up a series on DataDiluvium. First up is some commentary on data generation:

In Parts 1 and 2, I set up the development environment and implemented the schema parsing functionality. Now, I’ll explore the data generation system and final implementation details that make DataDiluvium a complete solution.

And finally we have some additional notes and wrapup:

In my previous three posts, I covered the core functionality of DataDiluvium. In this follow-up post, I’ll explore the additional features, utilities, and implementation details that I’ve added to enhance the application’s functionality and developer experience.

Check out both posts. I do enjoy seeing people walk through and explain some of the key concepts and decisions they’ve made when developing solutions.

Leave a Comment

An Overview of DataDiluvium

Published 2025-03-20 by Kevin Feasel

Adron Hall has a new tool and a new blog series. The first post is a product overview:

DataDiluvium is a web-based tool available at datadiluvium.com that helps developers, database administrators, and data engineers generate realistic test data from SQL schema definitions. Whether you’re setting up a development environment, creating test scenarios, or preparing data for demonstrations, DataDiluvium streamlines the process of data generation.

The second covers some of the development precepts Adron used:

DataDiluvium is a web-based tool I’ve built designed to help developers, database administrators, and data engineers generate realistic test data based on SQL schema definitions. The tool takes SQL table definitions as input and produces sample data in various formats, making it easier to populate development and testing environments with meaningful data.

The tool is free, so if you’re looking for a sample data generator, check it out.

Leave a Comment

Using the Excel Solver with Power Pivot

Published 2025-03-18 by Kevin Feasel

Chris Webb remembers a few functions:

After years of meaning to learn how to use Excel Solver, this week I’ve finally made a start: I want to use it to work out the optimal way of distributing workspaces across capacities in a Fabric tenant from the point of view of CU usage. I’m a long way from knowing how to do this properly (I’ll blog about it when I’m ready) but one of the first things I found is that while there are lots of resources on the internet showing how to use Solver, there are no examples of how to use Solver when your source data is stored in the Excel Data Model, aka Power Pivot. Getting that data onto the worksheet is fairly straightforward, but what if you also need Solver to change how that data is sliced and diced? It turns out that not hard to do if you know how to use cube functions.

Read on to see what Chris means and how you can send data from the Excel Data Model into the Solver.

Leave a Comment

An Introduction to pgNow

Published 2025-03-11 by Kevin Feasel

Ryan Booz introduces a new free tool from Redgate:

pgNow is a free, cross-platform desktop tool created by Redgate that helps you identify key performance metrics and configuration optimizations in your running Postgres instance. Available now as a public preview application, it’s designed to help when you’re in a pinch and don’t have the Postgres experience or monitoring solution already in place to help identify why your server or database is experiencing a degradation in performance. Even in its current preview offering, pgNow is a helpful front-line tool for troubleshooting your Postgres cluster. And I couldn’t be more excited to share it with you.

Click through to see what’s in the preview. Ryan has mentioned that the tool will remain free even when it’s in a stable release.

Leave a Comment

Visualizing PostgreSQL Query plans

Published 2025-03-06 by Kevin Feasel

MIchael Christofides views a plan:

When you’ve got a slow Postgres query, EXPLAIN and its parameters are incredibly useful for working out why.

However, the information returned can be difficult and time-consuming to interpret, especially for more complex queries. Over the years, people have built quite a few tools for visualizing Postgres query plans. As one of those people, I’m a little incredibly biased, but as a fan of many of the others, I hope to do them justice.

Read on for the options. I think the SQL Server world is a bit spoiled with Solarwinds Plan Explorer (even if the product hasn’t really changed since Solarwinds bought Sentry One), but some of these options look very solid.

Comments closed

Orchestrating Data Pipelines in R with maestro

Published 2025-02-27 by Kevin Feasel

Will Hipson moves some data:

If you look at data orchestration tools today you are bombarded with a dizzying array of software platforms that claim unsurpassed processing capability, AI-readiness, elegant UIs, etc. Apache Airflow is just one example of a popular orchestration platform that scales to meet virtually any orchestration need. And while these claims may be true, I argue it is rarely the case that these gargantuan platforms are needed in the first place. For most data engineers, you probably only need to process a moderate amount of data at a moderate time scale. Moreover, if you’re an R user, you don’t want to have to define your data pipelines using drag-and-drop tools or learn another programming language. Not only will this reduce cloud costs but also development time costs.

Click through to see why Will developed maestro and how it works. H/T R-Bloggers.

Comments closed

Concerns with Azure Data Studio’s Deprecation

Published 2025-02-21 by Kevin Feasel

Thom Andrews raises a valid concern:

Some of you may, or may not know, Microsoft announced the deprecation of the Azure Data Studio (ADS) on 06 February, with support ending next year on 28 February. For those of using ADS Microsoft recommends migrating to VSCode and using the mssql extension.

Honestly, I’m really concerned about this. I tried the mssql extension for VSCode (mssql going forwards) some time ago, and compared to ADS it was incredibly feature lacking. A lot of stuff I want to do as a DBA was completely missing, so I went straight back to ADS and didn’t look back. I like ADS, and I’ve got good use out of it; especially as (as those who know me well) a Linux user at home, where I can’t use SSMS.

Anyway, I thought “Ok, I’ll give mssql another go, maybe it’s come a long way since I tried last”. Oh boy has it not; at least for someone like myself.

The upshot of Thom’s post is that the mssql extension is definitely not ready for prime time, and there’s going to be an uphill slog to get it, within one year, back to where Azure Data Studio is today. My hope on this is that, because the mssql extension team (who may be the same people as the Azure Data Studio team) doesn’t need to continuously fork and work around changes to Visual Studio Code, that it will allow them to re-use and re-implement relevant code quicker than otherwise. But if not, that’s a tough story to tell.

Comments closed

An Overview of PostgreSQL Performance Monitoring via pgNow

Published 2025-02-19 by Kevin Feasel

Grant Fritchey announces a product:

I’ve been putting together a new PostgreSQL session called “Performance Monitoring for the Absolute Beginner.” There are several ways to get an understanding of how well your queries are running in PostgreSQL, but, frankly, all of them are a bit of a pain to someone coming from the land of Extended Events (ah, my one true love). Because of this, I saw it as an opportunity to help those just getting going in PostgreSQL. I’ll be presenting it for the first time at Postgres Conference in Orlando on March 19, 2025. Come on by.

Anyhoo, wouldn’t it be nice to maybe have a shortcut, an easier way to look at this information?

Well, there is. Redgate has been working on a completely free tool for leveraging just this sort of data called pgNow. Go here to check it out yourself, but I’ll do a quick run through here.

Click through to see how it works.

Comments closed