Press "Enter" to skip to content

Category: Tools

Orchestrating Data Pipelines in R with maestro

Published 2025-02-27 by Kevin Feasel

Will Hipson moves some data:

If you look at data orchestration tools today you are bombarded with a dizzying array of software platforms that claim unsurpassed processing capability, AI-readiness, elegant UIs, etc. Apache Airflow is just one example of a popular orchestration platform that scales to meet virtually any orchestration need. And while these claims may be true, I argue it is rarely the case that these gargantuan platforms are needed in the first place. For most data engineers, you probably only need to process a moderate amount of data at a moderate time scale. Moreover, if you’re an R user, you don’t want to have to define your data pipelines using drag-and-drop tools or learn another programming language. Not only will this reduce cloud costs but also development time costs.

Click through to see why Will developed maestro and how it works. H/T R-Bloggers.

Comments closed

Concerns with Azure Data Studio’s Deprecation

Published 2025-02-21 by Kevin Feasel

Thom Andrews raises a valid concern:

Some of you may, or may not know, Microsoft announced the deprecation of the Azure Data Studio (ADS) on 06 February, with support ending next year on 28 February. For those of using ADS Microsoft recommends migrating to VSCode and using the mssql extension.

Honestly, I’m really concerned about this. I tried the mssql extension for VSCode (mssql going forwards) some time ago, and compared to ADS it was incredibly feature lacking. A lot of stuff I want to do as a DBA was completely missing, so I went straight back to ADS and didn’t look back. I like ADS, and I’ve got good use out of it; especially as (as those who know me well) a Linux user at home, where I can’t use SSMS.

Anyway, I thought “Ok, I’ll give mssql another go, maybe it’s come a long way since I tried last”. Oh boy has it not; at least for someone like myself.

The upshot of Thom’s post is that the mssql extension is definitely not ready for prime time, and there’s going to be an uphill slog to get it, within one year, back to where Azure Data Studio is today. My hope on this is that, because the mssql extension team (who may be the same people as the Azure Data Studio team) doesn’t need to continuously fork and work around changes to Visual Studio Code, that it will allow them to re-use and re-implement relevant code quicker than otherwise. But if not, that’s a tough story to tell.

Comments closed

An Overview of PostgreSQL Performance Monitoring via pgNow

Published 2025-02-19 by Kevin Feasel

Grant Fritchey announces a product:

I’ve been putting together a new PostgreSQL session called “Performance Monitoring for the Absolute Beginner.” There are several ways to get an understanding of how well your queries are running in PostgreSQL, but, frankly, all of them are a bit of a pain to someone coming from the land of Extended Events (ah, my one true love). Because of this, I saw it as an opportunity to help those just getting going in PostgreSQL. I’ll be presenting it for the first time at Postgres Conference in Orlando on March 19, 2025. Come on by.

Anyhoo, wouldn’t it be nice to maybe have a shortcut, an easier way to look at this information?

Well, there is. Redgate has been working on a completely free tool for leveraging just this sort of data called pgNow. Go here to check it out yourself, but I’ll do a quick run through here.

Click through to see how it works.

Comments closed

Automating Backups with Minion Backup

Published 2024-12-18 by Kevin Feasel

I have a new video:

In this video, we will learn how to use Minion Backup to automate backup operations, including setting backup frequency, dynamic backup tuning, and scripting out database restorations.

This video wraps up a series on backup and recovery in SQL Server. Now I have to come up with some new videos.

Comments closed

Fabric Studio 1.0

Published 2024-12-10 by Kevin Feasel

Gerhard Brueckl makes an announcement:

I am very proud to announce the first public release of Fabric Studio v1.0 – a VSCode extension that allows you to manage and develop your Fabric workspace(s). Similar to Power BI Studio, it seamlessly integrates into VSCode for increased productivity for professional developers and admins alike.

Click through for some of the functionality available in Fabric Studio. You can download the extension from the VS Code marketplace and Gerhard includes a link to the GitHub repo in the blog post.

Comments closed

CPU Monitoring in SQL Server with Datadog

Published 2024-11-22 by Kevin Feasel

Kendra Little has a recommendation:

What makes me a raving fan is the flexibility of Datadog’s notebooks and dashboards, combined with the ability to create all sorts of custom metrics and monitors. There are always things in SQL Server monitoring packs that I have strong opinions about. Datadog lets me take what I want, build what I need that isn’t contained in that, and ignore the rest. For a team that has the budget to afford Datadog paired with dedicated database staff with the time and resources to do this work, this can be a great fit.

One of the weirdest and worst parts of the Datadog SQL Server monitoring tooling, though, is how it handles wait stats. In my opinion, it’s a case of someone reinventing a wheel that didn’t need to be reinvented, and then not documenting what they did clearly (at least not in a way I can find).

Two of the most confusing Datadog “waits” are labeled “CPU” and “Waiting on CPU”. I opened a support ticket with Datadog a while back to ask what these are, because I couldn’t find any way they correspond to actual wait stats in SQL Server. I learned they aren’t wait stats at all. In fact, I think you should largely ignore them. Here’s why.

Read on for the full story.

Comments closed

A Review of Stellar Repair for MSSQL

Published 2024-11-21 by Kevin Feasel

Mika Sutinen shares a product review:

Stellar Repair for MS SQL is a tool that greatly simplifies one of all time most dreaded tasks of DBAs and DBREs. Recovering a database that has, for one reason or another, become corrupt.

In this post, I go through one of the more common types of database corruption scenarios, and using Stellar Repair for MS SQL to get the database back up and running.

Click through for the review.

Comments closed

Configuring SQL Server Alerts in SQLMonitor

Published 2024-11-07 by Kevin Feasel

Ajay Dwivedi has an update to SQLMonitor:

If you are responsible for managing & monitoring SQLServers, then it is mandatory to have monitoring and alerting for critical issues of your SQL Servers.

For this purpose, open source SQLMonitor now has built-in Alert Engine. With the capability of this alert engine, alerts can be sent to Slack & Email to various teams. The following are some advantages –

Read on to see some of those capabilities and how to set things up.

Comments closed