Press "Enter" to skip to content

Day: July 23, 2024

Telegraf Performance Optimization

Published 2024-07-23 by Kevin Feasel

Riya shares a few tips on making Telegraf stream data more efficiently:

As businesses grow and their infrastructures become more complex, monitoring becomes a critical component of maintaining system health and performance. Telegraf, an open-source server agent for collecting and sending metrics and events from databases, systems, and IoT sensors, is widely used for this purpose. However, handling high volumes of metrics can strain resources and degrade performance. This blog will explore strategies for optimizing Telegraf’s performance when dealing with high-volume metrics.

Click through for an architectural overview and five things you can do to optimize performance.

Comments closed

Calling a REST Endpoint from Azure SQL Database

Published 2024-07-23 by Kevin Feasel

Meagan Longoria makes a call:

External REST endpoint invocation in Azure SQL DB went GA in August 2023. Whereas before, we might have needed an intermediate technology to make a REST call to an Azure service, we can now use an Azure SQL Database to call a REST endpoint directly.

One use case for this would be to retrieve a file from blob storage. I explain how to set this up below.

Read on to learn more about the process, as well as a few ideas on when you might use it.

Comments closed

A Glossary of Microsoft Fabric Terms

Published 2024-07-23 by Kevin Feasel

Reza Rad talks terminology:

There are a lot of similar terminologies and words when working with Fabric. We have Data Lake, Delta Lake, OneLake, and Lakehouse, and the list continues. I found it confusing for some to understand the differences between them. Although explaining these terms and their differences requires many different blog articles, having one place for a quick definition for each can be helpful. Hence, this Glossary.

I will keep this updated as we have added new features, tools, and workloads, and I welcome all your input regarding what keywords, terms and terminologies you would like to see added here.

Click through for the video, as well as a list of terms and brief definitions.

Comments closed

The Challenges of Index Cleanup

Published 2024-07-23 by Kevin Feasel

Aaron Bertrand talks index maintenance of a different sort:

I’m not the first person to write about cleaning up unused or redundant indexes. You can read many articles about how non-clustered indexes are expensive to store and maintain, that they can occupy valuable space in memory, and that they can negatively impact the performance of the write portion of any SQL Server workload. There are great scripts out there already – like sp_BlitzIndex – that can help identify index issues without any grunt work. So, nothing new in this area.

But reinforcement can be beneficial.

Aaron raises some good points and explains why simply deleting any index with 0 reads is not necessarily a good idea. All bets are off, however, when you see 45 indexes on the same table with the same dta_ prefix.

Comments closed

Calculating the Size of Dataflow Gen2 Staging Lakehouses

Published 2024-07-23 by Kevin Feasel

Sandeep Pawar busts out the calculator:

My friend Alex Powers (PM, Fabric CAT) wrote a blog post about cleaning the staging lakehouses generated by Dataflow Gen2. Before reading this blog, go ahead and read his blog first on the mechanics of it and the whys. Note that these are system generated lakehouses so at some time in the future, they will be automatically purged but until then the users will be paying the storage cost of these lakehouses. If you want to read more about how dataflow gen2 works and whether you should stage or not , read this and this blog.

Read on for a Python script using the SemPy library.

Comments closed

Creating Elastic Jobs in TerraForm

Published 2024-07-23 by Kevin Feasel

Josephine Bush builds a script:

If you aren’t familiar with Terraform and its requisite bits and pieces, I have a post that walks you through them. Let’s hit the ground running. To create Elastic Jobs with Terraform, you need to use the azapi_resource, but before we go down that path, you will use the azurerm resource to create some supporting objects.

Read on for a guide covering requirements and setup.

Comments closed

Calculation Options in Power BI

Published 2024-07-23 by Kevin Feasel

Marc Lelijveld builds a list:

There are tons of options in Power BI to add your custom logic. Varying from SQL expressions at data ingest all the way up to Visual Calculations to add logic on a visual level. But how do they compare to each other, where should you do what?

As I’ve been a fan of making DAX easier since day one, I thought it may be a good moment to highlight the differences between calculation options using DAX in Power BI, like measures and calculated columns and how that is significantly different with Power BI Visual Calculations and how those can make your DAX easier.

Read on for a half-dozen options and some further thoughts about calculation options.

Comments closed