A Fine Slice Of SQL Server
Julie Behrens, via Kevin Hill, covers the concept of virtual log files:
It is especially evident there is an issue with VLFs when SQL Server takes a long time to recover from a restart. Other symptoms may be slowness with autogrowth, log shipping, replication, and general transactional slowness. Anything that touches the log file, in other words.
Read on to understand how to figure out if you have a problem with virtual log file counts and a resolution.
Comments closedAjay Gupta clues us in on a process:
What Is a Straggler in a Spark Application?
A straggler refers to a very very slow executing Task belonging to a particular stage of a Spark application (Every stage in Spark is composed of one or more Tasks, each one computing a single partition out of the total partitions designated for the stage). A straggler Task takes an exceptionally high time for completion as compared to the median or average time taken by other tasks belonging to the same stage. There could be multiple stragglers in a Spark Job being present either in the same stage or across multiple stages.
Read on to understand the consequences and causes of these straggler tasks, as well as what you can do about them.
Comments closedJess Pomfret contributes to dbatools:
Once I was happy with my settings, I realised we were missing a ‘test’ command for dbatools. The suite of ‘test’ functions in dbatools (a lot that end up as checks in dbachecks btw!), give us an easy way to check our environment against best practices, or our desired settings.
Since dbatools is open-source I was able to write this function (Test-DbaDbQueryStore) and get it added into the module. It’s included as of version 1.0.131, so make sure you’re up to date. Taking Erin’s suggestions and wrapping them in a little PowerShell, I can make it easier for myself and everyone else to make sure we’re following her guidelines.
Click through to see what those settings look like and how you can compare against current settings.
Comments closedAveek Das has an idea for documentation:
In this article, I am going to explain in detail how to document SSIS packages using Sequence Diagrams and the importance of these diagrams in the field of software engineering, no matter which programming language are you using. In my previous article, I have talked about the various UML Diagrams that are being used to document various software engineering processes. Also, I have talked about modular ETL architecture and how to create such a modular package in SSIS. Sequence diagrams are also a part of the broader UML Diagrams which define the interaction between the various components in the system in a chronological manner.
My gut feeling is that this works best with medium-sized collections of packages, where we’re talking 10-30 or so packages in total, and that for something much larger, I’d want an automated tool to build diagrams for me. But I could be way off base on that.
Comments closedAndrew Pruski contrasts techniques:
What is the difference between using a load balanced service and an ingress to access applications in Kubernetes?
Basically, they achieve the same thing. Being able to access an application that’s running in Kubernetes from outside of the cluster, but there are differences!
Read on to learn about the key differences between these two.
Comments closedChris Webb has an interesting problem to solve:
This week a customer came to me with the following problem: they had scheduled the refresh of their dataset but their source data wasn’t always ready in time, so the old data was being loaded by mistake. The best solution here is to use some kind of external service (for example Power Automate) to poll the data source regularly to see if it’s ready, and then to refresh the dataset via the Power BI REST API when it is. However, it got me thinking about a different way of tackling this: is it possible to write some M code that will do the same thing? It turns out that it is, but it’s quite complicated – so I don’t recommend you use the code below in the real world. Nevertheless I wanted to write up the solution I came up with because it’s interesting and you never know, it might be useful one day.
Read on for the less-than-optimal solution, but do check out the better solution Chris describes.
Comments closedAndy Leonard has an update for us:
I’m happy to announce the latest version of our SSIS Framework includes global parameters! I can hear some of you thinking, …
“What Are Global Parameters, Andy?”
I’m so glad you asked! SSIS ships with package-scoped and project-scoped parameters. Project-scoped parameters may be used in any SSIS package in the project; package-scoped parameters are only available within the context of a single SSIS package. This functionality reduces repetition in SSIS package development and execution configuration.
Global parameters allow our SSIS Framework customers to set parameters and values that apply to the entire SSIS Catalog.
Now that you know what they are, Andy has an example of them in action. Global parameters aren’t part of the community edition, but they do look interesting.
Comments closed