Press "Enter" to skip to content

Curated SQL Posts

Checking Percentage of Physical Memory SQL Server is Using

Mike Hays does the math:

Note: Unlike other queries I have share here at TheSQLReport, I am the author of this one.   Sharing because I could not find it in my Google Searches.  Tested this back to version SQL Server 2012.   Also please remember that physical_memory_in_use_kb may be a constantly changing number depending on how SQL Server  is configured & the activity of the operating system.

Click through for the query and an example of it in action.

Comments closed

A Primer on Power BI Apps

Melissa Coates offers up an explanation:

If you hear the question: “What’s a Power BI app?” you might think there’s a straightforward answer. However, the term ‘app’ is pretty overloaded term in the world of Power BI.

One time I was presenting a session and talking about Power BI apps vs. workspaces – right in the middle, someone in the audience starts asking me a Power Apps question because they thought I was talking about that (which is a VERY different thing). That was a big reminder to not to assume that we all have the same understanding of terms.

Read on to learn what a Power BI app is, as well as why it’s a rather useful concept.

Comments closed

Data Mesh Q&A Round 2

Jean-Georges Perrin didn’t hear no bell:

How does the Data Mesh concept differ from similar efforts in the past, like EDM (Enterprise Data Model) or MDM (Master Data Model)?
Data Mesh will help us achieve those goals more quickly as those EDM and MDM projects are usually slow, and the ROI starts showing only after deployment. The product approach of Data Mesh for its data products enables a product lifecycle mentality that will help get from a current state to an (end?) state like EDM through versioning. It also allows EDM to be versioned more efficiently and reduces time to market.

Read on for a series of questions and answers around the topic of data mesh architecture.

Comments closed

Deploying a Database via Azure DevOps Pipeline

Olivier Van Steenlandt deploys a database:

After we successfully introduced a database development strategy in my previous blog post series, Getting Started With Database Projects & Azure DevOps, we can look at how to introduce a database deployment automation strategy using Database Projects and Azure DevOps Pipelines.

As a starter, we will first be implementing a build automation process and in future blog posts, we will go through the different ways of deployment to different environments. On top of that, we will also discuss the differences between SQL Server and AzureSQL database deployments.

Read on for the full story.

Comments closed

Lateral Joins in Snowflake

Kevin Wilkie makes a lateral move:

In Snowflake, you can do something akin to an APPLY, but not. That, my friends, is a LATERAL join. According to Snowflake documentation, this type of join “allows an inline view to reference columns from the table expression that precedes the inline view.”

Yes, that’s gibberish. Let’s show what you’re doing.

I’m such an APPLY apologist that the description made sense to me.

Comments closed

February 2023 Updates for Azure Synapse Analytics

Ryan Majidimehr has a new round-up for us:

Azure Synapse Runtime for Apache Spark 3.3 has been in Public Preview since November 2022. We are excited to announce that after notable improvements in performance and stability, Azure Synapse Runtime for Apache Spark 3.3 now becomes Generally Available and ready for production workloads.   

The essential changes include features that come from upgrading Apache Spark to version 3.3.1, Delta Lake to version 2.2.0, and Python to 3.10. 

This month’s set of changes isn’t quite as big as some prior months, though there are a couple items of great importance to make up for it.

Comments closed

The Importance of Monitoring Tools

Louis Davidson talks turkey about tooling:

When I was a DBA involved with the management of a large number of database servers, I didn’t have many third-party tools to help me do my job. For the most part, I relied on scripts that I found or wrote. I enjoyed writing scripts to manage the servers, as it taught me a lot about the internals of SQL Server. Many of these scripts were eventually automated using SQL Server’s agent to run and save data on the different servers so we could review the results, looking for issues.

Some of these tools written over 20 years ago still run to this day. We captured tons of data about everything we wanted to know about the server in case there were issues. Loads and loads of data. We had some processes that would scan that data and send emails when obvious errors occurred, but it was hard to keep synchronized over many different servers.

Click through for Louis’s thoughts. I believe good tools can make a DBA’s life a lot easier, though mediocre tools might make it worse: you become the proverbial drunk looking for his keys under a streetlamp because that’s where the light is.

Comments closed

Combining CSV Files via Powershell

Chad Callihan smooshes files together:

I recently had a handful of CVS files that needed reviewed. Each CSV file was the same format, and while I could have opened them each individually to sort and review, I thought it would be much easier to combine them into one file. It was time to turn to PowerShell. Let’s look at a few examples of how PowerShell can be used to combine multiple CSV files into a single file.

A core assumption here is that the structure of each file—particularly the number of columns but also the semantic meaning of each column—is the same.

Comments closed

pmap and imap Examples in purrr

Steven Sanderson has a multi-parter for us. First up is a look at the pmap() function in R’s purrr library:

The pmap() function in R is part of the purrr library, which is a package designed to make it easier to work with functions that operate on vectors, lists, and other types of data structures.

The pmap() function is used to apply a function to a list of arguments, where each element in the list contains the arguments for a single function call. The function is applied in parallel, meaning that each call is executed concurrently, which can help speed up computations when working with large datasets.

Next up is the imap() function:

The imap() function is a powerful tool for iterating over a list or a vector while also keeping track of the index or names of the elements. This function applies a given function to each element of a list, along with the name or index of that element, and returns a new list with the results.

The imap() function takes two main arguments: x and .fx is the list or vector to iterate over, and .f is the function to apply to each element. The .f function takes two arguments: x and i, where x is the value of the element and i is the index or name of the element.

Both of these sound a little complex and abstract at first, though as you get more familiar with them, you get to see just how powerful they are.

Comments closed