Press "Enter" to skip to content

Curated SQL Posts

Ways to Convert a List to DataFrame in R

Tomaz Kastrun starts the stopwatch:

When you are working with large datasets performance comes to everyone’s mind. Especially when converting datasets from one data type to another. And choosing the right method can make a huge difference.

So in this case, I will be creating a dummy list, and I will convert the values in the list into data.frame.

Click through for a variety of techniques and how well they performed. There are some good solutions in the comments as well.

Comments closed

KQL Contains and In

Robert Cain continues a series on KQL:

There are versions of these which are case sensitive. We’ll see a few here, focusing on the contains keyword. In addition there are not versions, which will also be demonstrated.

There is another operator we’ll discuss here, in. It is a bit of an odd duck, in that it is case sensitive by default. We’ll see it and its variants later in this post.

Case sensitivity in search is a curse.

Comments closed

Indexing in Postgres

Grant Fritchey provides an overview:

In this article I’m going to go over the different types of indexes and some index behaviors. We’ll get into what the indexes are, how they work, and how best you can apply them within your databases. I’m hoping you’ll develop an understanding of which indexes are likely to work better in each situation.

However, I won’t be covering the indexes in detail. Further, I won’t be going very far into how these indexes affects performance. I’m not yet up on performance tuning within PostgreSQL. That article is quite a way down the track from here. When you know which index to use where, you can then drill down to better understand that index, and how best to measure the performance impacts.

This set of indexes is quite different from what we’re used to in SQL Server and much closer to what you get in Oracle.

Comments closed

A Report of 2022’s Best Power BI Features

Nicky van Vroenhoven goes the extra mile:

Last night we had a work meeting at Powerdobs, with a few presentations about business updates, new joiners in the coming months and Power BI projects. I had the pleasure of closing the evening with a presentation on Power BI.

So I decided to do a presentation on my favourite additions to Power BI over the past year. But instead of doing a regular PowerPoint presentation, I decided to do an all-Power-BI presentation. It was very well received and I was encouraged to share it here!

Click through for the embedded report. Great job by Nicky to put this together.

Comments closed

DB Restoration and Upgrade Management for Arc-Enabled Data Services

Warwick Rudd continues a series on Azure Arc-Enabled Data Services. Part 9 covers database restoration:

Now that we have our Azure Arc-enabled SQL Managed Instance deployed, potentially one of the most common tasks you will need to undertake is being able to perform a database restore of an existing database to your newly provisioned Azure Arc-enabled SQL Managed Instance.

In our previous post, we deployed our Azure Arc-enabled SQL Managed Instance – “ae-sqlmi-demo” that we will continue to investigate and learn about how we can utilize in our data environments. In this post, we are having a look at restoring a database.

Part 10 looks at upgrade management:

Managing your Data Controller through Azure Data Studio provides an easy approach for controlling when you wish to apply upgrades supplied by Microsoft. This is something you need to stay on top of as the upgrade path is an incremental approach and only allows you to upgrade to the next version. If you leave it too long, you could be faced with multiple upgrades to be undertaken. This could lead to extended outages while you get up to date.

Comments closed

Pull Requests and Database Projects

Olivier Van Steenlandt continues building out a database project:

A Pull Request is an alternative way of merging branches. Instead of executing the merge yourself, you will create a Pull Request for someone else to revise your development and approve and merge when ok. By doing this you are introducing peer reviewing into your development process. From my experience, using Pull Requests will improve your development quality since someone needs to validate your changes before they can be deployed to a certain environment.

I wouldn’t advise using Pull Requests to get changes in a development environment but I would advise using it to get changes to an Acceptance- or Production environment. By doing this, you can already find issues in an earlier stage than in production.

Read on to see how they work, using Azure DevOps as the example. GitHub pull requests are very similar in nature.

Comments closed

Parallel Loading in Spark Notebooks

Dustin Vannoy answers some questions:

I received many questions on my tutorial Ingest tables in parallel with an Apache Spark notebook using multithreading. In this video and post I address some of the questions that I couldn’t just answer in the YouTube comments. Watch the video for more complete answers but here are quick responses with links to examples where appropriate.

Click through for the video and some text versions. Dustin includes examples for Synapse and Databricks.

Comments closed

Creating a DNS Alias for a Dedicated SQL Pool

Resham Popli creates an alias:

This blog will walk through the details on how to enable custom DNS (Domain Name System) entries on an Azure Synapse dedicated pool inside an Azure Synapse workspace in case of disaster recovery.

The DNS alias provides a translation layer that can redirect your client programs to different servers. This layer spares you the difficulties of having to find and edit all the clients and their connection strings (in disaster recovery implementation). This is not supported out-of-box, so we need to take extra steps to enable this feature. There are some limitations, so please read these steps carefully.

Read on to learn how and what those limitations are.

Comments closed

Timeouts Importing a Bacpac File

Jose Manuel Jurado Diaz hits one of the three most annoying parts of dealing with bacpac files:

Today, I worked on a service request that our customer got the following error message: Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding importing a bacpac file, in this situation, was at the moment that SQLPackage was enabling (rebuilding) an index, for example, Enabling index ‘IX_MyIndex’…

The other two are, of course, getting one to export without any errors and getting one to export without timing out.

Comments closed

Securing the Root Keys when using TDE

Matthew McGiffen puts the keys behind lock and key:

In this quick post we’re going to look at an additional step you should take to secure your TDE databases. This is a step you won’t find in any other documentation on setting up TDE in SQL Server that I’ve seen, so it probably bears a little explaining.

Matthew has solid advice, though if someone does get the Local Admin credentials (e.g., through a spearphishing attack), that person could still eventually gain access to the underlying data. I get that at some point, things will break, so it’s mostly a matter of being aware what can cause that break and minimizing the likelihood of it occurring.

Comments closed