Press "Enter" to skip to content

Category: Cloud

Rebuilding a Dedicated SQL Pool via Azure DevOps

Published 2022-09-23 by Kevin Feasel

Sarath Sasidharan clones an Azure Synapse Analytics dedicated SQL pool:

There are many scenarios where you want to create a new Synapse dedicated SQL pool environment based on an existing Synapse dedicated SQL pool environment. This may be required when you need to create a development or test environment based on your production environment by copying complete schemas and without copying data.

Note that this process won’t move the data itself—given that you’re starting with terabytes for an effective dedicated SQL pool, trying to create a bacpac would be an exercise in misery.

Comments closed

Auto-Scaling SQL Managed Instances

Published 2022-09-20 by Kevin Feasel

Amanda Ibrahim answers a customer request:

If the customer needs to configure auto scaling for SQL Managed Instance, where the vCores can be increase and decrease automatically on specific time.

Read on to learn how. This can be useful for dev environments or places where your use case has “office hours”—that is, 90% of your activity takes place during a block of 4-12 hours—and you can afford the time in which the service is unavailable due to scaling.

Comments closed

Azure Data Explorer September 2022 Updates

Published 2022-09-19 by Kevin Feasel

Shaf Mahmood has a few updates for us:

The  optimized autoscale feature has been in place and using reactive logic. It has been helping ADX users by adjusting the cluster size when there is an increase in resources due to ingestion or query load.  The optimized autoscale feature has been further improved with predictive logic. This logic monitors the same metrics as the reactive logic and over time builds up the cluster usage pattern and uses this to forecast and plan the size of the cluster.  The reactive logic is still used to ensure any forecast anomalies or usage pattern changes are still autoscaled appropriately.

There are a few cost-related updates but also updates to visuals, data ingestion, and more.

Comments closed

Basic Tier Performance in Azure SQL DB

Published 2022-09-19 by Kevin Feasel

Reitse Eskens keeps things basic:

When you look at the documentation, the basic tier has low CPU (at max less than one), 1 to 4 IOPS per DTU (translating to 5-20 IOPS in total), a latency of 5ms read and 10 ms write and a maximum of 7 days backup retention. Even though it’s advertised as a production database, I wouldn’t store essential data there. However, I can use it for metadata storage in a Data Factory or Synapse Analytics environment.

About the IOPS, one IOP is a read of a disk cluster, usually 4 Kb. In this case, this means that the database is capable of reading 20 to 80 Kb of data per second.

When they say Basic, they aren’t kidding.

Comments closed

Restoring Azure SQL MI Databases to SQL Server 2022

Published 2022-09-14 by Kevin Feasel

Garry Bargsley restores a backup:

My post for T-SQL Tuesday #154 is going to be a demo of the new SQL Server feature that allows you to restore an Azure SQL Managed Instance backup to a SQL Server 2022 instance. Actually, I am not sure if this is a feature or just an enhancement that allows for this behavior to work.

Current versions of SQL Server do not allow the restoration of backups taken on Azure SQL Managed Instance.  Managed Instances are considered “vNext” and runs a different version that is beyond anything on-premises could keep up with due to the frequent changes being applied to Managed Instance. There has always been a disconnect with the restorability between the two platforms, so it is good that Microsoft has found a solution to this limitation.

I’d still like to see the ability to fail from SQL MI to on-prem (or VM) SQL Server—you can go from on-prem to Azure SQL MI, though it sounds like right now, failback is a database restore.

Comments closed

Data Virtualization in Azure SQL Managed Instance

Published 2022-09-14 by Kevin Feasel

Mladen Andzic has an announcement:

We are excited to announce the general availability (GA) of data virtualization capabilities in Azure SQL Managed Instance, with improved query performance and managed identity as a new supported option for authenticating to storage accounts.  

The data virtualization enables you to execute Transact-SQL (T-SQL) queries on files storing data in common data formats in Azure Data Lake Storage Gen2 or Azure Blob Storage and combine it with relational data stored locally in the managed instance using logical joins. This way you can transparently access external data while keeping it in its original format and location. There is no data duplication or need to run and maintain ETL processes, which means that you can extract and deliver insights faster. The supported file formats are Parquet, CSV, and JSON.

This is similar to PolyBase in SQL Server 2019 but is a different underlying technology. In SQL Managed Instance, it looks like we only get API-based data virtualization, not the ODBC-based PolyBase we saw in SQL Server 2019.

Comments closed

SQL Server 2022 and Azure Data Lake Storage

Published 2022-09-12 by Kevin Feasel

I continue a series on data virtualization in SQL Server 2022:

Today’s post is a fairly short one. Let’s say you want to connect to Azure Data Lake Storage Gen2 from SQL Server 2022. We saw in a recent post how to connect to Azure Blob Storage. When it comes to Azure Data Lake Storage, the story is almost the same, though there are a couple pitfalls you will want to avoid.

Read on for that error, which stymied me for a good 10 minutes.

Comments closed

Comparing Performance of Azure SQL Databases

Published 2022-09-12 by Kevin Feasel

Reitse Eskens begins an interesting series:

The trigger for this feast were a number of questions from my employer. What database tier and SKU (Stock Keeping Unit) should we choose for customer C. Well, the answer depends on their budget and the performance they’re expecting. And we didn’t have a conclusive answer. Microsoft will provide you with their statistics on database performance, but what does that mean in real life. Because no matter what the good people in Redmond can come up with as databases, we as data engineers can do worse. Much worse.

So, I created some horrible query’s to insert, select and delete data and fired them off to a database. My own laptop with nvme disks provided the baseline, after that I gradually worked my way up from the Basic DTU database to the Hyperscale database.

This first post is an introduction to the method and laying out expectations. I’m looking forward to the remaining posts in the series.

Comments closed

Deploying an Arc-Enabled SQL Managed Instance

Published 2022-09-12 by Kevin Feasel

Warwick Rudd continues a series on Azure Arc-enabled data services:

Now that we have our Azure Arc-enabled Data Controller configured and available, we can now deploy our first Arc-enabled SQL Managed Instance into our environment. As previously mentioned depending on the type of configuration required for your environment with your Arc-enabled Data Controller (Directly connected or Indirectly connected modes) this will dictate the approach available for you to setup / configure your Arc-enabled SQL Managed Instance.

Click through for a step-by-step guide.

Comments closed