A Fine Slice Of SQL Server
We have proof that Meagan Longoria is a consultant:
Question: When an activity in a Data Factory pipeline fails, does the entire pipeline fail?
Answer: It depends
Read on to understand the circumstances of when a pipeline activity failure will doom an entire pipeline run and when recovery might be possible.
Comments closedRayis Imayev wants to pass a JSON array from one Azure Data Factory pipeline to another:
This next post came out of an error message during my attempt to pass a hard-coded array value between pipelines. Strangely, this use-case worked well in the pipeline that was already deployed in ADF, however, I was getting an error message while trying to test and execute this very same pipeline in a Debug mode.
Click through for the explanation.
Comments closedNiels Berglund takes us through the steps of creating an Azure VM running SQL Server:
A while ago, I wanted to do a quick test on a new SQL installation, and I wanted the SQL installation to be on a “pristine” server. I was not keen on creating a new virtual machine on my local dev-box, as for that I would need to create a VM image etc., and it seemed like too much hassle for a lazy person like me. The obvious choice then is to do it in the cloud. How hard can that be, what could possibly go wrong?!
It turned out to not be as straight-forward as I thought it would be, but eventually, I managed to get it right. Since I probably need to do it again some time, I thought I’d write a post about it, so I have something to go back to. So here we go …
Niels goes through this in meticulous detail, as is the norm.
Comments closedDisclaimer: I am using the word shake by my own initiative and no Microsoft Documentation ever to my knowledge ever mentioned that situation. Those shakes are represented most of the time as health events to the cluster, such as the Lease Timeout resulting in a sudden attempt of Failover.
Why did I choose that word ? I don’t know. Honestly. 🙂
Read on to see it in context around hosts, CPU, and especially I/O.
Comments closedJohn McCormack has a tip for us:
The traditional methods used for to find the start up time for SQL Server don’t work in Azure SQL DB.
I searched high and low to find this and thought I’ve got to share, and hopefully make it search engine friendly. A traditional google or bing search wasn’t bringing up the best way to find this out. I saw a lot of complicated queries to pull data, convert it and estimate start up time using functions and all kinds of magic.
Click through for the one-liner script.
Comments closedReitse Eskens runs into a strange bug in Azure SQL Database Hyperscale:
This single wait made sure our complete environment went dead in the water. Everything halted. To get some context, Microsoft has some documentation on this wait:
Occurs when a Hyperscale database primary compute node log generation rate is being throttled due to delayed log consumption at the page server(s).
Well, that’s not really helping, because that’s about everything they tell you about it.
Click through for Reitse’s findings and Microsoft’s advice.
Comments closedAbhinav Garg and Premal Shah have an announcement:
We’re excited to announce the general availability of Secure Cluster Connectivity (also commonly known as No Public IP) on Azure Databricks. This release applies to Microsoft Azure Public Cloud and Azure Government regions, in both Standard and Premium pricing tiers. Hundreds of our global customers including large financial services, healthcare and retail organizations have already adopted the capability to enable secure and reliable deployments of the Azure Databricks unified data platform. It allows them to securely process company and customer data in private Azure Virtual Networks, thus satisfying a major requirement of their enterprise governance policies.
Read on fore more detail about how this works.
Comments closedAngela Chu, et al, take us through a streaming data ingestion process:
How do you process IoT data, change data capture (CDC) data, or streaming data from sensors, applications, and sources in real time? Apache Kafka® and Azure Databricks are widely adopted technologies in the industry, but they require specific skills and expertise to run. Leveraging Confluent Cloud and Azure Databricks as fully managed services in Microsoft Azure, you can implement new real-time data pipelines with less effort and without the need to upgrade your datacenter (or set up a new one).
This blog post demonstrates how to configure Azure Databricks to interact with Confluent Cloud so that you can ingest, process, store, make real-time predictions and gain business insights from your data.
Click through for a detailed demonstration.
Comments closedArun Sirpal takes us through a fairly new feature in Azure SQL Database:
There is a new (ish) interface to looking and configuring backups for your Azure SQL Database. This can be found within the settings section of the SQL Server.
As you can see, by default we have 7 days retention to allow for PITR – Point In Time Recovery, anything longer you will need to setup long term retention.
Click through to see how to set this up.
Comments closedIn this post I want to cover how you can transform your SQL Server database schema for a dedicated SQL Pool if you are using Azure DevOps. Because I covered it at Data Toboggan over the weekend and it can be very useful.
By the end of this post, you will know one way you can transform the schema of a database project for SQL Server 2019 if you are using Azure DevOps. So that you can make it optimal for dedicated SQL Pools.
Click through for the process and an example. Note that this isn’t a quick “check this box and you’re done” type of solution, but if you already have a proper star schema, this will help you think through some of the things you’ll need to do.
Comments closed