Press "Enter" to skip to content

Category: Cloud

Azure SQL Managed Instance Extreme Storage Latency

Published 2024-12-19 by Kevin Feasel

Kendra Little has another caveat emptor message:

What are your stories of unbelievably bad performance from cloud vendors? I’ll go first. For years, Azure SQL Managed Instance’s General Purpose Tier has documented “approximate” storage latency as being “5-10 ms.” This week they added a footnote: “This is an average range. Although the vast majority of IO request durations will fall under the top of the range, outliers which exceed the range are possible.”

How approximate is that 5-10 milliseconds, you might wonder? If you use Azure SQL Managed Instance these days, you will regularly find messages in your SQL Server Error log indicating that all data and log files have experienced latency of up to 60 seconds. At least, 60 seconds is the maximum I’ve observed personally, looking in the logs of several customers’ Managed Instances. Could it be worse? Microsoft hasn’t documented a ceiling. My testing shows that this latency occurs randomly to your workload and is not related to your resource usage: using less IO will not make the errors less likely. You have no way to avoid these storage failures (I don’t see how 15-60 second latency is not a failure), and they can occur anytime.

This is a major strike against SQL Managed Instance General Purpose. Considering the cost of the product, you could buy a new server with direct-attached NVMe storage, have it paid off after one year, have better performance, and get to depreciate the entire expense over a 3-5 year window (something you cannot do with the hardware side of cloud services–you can only depreciate the cost of licensing, assuming you have a 3-year reservation).

2 Comments

Linked Servers into Azure

Published 2024-12-18 by Kevin Feasel

Andy Brownsword goes old-school:

Connecting different versions of SQL Server can allow us to combine or transfer data between environments. This can become a challenge when the versions are really different.

Have you tried to connect SQL Server 2008 to a SQL database in Azure? – it can throw up a few curve balls.

In this post we’ll look at how to solve 3 of the issues you might come up against.

When reading the title, my first response was, “But why not use PolyBase?” Then Andy threw the SQL Server 2008 bit at me, and then my response was, “But why not use a product that isn’t nearly old enough to vote?”

Nonetheless, Andy does a great job of demonstrating how this would work, and it can work for later versions of SQL Server as well.

Comments closed

Working with the Azure AI Document Service

Published 2024-12-16 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI. First up is a visual review of the Azure AI Document service:

Vision and Document services gives your apps the ability to analyze images, process documents and use technologies for optical character recognition (OCR) with combinations to machine learning.

That product has gone through a few name iterations, including Document Recognizer. But wait, there’s more!

Tomaz also takes a look at the Python SDK:

Vision and Document SDK for Python gives you extra extensibility of the services to add it to your apps.

Using Vision and Document SDK with Python, you will need to have the resource up and running (for the starters go with free pricing tier (F0)) and get the Document intelligence API Key and Endpoint address.

Click through for an example of how that works.

Comments closed

Using the Azure AI Language and Translation Python SDK

Published 2024-12-12 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI:

Using SDK options for “Language + Translation” service is

pip install azure-ai-textanalytics==5.2.0

and adding your endpoint in format like: https://yyyyy_azurehub_xxxxxxx.cognitiveservices.azure.com/

and secret to your endpoint. And you will also need the region name (e.g.: west-europe).

Once you’ve set up the necessary credentials, Tomaz shows how easy it is to call the service.

Comments closed

SQL Database in Microsoft Fabric

Published 2024-12-11 by Kevin Feasel

Deepthi Goguri is pleased with a new spin on an existing product:

“SQL database in Microsoft Fabric is a developer-friendly transactional database, based on Azure SQL Database, that allows you to easily create your operational database in Fabric. A SQL database in Fabric uses the same SQL Database Engine as Azure SQL Database.”

As you read, this is a transactional database that can be created in fabric and can be replicated to Data Lake for the analytical workloads. The other main goal is to help build AI apps faster using the SQL Databases in Fabric. The data is replicated in near real time and converted to Parquet, in an analytics-ready format.

Read on to learn more about the offering. I’m still not 100% sold on its virtues versus simply having an Azure SQL Database and enabling mirroring.

Comments closed

Metadata-Driven Spark Clusters in Azure Databricks

Published 2024-12-10 by Kevin Feasel

Matt Collins ties the room together with a bit of metadata:

In this article, we will discuss some options for improving interoperability between Azure Orchestration tools, like Data Factory, and Databricks Spark Compute. By using some simple metadata, we will show how to dynamically configure pipelines with appropriately sized clusters for all your orchestration and transformation needs as part of a data analytics platform.

Click through for an explanation of the challenge, followed by the how-to.

Comments closed

An Overview of the Azure AI Services Speech Service

Published 2024-12-09 by Kevin Feasel

Tomaz Kastrun has been busy with the Azure AI series. First up is an overview of Azure AI Services (nee Cognitive Services) available in the Azure AI Foundry:

In Azure AI Foundry, you can always gow to Azure AI Services, where you can create intelligent apps with different AI models. These services are simple and ready to use with relative low costs.

Then Tomaz drills into the Speech service:

In Azure AI Foundry you will find the speech playground with the vast variety of solutions to enhance and add the functionalities to your applications.

Speech service will give you capabilities to convert speech to text, realtime translations, fast transcriptions, voice assistant and others.

After that, we get an intro of Speech Studio:

Speech studio (available at URL: https://speech.microsoft.com/portal)  is a set of UI-based tools for building and integrating features from Azure AI Speech service (available in Azure portal) into your applications using no-code approach. You can also create projects by using and referencing the assets and services using  Speech SDK, the Speech CLI, or the REST APIs.

The Speech service is by no means perfect, but it’s interesting just how well it can do at detecting languages (one set of functionality) and translating arbitrary audio from one language to another (via a different call).

Comments closed

Mounding ADF Instances in Microsoft Fabric

Published 2024-12-06 by Kevin Feasel

Koen Verbeeck has an existing Azure Data Factory:

We recently started using Microsoft Fabric for our cloud data platform. However, we already have quite an estate of Azure data services running in our company, including a huge number of Azure Data Factory (ADF) pipelines. It seems cumbersome to migrate all those pipelines to Microsoft Fabric, especially because some features are not supported yet and ADF is the mature choice at the moment. We like the concept of Microsoft Fabric’s centralization, where everything is managed in one platform. Is there an option to manage ADF in Fabric?

Read on for the answer, but make sure to check out its limitations as well.

Comments closed

Creating a Project in Azure AI Foundry

Published 2024-12-04 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI:

Azure AI models inference service provides access to the most powerful models available in the Azure AI model catalog. Coming from the key model providers in the industry including OpenAI, Microsoft, Meta, Mistral, Cohere, G42, and AI21 Labs; these models can be integrated with software solutions to deliver a wide range of tasks including content generation, summarization, image understanding, semantic search, and code generation.

The Azure AI model inference service provides a way to consume models as APIs without hosting them on your infrastructure. Models are hosted in a Microsoft-managed infrastructure, which enables API-based access to the model provider’s model. API-based access can dramatically reduce the cost of accessing a model and simplify the provisioning experience.

Read on to learn more about what you get when you create a project.

Comments closed