Press "Enter" to skip to content

Category: Cloud

Identifying Rows in sp_wait_for_database_copy)_sync

Published 2022-12-14 by Kevin Feasel

Jose Manuel Jurado Diaz troubleshoots a problem:

As you could see in this public documentation Auto-failover groups overview & best practices – Azure SQL Database | Microsoft Learn about sp_wait_for_database_copy_sync “sp_wait_for_database_copy_sync prevents data loss after geo-failover for specific transactions, but does not guarantee full synchronization for read access. The delay caused by a sp_wait_for_database_copy_sync procedure call can be significant and depends on the size of the not yet transmitted transaction log on the primary at the time of the call.”

Our customer asked about several scenarios to understand this behaviour and also, verify if there is possible to identify the rows that have not been synced. For this, I developed a POC to test it:

Read on to see what you’d need to do.

Comments closed

FHIR and Azure Health Services

Published 2022-12-14 by Kevin Feasel

Steve Hughes provides an overview of FHIR and what Azure has to offer:

With the recent updated mandates in the healthcare environment in the United States, Microsoft has continued to expand its capability to support the FHIR standard for integrating healthcare data. While the standard is well documented and Microsoft’s capabilities are expansive, it falls on data professionals to interpret that data and build meaningful reports and produce meaningful insights from the data as it is collected and integrated across environments. This requires a good working knowledge of JSON in SQL to manipulate complex data models. In the session, we did a short review of the FHIR standard and the overall implementation of FHIR in Azure. From there we reviewed the resulting data in the data lake and in Synapse. That was followed up with an overview into the heart of complex SQL using JSON functions in Synapse. Whether or not you are active in healthcare today, this will be an enlightening session on how to use JSON SQL functions within the Azure SQL platforms.

Read on to learn more.

Comments closed

Managing Azure Log Analytics Data Access

Published 2022-12-12 by Kevin Feasel

Bruno Gabrielli wants to limit data access to Log Analytics:

I am back with another important topic arising from my customers’ visits. How can I give very specific access to Log Analytics data, whether they be Security or Monitoring data?

Tricky one, isn’t it? A very simplistic answer could be: “manage your access list through IAM on the workspace”, but this is not enough. Say, for instance, that you would give scoped access to data coming from specific resources or, even more complicated, you would like that given the same resource one team can see some info and another one all the rest.

Looks complicated, but hey … good news: this is doable

Read on to learn how.

Comments closed

Working with the AML Python SDK

Published 2022-12-12 by Kevin Feasel

Tomaz Kastrun continues a series on Azure Machine Learning. Day 9 takes us through a piece of the Python SDK:

Python SDK namespace is azureml.core.environment. Environments specify the set of Python packages, environment variables, and software settings around your training and scoring scripts. In addition to Python, you can also configure PySpark, Docker and R for environments.

You can use namespace  Environment (or created object/asset) to make deployment and code reusable for training purposes at given docker images, configurations and compute type.

Day 10 shows us how to work with the Python SDK via VS Code or a local Jupyter notebook:

Let’s continue to explore the power of SDK and the namespaces. And we will look into namespace that will help you connect to Azure ML resources with Python SDK.

Comments closed

AML Environments and SDKs

Published 2022-12-09 by Kevin Feasel

Tomaz Kastrun continues an advent of Azure ML. First up is environments:

We have explored how to create a compute instance and compute target and learned that ML frameworks and scripting packages always come preinstalled.

Choosing the right set of components (CPU, GPU, RAM, Core) and corresponding software (OS, ML Framework, packages) can be time-consuming.

Under Curated environments, you will find predefined environments, with settings for running particular frameworks, like PyTorch or TensorFlow.

Then an overview of the Azure CLI and Python SDK for AML:

What is Azure CLI? It is an Azure Command Line, a great tool for running commands out of CMD. It is a multi-platform and can be run from Azure or from the client’s machine. It is great for scripting and automating repetitive tasks or making the complex task look like lines of code, especially when it comes to infrastructure, managing, provisioning and monitoring. It can also be run from Azure Cloud Shell. It is native to Azure and can be used across all the services and offerings. Usually, the Azure CLI commands start with “az ..”. On top of that, you can also install Azure Machine Learning CLI, as an extension to Azure CLI. The AML CLI will give you additional commands to manage resources for machine learning.

The same functionality (to some extent) in Azure Machine Learning can be achieved with Python SDK. In addition to that, it offers also great ways to create and manage resources you use for training and deployment of models.

And, so that we can catch up a bit to Tomaz, one more post covering the Python SDK:

Looking briefly into Azure CLI and Python SDK, let’s explore the power of SDK and the most important namespaces.

Comments closed

Data and Compute in Azure ML

Published 2022-12-06 by Kevin Feasel

Tomaz Kastrun continues an advent series on Azure ML. Day 4 covers data sources:

Yesterday, we have learned the general outlook of the Studio and in this blog post, we will be focusing primarily on getting data to the workspace and reading data from other data sources.

Day 5 has you provision some compute:

With a basic understanding of data assets, let’s create compute instances. Under “Manage” in the navigation bar, select “Compute” (denoted as 1), select “Compute instances” (d. 2) and click on “+ New”.

Comments closed

Thoughts on a Migration: Azure Analysis Services to Power BI Premium

Published 2022-12-06 by Kevin Feasel

Dan English shares some thoughts:

Over the past month couple of months I got the opportunity to test out the new migration experience that was just made available for Public Preview this past month during the PASS Data Community Summit and announced on the Power BI blog here Accelerate your migration experience from Azure Analysis Services to Power BI Premium with the automated migration tool. The blog post also shows a very quick animated gif walkthrough of the process and there is a thirteen minute video from the MS Build conference earlier this year where this was first demoed that you can check out here as well The Future of Enterprise Semantic Models.

Click through for a detailed analysis.

Comments closed

Creating an AML Workspace and Trying the Studio

Published 2022-12-05 by Kevin Feasel

Tomaz Kastrun continues an advent of Azure ML. First up, Tomaz creates a workspace:

You will select “New workspace”. For now, we will work on a workspace. But just to mention, the “New registry” will enable you to share assets among different workspaces, support multi-region replication and help you provision all resources to facilitate region replications.

From there, the focus shifts to using Azure Machine Learning Studio:

In this overview page, you can click the button “Launch studio” in the middle of the workspace or you can copy and paste the Studio web URL provided under the “Essentials” to start the Studio.

But before we launch the Studio, let’s explore some additional settings, worth mentioning.

Comments closed

Azure SQL Managed Instance Performance

Published 2022-12-05 by Kevin Feasel

Reitse Eskens wraps up a series on Azure SQL performance comparisons:

So far, the blogs were about the really SaaS databases; the database is deployed and you don’t have think about it anymore. This ease of use comes at a ‘price’. You’ve got no control whatsoever on files, you’ve lost the SQL Agent and a number of other features. The managed instance is a bit different. When I was testing you could see the TempDB files but not change them, since then a few changes have been made to this tier where you’re able to change settings and, Niko Neugebauer told the data community on twitter, there are more changes coming. With the managed instance, you get the agent back and you can run cross database query’s again. So you can safely say the managed instance is a hybrid between your trusty on-premises server and the fully managed Azure SQL database.

Click through for Reitse’s thoughts.

Comments closed

An Intro to Azure Machine Learning

Published 2022-12-02 by Kevin Feasel

Tomaz Kastrun has a new Advent challenge:

Azure Machine Learning (or Azure Machine Learning Service and abbreviation AML) is Azure’s cloud service for creating, managing and productionalising machine learning projects. It is a collaborative tool for Data Scientists, Machine Learning Engineers, and data engineers, covering their daily and operational tasks. From creating and training to deploying and managing predictive models and machine learning solutions.

Click through for the introduction.

Comments closed