Press "Enter" to skip to content

Category: Cloud

On-Premises SQL Server is Still Relevant

Published 2020-12-14 by Kevin Feasel

John Morehouse does not abide by Betteridge’s Law of Headlines:

While I’m a firm believer that the cloud is not a fad and is not going away, it’s just an extension of a tool that we are already familiar with.  The Microsoft marketing slogan is “It’s just SQL” and for the most part that is indeed true.  However, that does not mean that every workload will benefit from being in the cloud.  There are scenarios where it does not make sense to move things to the cloud so let’s take a look at a few of them.

Read on for several reasons why the cloud might not be right for you.

Comments closed

Setting up Azure Purview for Power BI

Published 2020-12-10 by Kevin Feasel

Soheil Bakhshi has a great step-by-step walkthrough for setting up Azure Purview:

Microsoft newly announced a piece of very exciting news that Azure Purview now supports Power BI. This is massive news from a data governance point of view. Azure Purview is the next generation of Azure Data Catalog with more metadata discovery power and the ability to use sensitivity labels. After reading the news, I immediately decided to set up my test environment and give it a go. I followed the steps mentioned in this article on the Microsoft documentation website but I faced some difficulties to get it to work. And here we are, another blog post to help you to set up the Azure Purview for Power BI.

Click through for a detailed walkthrough.

Comments closed

Clouds as Single Points of Failure

Published 2020-12-09 by Kevin Feasel

Denny Cherry argues that you should not consider a cloud provider as a single point of failure:

Having a two cloud providers isn’t going to save you from an outage. The public cloud providers (Microsoft Azure, Amazon AWS, Google GCP, etc.) have specifically designed their networks so that an outage at one region doesn’t impact other regions.

The day before US Thanksgiving (November 25, 2020), AWS had a major outage where the east-us facility suffered an outage for several hours. But you’ll notice something very interesting about this outage. No other AWS region was impacted by this outage. This is a very important distinction, as it shows that having multiple regions within AWS would give a solid Disaster Recovery strategy a great fail-over experience.

I’m mostly in agreement with Denny on this, but then I’d also have to point out the Azure AD issue which crippled Azure work across the globe, or the Azure DevOps service going down for a period of time (because everything was hosted in one data center and there was an issue). Depending on just how important uptime is, it can still make sense to be multi-cloud, especially if we use a broad enough definition which includes on-premises as a “local cloud.” In extreme cases—say, you lose millions of dollars per hour of downtime—the cost of a belt + suspenders approach is well below the expected loss from an outage.

Comments closed

Azure Purview: External Connections and the Starter Kit

Published 2020-12-09 by Kevin Feasel

Wolfgang Strasser continues a series on Azure Purview. First up is a look at external connections:

During my tests with Azure Purview I found out, that in my demo accounts (I created multiple to test it) the Management menu was missing some items.

Read on to learn why and how you can rectify this. Then, check out Wolfgang’s take on the Starter Kit:

Very often, when I want to test some new services I miss some infrastructure and environments I can start and play with. I am not talking about creating a new Azure Purview account (see my previous blog post – Creating an Azure Purview account) – I am talking about the data infrastructure to analyze, catalog and gain knowledge out of it.

We could start to create such an infrastructure, BUT: the Purview team create a Starter Kit to quickly create a data estate and configure everything that you can start with Purview within a view minutes.

Read on for one issue (of the self-inflicted variety) Wolfgang ran into during deployment. But it does look like a great way to get started with Purview and build up a relevant demo environment.

Comments closed

Using Notebooks to Load Data into the Databricks File System

Published 2020-12-07 by Kevin Feasel

Tomaz Kastrun is putting together an Advent of Azure Databricks:

Yesterday we started working towards data import and how to use drop zone to import data to DBFS. We have also created our first Notebook and this is where I would like to start today. With a light introduction to notebooks.

Read on for a depiction of notebooks, as well as an example which loads data into the Databricks File System (DBFS).

Comments closed

Creating an Azure Purview Catalog Instance

Published 2020-12-07 by Kevin Feasel

Wolfgang Strasser wants to try out Azure Purview:

Basics – Resource group, purview account name (this cannot be changed afterwards) and the location.

As of today (2020-12-06), there are only 5 Azure regions you can choose from to store the Purview metadata. But – in-region scanning from 16 other Azure regions is available in the preview (source)

This is part one of a multi-part series, so stay tuned for more.

Comments closed

Introducing Azure Purview

Published 2020-12-04 by Kevin Feasel

Wolfgang Strasser gives us a once-over on a new service:

Today, at the Azure Data and Analytics event, a new Azure data governance service called Azure Purview (https://aka.ms/AzurePurview) was presented and made available in a public preview.

I have not had a chance to try the actual service, but I found a very interesting video (Microsoft mechanics video) where I took the following screenshots from.

Read on for Wolfgang’s thoughts. It’s definitely a step up from Azure Data Catalog.

Comments closed

So You’ve Hit the Limits of ADF Concurrency

Published 2020-12-03 by Kevin Feasel

Paul Andrew shows what happens you you break the ADF concurrency barrier:

Firstly, understanding how these limits apply to your Data Factory pipelines takes a little bit of thinking about considering you need to understand the difference between an internal and external activity. Then you need to think about this with the caveats of being per subscription and importantly per Azure Integration Runtime region.

Assuming you know that, and you’ve hit these limits!

Click through to see what happens. It’s not pretty.

Comments closed

Working with Self-Hosted Integration Runtimes

Published 2020-12-03 by Kevin Feasel

Craig Porteous walks us through some of the planning necessary for self-hosted integration runtimes:

If your Data Factory contains a self-hosted Integration runtime, you will need to do some planning work before everything will work nicely with CI/CD pipelines. Unlike all other resources in your Data Factory, runtimes won’t deploy cleanly between environments, primarily as you connect the installed runtime directly to a single Data Factory. (We can add more runtime nodes to a single Data Factory but we cannot share a single node between many data factories*). An excerpt from Microsoft’s docs on Continuous integration and delivery in Azure Data Factory mentions this caveat.

Read on for the consequences and two options available to you.

Comments closed

dbachecks Against Azure SQL Databases

Published 2020-12-02 by Kevin Feasel

Jess Pomfret takes us through running dbachecks on an Azure SQL Database:

Last week I gave a presentation at Data South West on dbachecks and dbatools. One of the questions I got was whether you could run dbachecks against Azure SQL Databases, to which I had no idea. I always try to be prepared for potential questions that might come up, but I had only been thinking about on-premises environments and hadn’t even considered the cloud.  The benefit is this gives me a great topic for a blog post.

Click through for the answer.

Comments closed