Press "Enter" to skip to content

Category: Purview

Elastic Data Maps with Azure Purview

Published 2021-08-19 by Kevin Feasel

Wolfgang Strasser has some good news for us:

It’s been a long and intensive discussion – the (initial) pricing structure of Azure Purview. As I already talked about it in my Purview pricing blog post, the basic cost calculation involves

– the cost for the data map (= the infrastructure to store metadata and provide the Purview UI + cataloging functionality)

– plus the costs involved for scanning sources.

And that has added up to a significant amount of money, especially in dev/test scenarios. But read on for the glad tidings Wolfgang has to share.

Comments closed

Naming Azure Purview Scans

Published 2021-08-17 by Kevin Feasel

Daniel Janik treats Azure Purview scans like pets rather than cattle:

If you’ve ever been a DBA and seen the mess that you get with SQL Agent Jobs without a clean naming standard for your job schedules and job names then you’ll appreciate this tip.

If you haven’t been a DBA that’s OK too. Years ago I came up with my own naming standard for SQL Agent artifacts and I’ve always felt better when the messy room was clean. No Really! That’s exactly what this is like. A messy room where you are pretty sure you put the item you’re looking for in but you just can’t seem to find it until you clean 95% of the mess and then you’re so exhausted that you don’t have time to do what you wanted to in the first place. Ever been there?

Read on to see what the scans look like by default, as well as some thoughts Daniel has regarding a better way to do things.

Comments closed

Reviewing Azure Purview Data Catalog Features

Published 2021-08-16 by Kevin Feasel

Angela Henry continues a series on Azure Purview:

The Data Catalog portion of Purview is where most people will spend their time. It provides the information about your organizations data assets in a searchable format. Depending on which level of Data Catalog you choose; you can also access a business glossary, lineage visualization, catalog insights, and sensitive data identification insights. This article will focus on the three different levels available within Data Catalog and offers scenarios demonstrating when you would use each offering.

Read on for the three tiers, all of which are currently free but that won’t stay the case.

Comments closed

Importing SQL Server Extended Properties into Azure Purview

Published 2021-08-12 by Kevin Feasel

Daniel Janik shows how you can use PyApacheAtlas to move specific SQL Server extended properties into Azure Purview:

This post is going to be restricted to only SQL Server Table Columns and only Extended Properties named MS_Description. Quite a few years ago I worked on a data catalog project where we added descriptions for many of the tables, views, and columns to the database using extended properties named MS_Description. Let’s assume you have some of these for this post keeping in mind that the Purview APIs provide so many functions beyond what this post covers and that the code here could be modified to do so much more as well.

Starting out I thought it would be great to import the sensitivity classifications that SSMS creates. Pre-SQL 2019 these were held in Extended Properties and now have their very own DMV (sys.sensitivity_classifications). While this sounded great in theory it wasn’t as exciting when I wrote the code. This is because Azure Purview already has system classifications at a more granular scale for each of the ones you find in SSMS and Purview also adds these as it executes a scan on the data source. It does a pretty good job too. With that said, I shifted my focus to adding descriptions instead.

Read on to see how you can do this.

Comments closed

An Overview of Azure Purview

Published 2021-08-04 by Kevin Feasel

Angela Henry gives us an overview of Azure Purview:

Organizations are amassing more data than ever, yet it is getting more difficult for their employees to find that data and use it with confidence. What if there was a solution out there that not only told us what data sources we have, but could tell us how those data sources should be used, and who the stewards/producers of that data are? What if it could allow us to classify our data, and provided us insights into what our entire data estate looked like? It might sound like data nirvana, but it just might be possible with the newest Platform as a Service (PaaS) offering from Microsoft, Azure Purview.

In the first part of the series, Angela covers the basics and pricing, so check it out.

Comments closed

Using Azure Purview to Catalog S3 Datasets

Published 2021-06-03 by Kevin Feasel

Gauri Mahajan crosses the clouds:

Data exists in various types of formats and hosted in equally varied type repositories depending on the format of data. With the advent of the cloud, generally, every public cloud provider like Amazon AWS, Microsoft Azure, and Google Cloud Platform provides a variety of data hosting avenues. A large-scale enterprise is very likely to have a multi-cloud footprint, where data is hosted on more than one cloud. As the data footprint on the cloud becomes larger, the need for a data catalog becomes increasingly important. Each cloud provider provides its format of the cloud-native data catalog. But even after employing a cloud provider-specific catalog, an enterprise may struggle to get a 360-degree view of data on the cloud. The reason being each cloud provider providers the ability to catalog data hosted on its own cloud only, which compels the end-users to reconcile or reference data from multiple catalogs, as the data catalog on the cloud may not integrate with cross-cloud data hosting services. As the client needs evolve, so do the cloud services. Azure Purview is the brand-new data catalog and governance-related service on the Azure cloud. It has introduced features to support cataloging data hosted on AWS Simple Storage Service (S3), which is typically considered the storage layer of data lake on AWS.

Read on to see how.

Comments closed

Azure Purview Support for Azure SQL Database Views

Published 2021-05-24 by Kevin Feasel

Wolfgang Strasser looks at a new feature in Azure Purview:

There was one big thing missing so far – the scanning of SQL Server / Azure SQL Views. Which – well – in many cases (customer databases) was a huge loss of information in the data catalog.

This known limitation is listed on the documentation page but many of us overread this sentence.

But check out Wolfgang’s most recent finding and it’s clear that the team is working on it.

Comments closed

An Introduction to Azure Purview Studio

Published 2021-03-29 by Kevin Feasel

Rahul Mehta takes a look at Azure Purview’s current user interface:

Metrics in Azure Purview are integrated and reported using Azure Monitor. Click on the Metrics section and it would have a link to Azure Monitor, which would open in a new tab as shown below. From this interface, we can filter and split different data metrics, plot the same on different types of graphs, as well as build custom dashboards to visualize the metrics to evaluate performance. The available metrics depend on the features enabled in the Azure Purview account. Metrics may have different levels of granularity and this may lead to a significant amount of data. The data generated can be aggregated in these dashboards using the available aggregation functions.

Click through for a quick walkthrough.

Comments closed

Combining Azure Synapse Analytics and Azure Purview

Published 2021-02-02 by Kevin Feasel

Wolfgang Strasser shows how we can integrate Azure Synapse Analytics with Azure Purview:

In the past months I had the chance to play with and build solutions based on Azure Synapse Analytics and Azure Purview.

Azure Synapse (my Synapse blog entries) as the foundation for a solid platform to store, analyze and build data solutions and Azure Purview (my Purview blog posts) as the data governance and data catalog solution in Azure.

During the writing of my latest blog post (What’s new in Azure Synapse Analytics?), I found a very interesting entry in the update feature list: Azure Purview Integration.

Read on to see how.

Comments closed