Press "Enter" to skip to content

Category: Cloud

A Quick Primer on Microsoft Entra ID

Jordan Boich gives us a high-level overview:

As a DBA with on-prem and cloud experience, I feel confident in saying that I’m fairly well versed on the “DBA Domain” part of the shared responsibility model. However, if we really want to do our jobs right as data professionals, having an understanding of the entire infrastructure does wonders when it comes to making architectural decisions that can impact the entire system.

The struggle I’ve experienced being a DBA is that hearing things like “Entra ID” or “Microsoft Entra Domain Services” has me sitting in a position where I know of those things because I’ve been around it for years but not really having as deep of an understanding as I’d like to. In this post, I’m going to talk a bit about authorization, and authentication from the cloud perspective, specifically Azure, so that other fellow DBAs and data professionals can gain some insight, see what it looks like to operate in the Entra Admin Portal, and show that it’s not as overwhelming as I thought it would be.

Read on to learn more.

Comments closed

Managing Resources via Azure Cloud Shell

Jordan Boich isn’t afraid of the command line:

As a DBA or any data professional, it is becoming more valuable and vital to have a well rounded understanding of your data estate and environment. However, sometimes it can be a bit cumbersome, especially when connecting to Azure resources via PowerShell. You have to fumble with authentication, connection cmdlets and more, which can be a bit discouraging to use at times.

There are some interesting and useful tools to help circumvent that overhead and manage your Azure resources. Azure Cloud Shell is a great quick tool to use if you want to have a lightweight, and easily accessible way to help manage and administer your Azure resources.

Read on to see how you can connect and some of the things you can do with it. You also get your choice of bash versus PowerShell.

Comments closed

Using the Azure Resiliency Agent

Reitse Eskens gives a copilot a spin:

And that isn’t limited to services like storage, web portals, user management, and data stuff. It’s also location, backups, and disaster recovery. But there’s a catch: you need to make sure this is configured. I always compare the cloud (Azure in my daily work) with a huge box of Lego. You have all the pieces and elements to make something cool, but you have to build it. Even when you automate it, you still need to think about what you want.

Now, before I continue my Azure Fundamentals training, let’s move on to what this post is about: resiliency. Or, how your environment is set up for disaster recovery.

Let’s use the Azure Copilot to guide the process, create the necessary resource and review the advice.

Click through to see how it works. Because it’s in preview right now, we don’t know how much it will cost later. But in the meantime, you can give it a try.

Comments closed

Contrasting LTAP and HTAP

Paul Andrew notes another convergence of OLTP and OLAP:

At the Data and AI Summit in June this year, Databricks introduced Lake Transactional/Analytical Processing, or LTAP. My first reaction, I’ll admit, was cynicism, just for a change. Every software vendor must invent new names for old things these days. And for those of us who have worked in data architecture long enough to have the grey hair to prove it, like me, this felt very familiar. Hybrid Transactional and Analytical Processing, or HTAP, a term Gartner coined back in 2014, has long advertised bringing operational transactions and analytics “closer together”. In my experience it never really became a strong implementation pattern in data platform deliveries.

Read on to learn how LTAP and HTAP differ, where Databricks and other competitors (like Microsoft) are going in this space, and how much they have yet to do.

Comments closed

Shortcuts to Iceberg Tables in Microsoft Fabric

Teo Lachev makes a connection:

Snowflake is an increasingly popular option for data warehousing. One nice Snowflake feature is the ability to configure a table to save its data in Iceberg file format in the Snowflake own managed storage or external storage, including S3, GCS, ADLS Gen 2, and OneLake. As I wrote in the post “Give Me Your Data!”, Fabric supports shortcuts to Iceberg tables, in which case the data is not copied but exposed as a Parquet Delta table in a Fabric lakehouse. This could be useful when building Fabric-centric solutions on top of Snowflake without moving the data.

Iceberg isn’t just a Snowflake thing, though this is certainly helpful if you do have both Snowflake and Fabric.

Comments closed

Streaming Tables in Databricks

Chen Hirsh explains a concept:

A Streaming Table in Databricks is a table that receives data… well, in a stream.

And no, this has nothing to do with Netflix.

Imagine a conveyor belt carrying packages into a warehouse. A robot picks up each package and places it on the shelves.

Read on for the metaphor, as well as thoughts on when streaming tables make sense.

Comments closed

Azure Databricks and Power BI Storage Modes

Chris Webb makes a choice:

In case you haven’t already seen the blog post on the Power BI blog or the discussion on LinkedIn, we at Microsoft published a new white paper last week to help you decide which storage mode to use when you’re using Power BI to create semantic models and reports on data stored in Azure Databricks. You can find the announcement and the link to the paper here.

Click through for Chris’s thoughts and check out the whitepaper.

Comments closed

Working with Blob Leases

Miles Cole explains how leasing works with Blob Storage and Data Lake Storage blobs:

Leased a blob? No, this isn’t some new car model with a great leasing offer. Blobs are files in ADLS / OneLake, and leasing is the process of temporarily holding an exclusive lock on one of those files. Does this sound useful in data engineering? Maybe not at first, but in this post I’ll show why understanding the extended features of object storage APIs like blob leases can be invaluable when building data systems that scale.

I’ve never actually done this, but it’s a smart way of preventing multiple job agents from picking up the same work.

Comments closed

Setting up Azure SQL DB in Podman

Vlad Drumea builds a local version of Azure SQL Database:

Microsoft has recently released in private preview (at the time of writing) the Azure SQL Database container image.
This is Microsoft’s solution for folks who want to build and test Azure SQL DB native software without having to incur additional Azure costs just for dev/test/demo work.

This provides a local option for developers who want to work on products that require Azure SQL DB’s specific developer surface: T-SQL, drivers, and engine behavior.

Click through to see how this works. As of right now, you’ll need to be in the private preview, but I imagine things won’t change significantly once they move to public preview.

Comments closed

Test and Validate Azure SQL Database Migrations

Marlon Ribunal has a tool:

Introducing azsql-migration-test, a small open-source CLI that validates your Azure SQL Database migrations against a local Azure SQL Database Developer container — the same engine as the cloud, running on your machine.

The problem: proving a migration works shouldn’t require the cloud

The tool is AI-generated and it looks like the blog post is as well, but it does look to be useful.

Comments closed