Press "Enter" to skip to content

Category: Cloud

Managing Resources via Azure Cloud Shell

Jordan Boich isn’t afraid of the command line:

As a DBA or any data professional, it is becoming more valuable and vital to have a well rounded understanding of your data estate and environment. However, sometimes it can be a bit cumbersome, especially when connecting to Azure resources via PowerShell. You have to fumble with authentication, connection cmdlets and more, which can be a bit discouraging to use at times.

There are some interesting and useful tools to help circumvent that overhead and manage your Azure resources. Azure Cloud Shell is a great quick tool to use if you want to have a lightweight, and easily accessible way to help manage and administer your Azure resources.

Read on to see how you can connect and some of the things you can do with it. You also get your choice of bash versus PowerShell.

Leave a Comment

Using the Azure Resiliency Agent

Reitse Eskens gives a copilot a spin:

And that isn’t limited to services like storage, web portals, user management, and data stuff. It’s also location, backups, and disaster recovery. But there’s a catch: you need to make sure this is configured. I always compare the cloud (Azure in my daily work) with a huge box of Lego. You have all the pieces and elements to make something cool, but you have to build it. Even when you automate it, you still need to think about what you want.

Now, before I continue my Azure Fundamentals training, let’s move on to what this post is about: resiliency. Or, how your environment is set up for disaster recovery.

Let’s use the Azure Copilot to guide the process, create the necessary resource and review the advice.

Click through to see how it works. Because it’s in preview right now, we don’t know how much it will cost later. But in the meantime, you can give it a try.

Leave a Comment

Contrasting LTAP and HTAP

Paul Andrew notes another convergence of OLTP and OLAP:

At the Data and AI Summit in June this year, Databricks introduced Lake Transactional/Analytical Processing, or LTAP. My first reaction, I’ll admit, was cynicism, just for a change. Every software vendor must invent new names for old things these days. And for those of us who have worked in data architecture long enough to have the grey hair to prove it, like me, this felt very familiar. Hybrid Transactional and Analytical Processing, or HTAP, a term Gartner coined back in 2014, has long advertised bringing operational transactions and analytics “closer together”. In my experience it never really became a strong implementation pattern in data platform deliveries.

Read on to learn how LTAP and HTAP differ, where Databricks and other competitors (like Microsoft) are going in this space, and how much they have yet to do.

Leave a Comment

Shortcuts to Iceberg Tables in Microsoft Fabric

Teo Lachev makes a connection:

Snowflake is an increasingly popular option for data warehousing. One nice Snowflake feature is the ability to configure a table to save its data in Iceberg file format in the Snowflake own managed storage or external storage, including S3, GCS, ADLS Gen 2, and OneLake. As I wrote in the post “Give Me Your Data!”, Fabric supports shortcuts to Iceberg tables, in which case the data is not copied but exposed as a Parquet Delta table in a Fabric lakehouse. This could be useful when building Fabric-centric solutions on top of Snowflake without moving the data.

Iceberg isn’t just a Snowflake thing, though this is certainly helpful if you do have both Snowflake and Fabric.

Leave a Comment

Azure Databricks and Power BI Storage Modes

Chris Webb makes a choice:

In case you haven’t already seen the blog post on the Power BI blog or the discussion on LinkedIn, we at Microsoft published a new white paper last week to help you decide which storage mode to use when you’re using Power BI to create semantic models and reports on data stored in Azure Databricks. You can find the announcement and the link to the paper here.

Click through for Chris’s thoughts and check out the whitepaper.

Leave a Comment

Working with Blob Leases

Miles Cole explains how leasing works with Blob Storage and Data Lake Storage blobs:

Leased a blob? No, this isn’t some new car model with a great leasing offer. Blobs are files in ADLS / OneLake, and leasing is the process of temporarily holding an exclusive lock on one of those files. Does this sound useful in data engineering? Maybe not at first, but in this post I’ll show why understanding the extended features of object storage APIs like blob leases can be invaluable when building data systems that scale.

I’ve never actually done this, but it’s a smart way of preventing multiple job agents from picking up the same work.

Comments closed

Setting up Azure SQL DB in Podman

Vlad Drumea builds a local version of Azure SQL Database:

Microsoft has recently released in private preview (at the time of writing) the Azure SQL Database container image.
This is Microsoft’s solution for folks who want to build and test Azure SQL DB native software without having to incur additional Azure costs just for dev/test/demo work.

This provides a local option for developers who want to work on products that require Azure SQL DB’s specific developer surface: T-SQL, drivers, and engine behavior.

Click through to see how this works. As of right now, you’ll need to be in the private preview, but I imagine things won’t change significantly once they move to public preview.

Comments closed

Test and Validate Azure SQL Database Migrations

Marlon Ribunal has a tool:

Introducing azsql-migration-test, a small open-source CLI that validates your Azure SQL Database migrations against a local Azure SQL Database Developer container — the same engine as the cloud, running on your machine.

The problem: proving a migration works shouldn’t require the cloud

The tool is AI-generated and it looks like the blog post is as well, but it does look to be useful.

Comments closed

Azure SQL Developer

Carlos Robles and Vandana Mahtani make an announcement:

Big news: Azure SQL Developer is here, in private preview. It’s the Azure SQL Database engine, on your laptop, in a container. Build against the exact engine you run in the cloud. Ship the same code to Azure. Change one line, the connection string, and you’re in production. Free for local dev and CI. No subscription. No credit card. No catch. Run it yourself, or hand it to an AI agent and watch it go. 

That’s pretty neat, especially because there is a surface area difference between Azure SQL Database and SQL Server.

Comments closed