Press "Enter" to skip to content

Category: Cloud

An Intro to Azure Machine Learning

Published 2022-12-02 by Kevin Feasel

Tomaz Kastrun has a new Advent challenge:

Azure Machine Learning (or Azure Machine Learning Service and abbreviation AML) is Azure’s cloud service for creating, managing and productionalising machine learning projects. It is a collaborative tool for Data Scientists, Machine Learning Engineers, and data engineers, covering their daily and operational tasks. From creating and training to deploying and managing predictive models and machine learning solutions.

Click through for the introduction.

Leave a Comment

Controlling Cosmos DB Time to Live

Published 2022-12-01 by Kevin Feasel

Rahul Mehta pulls out the stopwatch:

As Microsoft states, Azure Cosmos DB “is a fully managed NoSQL database service for building scalable, high-performance applications”. Cosmos DB is widely used for storing NoSQL data with options to create using different Core (SQL), MongoDB, Cassandra, Table, and using gremlin.

With wide usage, the content storage also increases, sometimes even in Gigabytes a day. With such content storage, retention and archival of data are one of the common ask from the customer. Today, we are going to talk about how to retain data and remove unnecessary data periodically from Azure Cosmos DB. Before we do that, we need to understand a storage concept called “Container”

Read on to learn about containers, as well as the built-in way to garbage collect data.

Leave a Comment

Migrating Azure Analysis Services to Power BI Premium

Published 2022-12-01 by Kevin Feasel

Gilbert Quevauvilliers dumps AAS:

I thought it would be a good idea to walk through the steps when looking to migrate AAS to PBI.

In the past when I had to do this for clients it was a lot of manual steps and a lot of small things to get just right. This process is now seamless and awesome!

Reviewing Gilbert’s step-by-step process, yeah, this is easy, though watch out for the pitfalls Gilbert found.

Leave a Comment

Redshift Query Editor v2

Published 2022-11-30 by Kevin Feasel

Anusha Challa, et al, announce a new version of a Redshift query editor:

Amazon Redshift is a fast, fully managed, petabyte-scale cloud data warehouse. You have the flexibility to choose from provisioned and serverless compute modes. You can start loading and querying large datasets conveniently in Amazon Redshift using Amazon Redshift Query Editor v2, a web-based SQL client application.

It’s worth a try if you’re a Redshift user, though I’d imagine that frequent Redshift users have already sorted out their IDEs of choice.

Leave a Comment

Unity Catalog in Azure Databricks

Published 2022-11-29 by Kevin Feasel

Meagan Longoria makes a recommendation:

Unity Catalog in Databricks provides a single place to create and manage data access policies that apply across all workspaces and users in an organization. It also provides a simple data catalog for users to explore. So when a client wanted to create a place for statisticians and data scientists to explore the data in their data lake using a web interface, I suggested we use Databricks with Unity Catalog.

Read on to learn more about what the Unity Catalog does.

Leave a Comment

Reading Serverless SQL Pool Data with Data Factory

Published 2022-11-28 by Kevin Feasel

Koen Verbeeck wants to read from the serverless SQL pool in Azure Synapse Analytics:

We have some data we can query using the serverless SQL pools in Azure Synapse Analytics. For this blog post, I’m querying data that is stored in Azure Cosmos DB. Read the blog post How to Store Normalized SQL Server Data into Azure Cosmos DB to learn more about how that data got there.

Suppose I now want to read the data using Azure Data Factory. You can read data from Cosmos DB directly, but let’s pretend I want to do some transformations first using my favorite language: SQL. How can we do this?

Read on to learn how.

Leave a Comment

Loading Normalized Data into Cosmos DB

Published 2022-11-22 by Kevin Feasel

Koen Verbeeck does a bit of shuffling:

I loaded the data into a table in Azure SQL DB. For demo purposes, I want to transfer this data from a SQL table to a container in Azure Cosmos DB (with the NoSQL API). There are plenty of resources on the web on how to transfer a simple relational table to Cosmos DB, but I have some additional complexity. One column – flavor profiles – contains a list of flavors that is assigned to a beer.

Click through for one way to organize the data when dealing with arrays.

Leave a Comment

Invoke External REST Endpoints from Azure SQL DB

Published 2022-11-17 by Kevin Feasel

Rob Farley is impressed:

This internal procedure is new in Azure SQL DB in 2022. I think it presents a significant change to the way we do things in the world of SQL, and makes some other tools a whole lot more useful as well.

sp_invoke_external_rest_endpoint lets me send data to a REST API from within a stored procedure. Invoking an HTTP REST endpoint – as simple as that. And while I know you’re probably thinking, “But I can send data to a REST API from anywhere – why do I need to do it from within a stored procedure?”, I want to describe a few scenarios to you.

I like having the functionality, though would want to control how frequently my teams would use it. The reason is that this potentially makes your database the a domain boundary (when thinking in domain-driven design concepts).

Leave a Comment

Continuous Backup for Cosmos DB

Published 2022-11-17 by Kevin Feasel

Manvendra Singh wants a backup:

This article will explore Continuous backup and steps to configure it for a new Azure Cosmos DB account or an existing Cosmos DB account. Azure Cosmos DB is a fully managed and highly secure, NoSQL database service on the Azure cloud that is designed for modern-day application development. It automatically runs backup for its databases on separate Azure blob storage at regular intervals without affecting the performance, availability, and provisioned resource units (RUs) to ensure data protection from a data recovery standpoint which can be needed in case of data corruption, deletion, or wrongly data updates.

Click through for the process and some limitations.

Leave a Comment