Press "Enter" to skip to content

Month: February 2023

Delta Lake Support in Azure Stream Analytics

Published 2023-02-20 by Kevin Feasel

Emma An makes an announcement:

Delta Lake has gained popularity in recent times due to its unique features and advantages over traditional data warehouse and other storage formats. For those already using traditional data storage format or moving to a lakehouse architecture, Delta Lake can offer several compelling benefits that can further enhance the performance and capabilities of their data pipelines. Many Azure services are integrated with Delta Lake, and now you can use Azure Stream Analytics to write in Delta format.

In this blog, we will explain the native support of Delta Lake in Azure Stream Analytics, that can help users take their workload to the next level, providing a seamless and scalable solution for large-scale data processing and storage. It is easy to start, taking only a few clicks to create an end-to-end pipeline, and write to either a new or existing Delta table stored in Azure Data Lake Storage Gen2.

This is a nice addition to Stream Analytics and Emma shows two ways you can write out results in Delta Lake format.

Comments closed

Tracking Azure SQL Managed Instance Logins

Published 2023-02-20 by Kevin Feasel

Michael Bourgon watches the doors:

This is a barebones Xevent (Extended Event) . I’m not using blob storage, just the existing “eh, you have a couple hundred meg worth of ring buffer you can use”. But I needed to see who was using one of our dev instances. I thought we’d moved everything over to the Azure SQL DBs, since there’s a ton of reasons to do so.

Read on for the event definition, as well as Michael’s thoughts on Azure SQL Database versus Azure SQL Managed Instance.

Comments closed

First Thoughts on Azure Hyperscale Serverless

Published 2023-02-20 by Kevin Feasel

Reitse Eskens shares some thoughts:

As some of you know, I’ve written a series of blog posts on Azure SQL Databases and there’s an accompanying session that I had the honour of presenting a number of times.
Now Azure keeps developing new offers and one of these went in public preview February 15th. An offer I hadn’t seen coming. You can read the introductory post here.

It’s the Azure Hyperscale Serverless option.

Read on for Reitse’s impressions from the preview. This wasn’t a torture test but did provide an overview of how to create and load data into the database. Reitse also calculates the cutoff point when you should switch from Serverless to traditional Hyperscale, so check that out as well.

Comments closed

ORDER BY Clause in Subqueries and SSMS Warnings

Published 2023-02-20 by Kevin Feasel

Ronen Ariely explains a warning message:

Warning: The ORDER BY clause is used only to determine the rows that are returned by the TOP clause in the view definition. The ORDER BY clause does not guarantee ordered results when the view is queried, unless ORDER BY is also specified in the query itself. Click CANCEL to discard your modifications. Click OK to save the view.

Read on for the full context of when you might see this warning message in SQL Server Management Studio.

Comments closed

Notifying when MCR Has New SQL Server Images

Published 2023-02-20 by Kevin Feasel

Andrew Pruski builds an alert:

A while back I wrote a post on how to retrieve the SQL Server images in the Microsoft Container Registry (MCR).

It’s pretty simple to check the MCR but what about automating that check to alert if there are new images present? Let’s have a look at one method of doing just that with a powershell script using the BurntToast module, and a scheduled task.

Click through for the process and keep those Docker images up to date.

Comments closed

Capabilities (and Limitations) of Power BI Migration Tools

Published 2023-02-20 by Kevin Feasel

Chris Webb talks tools:

I have so many customers wanting to migrate from legacy BI tools to Power BI. They are concerned that their current BI tool has an uncertain future. Licence renewals are looming and in the current economic climate organisations are looking to save money. Power BI is not only a lot cheaper than other BI tools, it’s a better tool overall and since Microsoft continues to make big investments in it then migration is clearly a no-brainer.

As a Power BI consultancy owner I have a problem though: I don’t have enough skilled people working for me to keep up with all this demand. What’s the answer? I know! Let’s build a tool that can help migrate all these legacy reports to Power BI!

The result is that, so far this year, I’ve seen or heard of five or six different Power BI migration tools built by various consultancies. That’s great and here at Microsoft we’re naturally supportive of our partners and want as many people to use Power BI as possible. I have reservations about some of these tools though, and these reservations fall into two categories.

Read on for Chris’s take on the topic.

Comments closed

tidyAML Now Available in CRAN

Published 2023-02-17 by Kevin Feasel

Steven Sanderson has a package make the big-time:

I’m excited to announce that the R package {tidyAML} is now officially available on CRAN! This package is designed to make it easy for users to perform automated machine learning (AutoML) using the tidymodels ecosystem. With a simple and intuitive interface, tidyAML allows users to quickly generate high-quality machine learning models without worrying about the underlying details.

Read on to learn more more about this package, as well as the broader healthyverse series of packages.

Comments closed

Building an Internal Load Balancer in Azure

Published 2023-02-17 by Kevin Feasel

Vaibhav Kumar balances the scales:

The Internal load balancer manages load for a private network with any inbound access from the public platform. As in the diagram below, the primary load balancer managing load from the internet is a public-type load balancer. But, the VMs communication to storage or database is managed through a type-internal load balancer.

Click through for a walkthrough of the process.

Comments closed

Pivoting in Postgres with CROSSTAB

Published 2023-02-17 by Kevin Feasel

Rajendra Gupta pivots abruptly:

A pivot table is a popular tool in Microsoft Excel that shows summarized data and helps you analyze it in various ways. Pivot tables collect and organize data from different rows, columns, and tables. Pivot tables are a great way to summarise data, and a handy tool for analyzing sales revenue, products sold, sales performance, etc.

Relational database tables store data in multiple rows and columns. You can calculate data using various functions such as count, sum, and average. SQL Server provides the PIVOT and UNPIVOT functions for working with pivot tables. How do we create the pivot tables in PostgreSQL? Let’s find it out.

Read on for a demonstration.

Comments closed

Recommendations for Dedicated SQL Pool Data Modeling

Published 2023-02-17 by Kevin Feasel

Bhaskar Sharma has some advice:

In this article, I will discuss how to physically model an Azure Synapse Analytics data warehouse while migrating from an existing on-premises MPP (Massive Parallel Processing) data warehouse solution like Teradata and Netezza. The approach and methodologies discussed in this article are purely based on the knowledge and insight I have gained while migrating these data warehouses to Azure Synapse dedicated SQL pool. 

Dedicated SQL pools are close enough to regular SQL Server that we make a lot of assumptions about it, some of which may be wrong.

Comments closed