In this video, I answer a viewer question about how to perform batch processing from the Azure Machine Learning Designer.
This video wraps up my work on Azure ML for now. I’m going to kick off a brand new series starting next week.
Comments closedCategory: Cloud
Drew Furgiuele concatenates a lot of files:
Lately, I’ve found myself with a few requests from friends and users that have a particular problem: they’ve got themselves a data lake in Azure, and they can read and write files just fine to it. The problem, though, is that sometimes they need to take a series of files and mash them all together, or as the cool kids call it: concatenate them. And when it comes to third party tools and methods that can do the trick, you’re spoiled for choice: Azure Data Factory, Spark via Databricks, or even PowerShell.
Case in point: I was working with someone who had tens of thousands of CSV files that they needed to merge together into one big file, but they were already out in their Azure storage account. That doesn’t sound so bad, does it?
Drew explains why it is, but also why it isn’t. So click through and check that out.
Comments closedReitse Eskens tells a tale of woe:
When I got the job to restore a bacpac file, I didn’t think much of it. Because it’s quite easy. Get the bacpac and restore it with Azure Data Studio or Sql Server Management Studio. Get coffee, done. Then I got these requirements:
- The bacpac needs to be downloaded daily from an sFTP server, without certificate
- The bacpac file is in a zip file
- The zip file is password protected
- Everything must be done serverless (Azure Automation, Logic App, Function App, Data Factory and/or Synapse Analytics)
- It’s a daily process that needs to be done without human supervision
Welcome to cloud development: Part A is easy, Part B is easy, Part C is mildly challenging, and combining A with B and C is a total nightmare because it turns out that A and B aren’t compatible, so by the end, you’re dealing with A” and b and D (because C, C’, C”, etc. wouldn’t work and c and c’ would work but had severe limitations preventing you from using it in this scenario).
Comments closedKendra Little has a note for us:
In Azure SQL Managed Instance, you get to use Resource Governor, even in the General Purpose tier. This is awesome.
Just make sure you execute commands in the context of the master database, or you’ll get error 40510:
Statement 'ALTER RESOURCE GOVERNOR' is not supported in this version of SQL Server.
Read on for more information about this error and how to circumvent it.
Comments closedMatt Eland tries out the TextAnalytics client:
We’ll talk about each one of these capabilities briefly as we cover the results, but at a high level what we want to do is:
- Perform sentiment analysis to determine if the text is positive, negative, neutral, or mixed.
- Summarize the text using abstractive summarization which summarizes the text with new text generated by a large language model (LLM).
- Summarize the text using extractive summarization which summarizes the text by extracting key sentences or parts of sentences to convey the overall meaning.
- Extract key phrases of interest from the text document.
- Perform entity recognition and linked entity recognition to determine the major objects, places, people, and concepts the document discusses.
- Recognize any personally identifiable information (PII) present in the document for potential redaction.
- Analyze the text for healthcare specific topics such as treatment plans or medications.
Read on to see how a certain passage of text fares.
Comments closedAndy Leonard’s speaking my language:
Regular readers of this blog know I am all about saving money. I’ve blogged about ways to set up inexpensive instances of Azure SQL DB. I recently learned a couple new ways to save even more money working with Azure SQL. Both methods begin the same:
Read on for a pair of tips. Serverless mode does work really well for dev environments and cases where you don’t need it up that much (and can wait the 3-5 minutes for the service to spin up the first time you use it that day). But if you’re using it 12+ hours a day, it’s actually more expensive than using the standard model.
Comments closedKendra Little gives us the lowdown:
Spoiler: a large amount of features from SQL Server 2022 are missing from Azure SQL Managed Instance. Some major features are missing that were introduced in SQL Server 2019– and here we are just a few weeks away from 2024.
But Microsoft’s top-line marketing claims about Azure SQL Managed Instance remain that “it’s always up to date with the latest SQL features and functionality.”
Let’s dig into some of the documented highlights on missing features, so you can decide for yourself what to think of that statement.
Read on for Kendra’s take. I agree with the overall argument: Managed Instance is an expensive service intended to serve as a SQL Server replacement. But there’s still a lot missing from the product and it still feels like it’s smarter simply to build a VM running SQL Server rather than play the Managed Instance game. Azure SQL DB I extend a bit more grace toward, as its intent is a bit different from the SQL Server box product or Managed Instance.
Comments closedDieu Phan takes us through the Azure Load Testing service:
Azure Load Testing is a fully managed load-testing service that enables you to generate high-scale loads. The service simulates traffic for your applications, regardless of where they’re hosted so I would like to share a walkthrough Azure Load Testing in this post.
Okay, this post isn’t very data platform-centric, but I do like the Load Testing service and think more companies and people should use it.
Comments closedIn this video, I show off how easy it is to integrate Azure ML and Power BI, at least once you get past all of the trouble trying to integrate them.
I expected this to be easy. It turns out that the “make it look easy” depends on having several things in place already and using the correct (by which I mean “old”) deployment type.
Comments closedLance Wright shows off a dashboard:
At Ignite 2023, we announced the public preview of performance dashboards for SQL Server enabled by Azure Arc. With these performance dashbaord, DBAs and IT Admins now get performance monitoring right from within Azure. No need to setup and login to another tool or remotely connect to the SQL Server to run performance queries. Let Azure Arc do the heavy lifting so you can get to your performance metrics faster.
This is another tool in the toolbox of DBAs, IT admins, and cloud specialists looking to gain better visibility into their hybrid and multi-cloud workloads. If a SQL Server enabled by Azure Arc meets the requirements for data collection (see “How to enable performance dashboards”), Azure Arc will automatically collect the following types of data from the Dynamic Management Views (DMV) datasets oof the SQL Server:
Click through to see what it includes and how to enable it.
Comments closed