Press "Enter" to skip to content

Day: December 23, 2021

Spark in Azure Databricks

Tomaz Kastrun starts winding down a series on Apache Spark. Part 22 covers Spark in Azure Databricks:

Azure Databricks is a platform build on top of Spark based analytical engine, that unifies data, data manipulation, analytics and machine learning.

Databricks uses notebooks to tackle all the tasks and is therefore made easy to collaborate. Let’s dig in and start using a Python API on top of Spark API.

Read on for that primer.

Comments closed

Azure App Service Source Code Breach

Catalin Cimpanu reports on a security problem:

Microsoft has notified earlier this month a select group of Azure customers impacted by a recently discovered bug that exposed the source code of their Azure web apps since at least September 2017.

The vulnerability was discovered by cloud security firm Wiz and reported to Microsoft in September. The issue was fixed in November, and Microsoft has spent the last few weeks investigating how many customers were impacted.

Click through for the full report, with the upshot that if you run Azure App Services on Linux, you were probably affected.

Comments closed

Azure Synapse Analytics: Success by Design

Wolfgang Strasser digs up some documents:

Today, I stumbled upon a very interesting link – the Azure Synapse Analytics – Success by Design site (follow this link).

If you need guidance, best practices links, POC playbooks, links to blogs & videos, tools, .. THIS is the site you need to bookmark.

Click through for a bit more information, as well as links to other relevant Azure Synapse Analytics resources.

Comments closed

Operation Requires Server to be a Registered Server

Garland MacNeill finds one way to solve a problem:

Anyway, it’s been a while since I worked on this AG and I need to get the migration/upgrade done. As I was working on configuring jobs, I ran into a problem where the AG node (as a target node) wasn’t downloading jobs from the Master node, in fact, the last poll was in July.

When I tried to force a poll, I was met with an error message that the server wasn’t registered, never mind it was clearly listed as a target server. Google didn’t find anything useful, other than some questions from 2013. I did come across the syntax to forcefully eject the server as a target with SQL. 

Read on to see how and what to do in the aftermath.

Comments closed

Extracting SQL Queries from SSIS Packages

Bill Fellows works from an…interesting…SSIS package:

Our job was to rewrite this into something more manageable and it appears Azure Data Factory will be the winner. Before we can do that, we need to document what the existing package is doing (the vendor has supplied the incremental load logic) so we can replicate it but in a more economical form. It appears to have the pattern (squint really hard at the picture) Execute SQL Task -> Execute SQL Task -> Sequence container => many data flows -> Data Flow -> Execute SQL Task. The Data Flow Task is named after the table being loaded. An ODBC source with a expression based query, named “ODBC Source 1” wired to an OLE DB Destination, named “OLE DB Destination”. How would you do it, especially given that there are 236 Data Flow Tasks embedded in a single container?

Click through for the answer.

Comments closed

Power BI Adding Labels for DirectQuery Operations

Chris Webb points out something of interest:

If you’re using Power BI in DirectQuery mode against a SQL Server-related data source (ie SQL Server on-prem, Azure SQL DB or Synapse) you may have noticed a new feature that was released a few weeks ago: query labels that allow you to link a SQL query to the Power BI visual that generated it.

There’s nothing you need to do to enable it – it happens automatically. Here’s an example of a TSQL query generated by a DirectQuery mode dataset with it in:

Though I’m not sure if on-prem SQL Server or even Azure SQL Database has a nice way of viewing data by label. I know that dedicated SQL pools do and they’re a rather helpful method for figuring out who (or what) is doing work.

Comments closed