Press "Enter" to skip to content

Month: February 2022

De-Scalafication in Flink

Published 2022-02-23 by Kevin Feasel

Seth Wiesman has a post leaving me feeling a little bittersweet:

If you have worked with a JVM-based application, you have probably heard the term classpath. The classpath defines where the JVM will search for a given classfile when it needs to be loaded. There may only be one instance of a classfile on each classpath, forcing any dependency Flink exposes onto users. That is why the Flink community works hard to keep our classpath “clean” – or free of unnecessary dependencies. We achieve this through a combination of shaded dependencies, child first class loading, and a plugins abstraction for optional components.

The Apache Flink runtime is primarily written in Java but contains critical components that forced Scala on the default classpath. And because Scala does not maintain binary compatibility across minor releases, this historically required cross-building components for all versions of Scala. But due to many reasons – breaking changes in the compiler, a new standard library, and a reworked macro system – this was easier said than done.

They did it, which means less Scala in the code base. But it also means that you aren’t tied to a particular version of Scala in your own code. I’m happy about it on the whole but it does expose a frustrating pain point with Scala.

Comments closed

Using KQL to Extract JSON Data in Power BI

Published 2022-02-23 by Kevin Feasel

Dany Hoter shreds some JSON:

In Kusto (aka Azure Data Explorer aka ADX) you can have columns in a table that contain JSON structures.

In KQL it is very easy to extract elements from these columns and use them as regular columns.

It requires more resources but overall, it is standard.

An example can be found in the table TransformedMetrics in the SampleMetrics databases in the help cluster.

Click through for that process.

Comments closed

Database Restores and Filestream Woes

Published 2022-02-23 by Kevin Feasel

Joey D’Antoni troubleshoots a problem:

We had a case over the weekend where our automated restore process at client got hung up on this wait type, for a single database. What was the unique characteristic about this relatively medium (2-300 GB) database? It had a lot of filestream data–it seemed like the file count wasn’t that high, but my guess is the filestream data was the majority of the data in that database. When the job hung up, the restore had been waiting on PREEMPTIVE_OS_FILEOPS for over a day and still had a null value for percentage complete.

Click through to see what the problem was.

Comments closed

Writing Extended Events to CSV via XESmartTarget

Published 2022-02-23 by Kevin Feasel

Gianluca Sartori shows one valuable use for XESmartTarget:

In this case, imagine that you wanted to observe the commands executed on a SQL Server instance and save them to a file to process them later. Of course, Extended Events can do that with the built-in targets. However, when you write to a file target, the file has to reside on the disks of the SQL Server machine (well, actually, the file could be sitting on a file share writable by SQL Server or even on BLOB storage on Azure, but let’s keep it simple). How do you use the storage of the client machine instead of using the precious filesystem of the server machine?

Here is where XESmartTarget can help you with a CsvAppenderResponse. This Response type writes all the events it receives to a CSV file, that can be saved on the client machine, without wasting disk space on the server. You can decide which events to process and which columns to include in the CSV file, but more on that later.

Read on to see how the whole thing works.

Comments closed

TRANSLATE() and REPLACE() in SQL Server

Published 2022-02-23 by Kevin Feasel

Kevin Wilkie compares a couple of functions:

There is another function within SQL Server that many people think does the same thing, but with a slight nuance.

Sometimes, you just need to change out one character with another. For example, you need to make a “(” into a “[” to make everything consistent.

I’ve probably used REPLACE() two (or three?) orders of magnitude more often than TRANSLATE() but that’s mostly because I keep forgetting what the latter does.

Comments closed

Mission-Critical Azure Architectures

Published 2022-02-23 by Kevin Feasel

Ben Brauer has some reference guidelines:

The AlwaysOn project strives to address the challenges of building mission-critical applications by providing organizations with a prescriptive architectural approach for the Microsoft Cloud.

It leverages lessons from numerous customer applications and first-party solutions, and applies Well-Architected best practices to provide actionable and authoritative guidance for building and operating a highly reliable solutions on Azure at-scale.

I guess marketing is calling it “AlwaysOn” again. Until they call it Always On. Or AlWaYsOn.

Comments closed

Variable Typing in Python

Published 2022-02-22 by Kevin Feasel

Adrian Tam notes that Python is for the birds:

Python is a duck typing language. It means the data types of variables can change as long as the syntax is compatible. Python is also a dynamic programming language. Meaning we can change the program while it runs, including defining new functions and the scope of name resolution. Not only these give us a new paradigm in writing Python code, but also a new set of tools for debugging. In the following, we will see what we can do in Python that cannot be done in many other languages. After finishing this tutorial you will know

– How Python manages the variables you defined

– How Python code uses a variable and why we don’t need to define its type like C or Java

Read on to learn how.

Comments closed