Press "Enter" to skip to content

Month: July 2020

Methods to Run Scheduled Tasks in Azure

Published 2020-07-15 by Kevin Feasel

Joey D’Antoni has a roundup of several techniques you can use to run scheduled tasks against an Azure SQL Database:

If you’ve worked with Microsoft SQL Server for any period of time, you are familiar with the SQL Server Agent. The Agent, which remains mostly unchanged since I started working with in 1999, is a fairly robust job scheduler that can also alert you in the event of job failures or system errors. I feel as though it’s a testament to the quality of the original architecture that the code hasn’t changed very much–it still meets the needs of about 90-95% of SQL Server workloads, based on an informal twitter discussion I had a few months ago. There are some cases where an enterprise scheduling tool is needed, but for maintaining most SQL Servers and executing basic ETL, the agent works fine. There’s one problem–the agent is only available in SQL Server and Azure SQL Managed Instance.

Read on to learn about those options.

Comments closed

Project Freta: Detecting Sensor Sabotage

Published 2020-07-14 by Kevin Feasel

Microsoft Research has a new project:

Incubated at Microsoft Research, Project Freta is a roadmap toward trusted sensing for the cloud that can allow enterprises to engage in regular, complete discovery sweeps for undetected malware. The project’s namesake, Warsaw’s Freta Street, was the birthplace of Marie Curie, a pioneer of battlefield imaging. While snapshot-based memory forensics is a field now in its second decade, no commercial cloud has yet provided customers the ability to perform full memory audits of thousands of virtual machines (VMs) without intrusive capture mechanisms and a priori forensic readiness. Just as yesteryear’s film cameras and today’s smartphones have similar megapixels but vastly different ease of use and availability, Project Freta intends to automate and democratize VM forensics to a point where every user and every enterprise can sweep volatile memory for unknown malware with the push of a button—no setup required.

There’s a prototype currently available, as well as information on where they’re taking the project. I wish them the best of luck there.

Comments closed

Understanding the Bayesian Nature of Kalman Filters

Published 2020-07-14 by Kevin Feasel

Holger von Jouanne-Diedrich gives us an interesting interpretation of Kalman filters:

The Kalman filter is a very powerful algorithm to optimally include uncertain information from a dynamically changing system to come up with the best educated guess about the current state of the system. Applications include (car) navigation and stock forecasting. If you want to understand how a Kalman filter works and build a toy example in R, read on!

The following post is based on the post “Das Kalman-Filter einfach erklärt” which is written in German and uses Matlab code (so basically two languages nobody is interested in any more 😉 ). This post is itself based on an online course “Artificial Intelligence for Robotics” by my colleague Professor Sebastian Thrun of Standford University.

In fairness, I regret only one thing about learning German: that I’ve forgotten so much over the years.

Comments closed

Showing Dates in Reverse Order on a Matrix

Published 2020-07-14 by Kevin Feasel

Gilbert Quevauvilliers solves an annoyance:

While doing some work I had a matrix that I had created. What I found was that when I had more Yr-Month values that could fit in the matrix due to having the scroll bar it would show me the older Yr-Months and not the latest ones as shown below.

I always like to think what the experience would be like for the report consumer. And if I was using this report to have to scroll each and every time to see the latest data would be a pain to say the least.

What I came up with was a relatively easy solution but made the reporting experience that much better. Where it will display the Yr-Months in descending order (The current month first)

Below are the steps I completed to achieve this.

By default I like to have dates moving from left to right, as it’s easier for a native-English speaking environment to interpret time that way. But in cases like Gilbert’s, I’d rather have glanceability, seeing the most important data without any sort of scrolling, clicking, zooming, or manual operation. The ideal might be to have the tool give us the option to start scrolled all the way to the right, but in lieu of that, Gilbert provides a nice second-best solution.

Comments closed

Azure Elastic Jobs Now GA

Published 2020-07-14 by Kevin Feasel

Niko Neugebauer gives into Azure Elastic Jobs, now generally available:

They key feature that needs to be well understood and which points to the potential of the Elastic Job Agent is that you are in no way limited by your own Azure SQL Database, nor by the logical Azure SQL Server where this database is located (contrary to the MSDB Database on the SQL Server), nor will you be limted by the Azure Region, Azure Resource Group or even Azure Subscription – you can configure the Elastic Job that will be reaching out to potentially any Azure SQL Database (given the necessary settings & permissions are correctly configured).

Read the whole thing. Niko shares some interesting thoughts on how it works, how you can tie your one server to a whole host of SQL Databases, and a wish list on what should come next.

Comments closed

Trust and Warehouse Data

Published 2020-07-14 by Kevin Feasel

Rob Farley explains one way that people might lose trust in your warehouse data:

The scenario is that there’s a source system, and there’s a table in a warehouse that is being used to report on it. Maybe it’s being populated by Integration Services or Data Factory. Maybe it’s being populated by T-SQL. I don’t really care. What I care about is whether the data in the warehouse is a true representation of what’s in the source system.

If it’s not a true representation, then we have all kinds of problems.

Mostly, that our warehouse is rubbish.

Read on for an example of how this might occur and what you can do to prevent it.

Comments closed

Locks in SQL Server

Published 2020-07-14 by Kevin Feasel

Dan Jackson provides us a primer on how SQL Server implements locking:

We have all heard of locks and can probably even name a few; shared or exclusive locks, for example. We notice them more when something goes wrong and we run into blocking or other performance problems, but what are locks and how do they work in SQL Server?

In this blog, I aim to give a basic answer to that question and provide you with an overview of the different lock modes in SQL Server and how it all works…

Let us make it clear from the off; locks are an essential part of SQL Server. In a multi-user system, there will be many users who wish to access the same resources at the same time. This means that SQL Server must have measures in place to handle concurrency and prevent adverse side effects. Locking is one of those measures.

Read the whole thing.

Comments closed