Press "Enter" to skip to content

Month: April 2022

S3 and Redshift Data Movement with Role Chaining

Published 2022-04-29 by Kevin Feasel

Sudipta Mitra, et al, talk AWS security:

This post presents an approach that you can apply at scale to achieve fine-grained access controls to resources in S3 buckets and Amazon Redshift schemas for tenants, including groups of users belonging to the same business unit down to the individual user level. This solution provides tenant isolation and data security. In this approach, we use the bridge model to store data and control access for each tenant at the individual schema level in the same Amazon Redshift database. We utilize ASSUMEROLE and role chaining to provide fine-grained access control when data is being copied and unloaded between Amazon Redshift and Amazon S3, so the data flows within each tenant’s namespace. Role chaining also streamlines the new tenant onboarding process.

Read on for an overview and tutorial.

Comments closed

Quantifying Model Uncertainty with Tensorflow Probability

Published 2022-04-29 by Kevin Feasel

Vini Jaiswal reviews the Tensorflow Probability library:

In this blog, we look at the topic of uncertainty quantification for machine learning and deep learning. By no means is this a new subject, but the introduction of tools such as Tensorflow Probability and Pyro have made it easy to perform probabilistic modeling to streamline uncertainty calculations. Consider the scenario in which we predict the value of an asset like a house, based on a number of features, to drive purchasing decisions. Wouldn’t it be beneficial to know how certain we are of these predicted prices? Tensorflow Probability allows you to use the familiar Tensorflow syntax and methodology but adds the ability to work with distributions. In this introductory post, we leave the priors and the Bayesian treatment behind and opt for a simpler probabilistic treatment to illustrate the basic principles. We use the likelihood principle to illustrate how an uncertainty measure can be obtained along with predicted values by applying them to a deep learning regression problem.

Read on for an interesting explanation and tutorial.

Comments closed

Exception Handling in Scala

Published 2022-04-29 by Kevin Feasel

Pallav Gupta shows several methods for handling errors using Scala:

Error handling is the process of handling the possibility of failure. For example, failing to read a file and then continuing to use that bad input would clearly be problematic. Noticing and explicitly managing these errors saves the rest of the program from various pitfalls.

When an exception occurs, say an Arithmetic Exception then the current operation is aborted. Then the runtime system looks for an exception handler that can accept an Arithmetic Exception. Control resumes with the innermost handler. If no such handler exists, the program terminates.

Pallav starts with the most expensive option and ends with the best option with the Either monad.

Comments closed

Preventing Accidental Runs of Powershell Scripts

Published 2022-04-29 by Kevin Feasel

Jana Sattainathan has us hold our horses:

You might want to read this to the end! You are into PowerShell and have a ton of scripts. The problem is some of them run things that are not meant to be run at any time by anyone except you when you first set it up  – like dropping the entire contents of a folder and sub-folders to reinitialize.

Once you realize that you accidentally hit F5 on your “DropFolderContents.ps1” script, it is already too late. The damage is done.

Click through for several techniques to prevent this.

Comments closed

Persisting Data in Azure Redis Cache

Published 2022-04-29 by Kevin Feasel

Arun Sirpal feeds the mogwai after midnight:

I mentioned before that you could use the idea of data persistency to rebuild your data from total failure. There are two types. RDB and AOF.

RDB – persists a snapshot of your cache in a binary format. The snapshot is saved in an Azure Storage account. AOF – saves every write operation to a log. The log is saved at least once per second into an Azure Storage account. 

I’m a big proponent of using Redis as a caching service. I’m not a big proponent of using Redis as a persisted database, mostly because I’ve had a lot of bad experiences with persistent Redis…

Comments closed

Tips for Better Graphs

Published 2022-04-28 by Kevin Feasel

Cole Nussbaumer Knaflic shares a couple recommendations:

When creating a graph to explain something to someone else, I recommend that you declutter and focus attention. These concepts are not new. We have taught and written about them—directly and indirectly—many times before. I wrote about them again when drafting my new book, but then decided to take a different approach. Rather than relinquish my original words to a dismal fate in my computer’s trash bin, I thought perhaps they might still be of use here. After all, even if we’ve shared good advice before, sometimes it bears repeating.

Those are great general principles and Cole has specific examples of the principles in action.

Comments closed

Change Data Capture in Azure SQL Database

Published 2022-04-28 by Kevin Feasel

Abhiman Tiwari announces that CDC has gone GA:

CDC is now generally available on Azure SQL databases, enabling customers to track insert / update / delete data changes on their Azure SQL Database tables. On Azure SQL database, CDC offers a similar functionality to SQL Server and Azure SQL Managed Instance, providing a scheduler which automatically runs change capture and cleanup processes on the change tables. These capture and cleanup processes used to be run as SQL Server Agent jobs on SQL Server on premises and on Azure SQL Managed Instance, but now they run automatically through the scheduler in Azure SQL databases. Customers can still run scans and cleanup manually on demand.

Looks like it works pretty much the same as on-premises SQL Server, so it’s got that going for it.

Comments closed

Deploying an Azure Function via Azure DevOps

Published 2022-04-28 by Kevin Feasel

Koen Verbeeck wants to deploy a Powershell-based Azure Function:

In the blog post Azure Function with PowerShell and the Power BI REST API I explained how you could create an Azure Function using the PowerShell scripting language. This Function connected with the Power BI REST API and retrieved the last refresh status of a dataset. Developing the Function is one thing, deploying it is another. In this blog post I’ll guide you through the set-up of a build and release pipeline in Azure Devops. As a prerequisite, the Azure Function and its dependencies (for example the requirements.psd1 file) are all checked into a Git repo. As a reminder, the folder structure looks like this:

Read on for the walkthrough.

Comments closed

Binding a “Preview” Shortcut in SSMS

Published 2022-04-28 by Kevin Feasel

Daniel Hutmacher previews a table with a keyboard shortcut:

On the surface, these query shortcuts are just what the name implies – a key combination that you can press to run a command or execute a stored procedure. But there’s a hidden super power: whatever text you’ve selected in SSMS when you press the keyboard combination gets appended to the shortcut statement.

That is quite useful, though I’ve already bound all of those SSMS shortcuts to various forms of WhoIsActive.

Comments closed