Press "Enter" to skip to content

Curated SQL Posts

Answering NiFi Questions

Pierre Villard has a few answers to questions about Apache NiFi:

Over the last few weeks, I delivered four live NiFi demo sessions, showing how to use NiFi connectors and processors to connect to various systems, with 1000 attendees in different geographic regions. I want to thank you all for joining and attending these events! Interactive demo sessions and live Q&A are what we all need these days when working remotely from home is now a norm.  If you have not seen my live demo session, you can catch up by watching it here

I received hundreds of questions during these events, and my colleagues and I tried to answer as many as we could. As promised, here are my answers to some of the most frequently asked questions. 

Click through for the questions and answers.

Leave a Comment

Build Your Own Number Generator

Itzik Ben-Gan has part one of solutions to a challenge:

Last month I posted a challenge to create an efficient number series generator. The responses were overwhelming. There were many brilliant ideas and suggestions, with lots of applications well beyond this particular challenge. It made me realize how great it is to be part of a community, and that amazing things can be achieved when a group of smart people joins forces. Thanks Alan Burstein, Joe Obbish, Adam Machanic, Christopher Ford, Jeff Moden, Charlie, NoamGr, Kamil Kosno, Dave Mason and John Number2 for sharing your ideas and comments.

Initially I thought of writing just one article to summarize the ideas people submitted, but there were too many. So I’ll split the coverage to several articles. This month I’ll focus primarily on Charlie’s and Alan Burstein’s suggested improvements to the two original solutions that I posted last month in the form of the inline TVFs called dbo.GetNumsItzikBatch and dbo.GetNumsItzik. I’ll name the improved versions dbo.GetNumsAlanCharlieItzikBatch and dbo.GetNumsAlanCharlieItzik, respectively.

One of the best parts about this happening in public is that there is a real benefit to having multiple people look over a problem, and then waves of people refining those solutions over time.

Leave a Comment

Quick Refresh of Excel files in Power BI

Gilbert Quevauvilliers needs things to be faster:

I was recently getting an error when refreshing an Excel file that was stored in SharePoint online.

This got me thinking what the issue is and is there a way to improve it and YES there is as detailed below.

When I investigated this further, I found that my 17kb Excel file was using at least 616kb of data when being refreshed in Power BI Desktop. Whilst this is a relatively small amount this is 36x larger than the file size. This led me to believe that it is possibly being read multiple times, but some other things might be going on!

Spoilers: there were things going on.

Leave a Comment

Creating a Hyperlink in Excel using Powershell

Mikey Bronowski continues a series on generating Excel documents with Powershell:

Last week I have mentioned the new functions waiting to be included into the module. This week I would like to write about another set of functions:

Add-ExcelHyperlink
Get-ExcelHyperlink
Remove-ExcelHyperlink

Those are fresh too and were inspired by a query I have got from Garry Bargsley (blog|twitter) who needed adding hyperlinks to the existing cells without using HYPERLINK() Excel function.

Read on to see how.

Leave a Comment

Power BI Helper Update

Reza Rad has an update to the Power BI Helper:

Happy New Year. We wish this be a year full of happiness and joy for all of you. We are glad to let you know that version 12 of Power BI Helper is now available to download with the below new features;

– Detecting the storage mode of tables (DirectQuery, Dual, Import)
– Detecting the connection mode of the file (Import, DirectQuery, Live Connection, Composite/Mixed)
– Report level measures
– Compare two files based on their report level measures
– Analyzing the visualization just by selecting the model
– Feedback form
– Documentation of all the above new information
– bug fixes

Click through for the full changelog.

Leave a Comment

The Editions of Powershell

Jeffrey Hicks gives us an update on the Powershell landscape:

The PowerShell community is beginning another year in the world of PowerShell 7. Most of you know what that means. However, there are newcomers to our community practically every day. Or I know there are occasional or reluctant users who might not pay enough attention to understand the world of PowerShell as it stands today. I wrote this post as a kind of virtual sticky note for the PowerShell community. Feel free to reference this post in your own work so that you don’t have to explain or define “Windows PowerShell” and “PowerShell”.

Click through to learn how to differentiate the two.

Leave a Comment

Countdown Number Puzzle

Tomaz Kastrun has a fun puzzle for us:

So the game is (was) known as a TV show where then host would give a random 3-digit number and the contestants would draw 6 random numbers from stack of numbers. Given the time limit, the winner was the one who would create a formula matching the result or being closest.

Many ways, tips, tricks and optimisations were already considered, maybe the most famous was the Reverse Polish notation where operators follow their operands and is a great fit for the game.

With useless functionality, I have decided to use permuteGeneral function from RcppAlgos or same functionality could be achieved with combn function.

Click through to see it in action.

Leave a Comment

Using Color Intentionally in Visuals

Elizabeth Ricks continues a series:

To create more impactful explanatory visuals, develop the habit of using color in a purposeful way. When applied thoughtfully, color is arguably the most important decision you make when focusing your audience’s attention where you want it. 

One way to start is by overriding the default color palette of your data viz tool and start with nothing emphasized.

Using color for emphasis is important. Color is a pre-attentive attribute, so it’s great to take advantage of that powerful implicit signal. And that means not using lots of different colors.

Leave a Comment

Soft Deletes in SQL Server

Erik Darling has some thoughts on soft deletes:

Implementing soft deletes for an app that’s been around for a while can be tough. In the same way as implementing Partitioning can be tough to add in later to get data management value from (rebuilding clustered indexes on the scheme, making sure all nonclustered indexes are aligned, and all future indexes are too, and making sure you have sufficient partitions at the beginning and end for data movement).

Read the whole thing. Incidentally, this also ties well into a recent post by Erik about deleting into a different table. It can be easier to implement soft deletes as deleting from the current table and adding to an archive table. That gives you the benefits of keeping deleted data while not running into some of the problems Erik mentions. And if you want to undo a deletion? Delete from the archive table and insert back into the main table.

Leave a Comment