Press "Enter" to skip to content

Day: August 16, 2019

Kafka 2.3 and Kafka Connect Improvements

Published 2019-08-16 by Kevin Feasel

Robin Moffatt goes over improvements in Kafka Connect with the release of Apache Kafka 2.3:

A Kafka Connect cluster is made up of one or more worker processes, and the cluster distributes the work of connectors as tasks. When a connector or worker is added or removed, Kafka Connect will attempt to rebalance these tasks. Before version 2.3 of Kafka, the cluster stopped all tasks, recomputed where to run all tasks, and then started everything again. Each rebalance halted all ingest and egress work for usually short periods of time, but also sometimes for a not insignificant duration of time.

Now with KIP-415, Apache Kafka 2.3 instead uses incremental cooperative rebalancing, which rebalances only those tasks that need to be started, stopped, or moved. For more details, there are available resources that you can read, listen, and watch, or you can hear the lead engineer on the work, Konstantine Karantasis, talk about it in person at the upcoming Kafka Summit.

Looks like some nice improvements here.

Comments closed

Python versus R (Again)

Published 2019-08-16 by Kevin Feasel

Alex Woodie looks at whether Python is dominating R in the data science space:

There is some evidence that Python’s popularity is hurting R usage. According to the TIOBE Index, Python is currently the third most popular language in the world, behind perennial heavyweights Java and C. From August 2018 to August 2019, Python usage surged by more than 3% to achieve a 10% rating (TIOBE’s proprietary metric that primarily measures search activity), easily the biggest gain among the 20 most popular languages.

R, by contrast, has not fared well lately on the TIOBE Index, where it dropped from 8th place in January 2018 to become the 20th most popular language today, behind Perl, Swift, and Go. At its peak in January 2018, R had a popularity rating of about 2.6%. But today it’s down to 0.8%, according to the TIOBE index.

I’ll say that rumors of R’s demise are premature.

Comments closed

Installing Microsoft Master Data Services

Published 2019-08-16 by Kevin Feasel

Garry Bargsley shows how you can install Master Data Services on SQL Server:

MDS Installation pre-requisites:
The first step is to add the IIS feature to the server where MDS is going to be installed

Follow these steps for more information
https://docs.microsoft.com/en-us/sql/master-data-services/master-data-services-installation-and-configuration?view=sql-server-2017#InstallIIS

Read on for full instructions.

Comments closed

The Importance of Interaction in Power BI

Published 2019-08-16 by Kevin Feasel

Marc Lelijveld continues a series on storytelling with Power BI:

Many times, I see reports with loads of visuals on the pages. This results in both a really poor performance, as well as the end user has no clue what the key message is of this report. You can always ask yourself, is this visual necessary to show on this page? What does it add to this page? Is this really needed? If not, remove it! If the visual does add some value, is it needed on this page? Maybe it is only distracting the user of where the report is about.
A good approach can be to put certain visuals on a different page or hide them by default until the user interacts with the report. Within the interaction, you will have multiple options in Power BI to interact with your user.

There’s a lot more to it, so read on.

Comments closed

Automate VM Shutdown

Published 2019-08-16 by Kevin Feasel

Meagan Longoria has a script to shut off an Azure VM when a SQL Agent job finishes:

The runbook sets the Azure context to the appropriate subscription (especially important when you are a guest user in someone else’s tenant). Then it checks if the VM is started. If it is, it goes into a do-while loop. This task isn’t super time sensitive (it’s just to save money when the VM isn’t in use), so it’s waiting 60 seconds and then calling the child runbook to find out if my SQL Agent job is running. This makes sure that the child runbook is called at least once. If the result is that the job is not running, it stops the VM. If the job is running, the loop starts over, waiting 60 seconds before checking again. This loop is essentially polling the job status until it sees that the job is completed.

Click through for the script.

Comments closed