Press "Enter" to skip to content

Day: August 19, 2021

Thoughts on Topic Replication in Kafka

Published 2021-08-19 by Kevin Feasel

Jeffrey Carter elaborates on a pair of concepts related to topic replication in Apache Kafka:

Apache Kafka has the default ability to allow a topic to be created on a broker when a message is written to it and when a topic with the name the message is attempting to be written to does not exist. This can be very helpful in early development or prototyping where code, topic names, and schemas are in flux. However, past that early stage, it is recommended that Kafka be configured to disable the auto-creation of topics from messages for a few reasons. In this article I am going to touch on two of these reasons that are also core principles of Kafka partitions and Kafka topic replication.

Read on to understand what partitions and replication factor have to do with all of this.

Comments closed

A Learning Path for Data Science with R

Published 2021-08-19 by Kevin Feasel

Holger von Jouanne-Diedrich has a greatest hits album:

Over the course of the last two and a half years, I have written over one hundred posts for my blog “Learning Machines” on the topics of data science, i.e. statistics, artificial intelligence, machine learning, and deep learning.

I use many of those in my university classes and in this post, I will give you the first part of a learning path for the knowledge that has accumulated on this blog over the years to become a well-rounded data scientist, so read on!

Read on for links to dozens of posts on interesting topics.

Comments closed

Elastic Data Maps with Azure Purview

Published 2021-08-19 by Kevin Feasel

Wolfgang Strasser has some good news for us:

It’s been a long and intensive discussion – the (initial) pricing structure of Azure Purview. As I already talked about it in my Purview pricing blog post, the basic cost calculation involves

– the cost for the data map (= the infrastructure to store metadata and provide the Purview UI + cataloging functionality)

– plus the costs involved for scanning sources.

And that has added up to a significant amount of money, especially in dev/test scenarios. But read on for the glad tidings Wolfgang has to share.

Comments closed

Ways to Filter Data in PostgreSQL

Published 2021-08-19 by Kevin Feasel

Gauri Mahajan shows off several techniques for filtering data in PostgreSQL:

Data is hosted in a variety of data repositories, one of which is relational databases. Out of tens of commercial and open-source relational databases, one of the most popular open-source relational databases is PostgreSQL. This database is offered on the Azure cloud platform through a service named Azure Database for PostgreSQL. One of the most fundamental operations performed on the database is reading and writing data to consume and host data. It goes without saying that when the data is consumed, it must be scoped based on the requirements or criteria specified by the consumer. This translates to filtering the data while querying it. Like every other relational database, Postgres offers different operators and options to filter data while querying. Let’s go ahead and learn some of the most fundamental ways to filter data hosted in PostgreSQL.

Most of them are the same as what you have in T-SQL, but not everything.

Comments closed

Eliminating False Positives when Searching for NOLOCK

Published 2021-08-19 by Kevin Feasel

Aaron Bertrand carries a lantern around Athens, searching for an honest NOLOCK:

In Part 1 of this series, I showed how to identify “NOLOCK in DML” patterns within a single statement using a Visitor pattern. In Part 2, I showed how to find those patterns in procedure and trigger bodies across multiple databases and instances. I still need to show how to eliminate false positives without tedious visual inspection.

Read on to see how we can narrow down the likelihood of a false positive when searching for the worst sorts of NOLOCK queries.

Comments closed

Power BI Push Tools

Published 2021-08-19 by Kevin Feasel

Marco Russo announces a new set of tools:

Power BI Push Tools is a set of open-source tools to work with Power BI push datasets.

Read Implementing real-time updates in Power BI using push datasets instead of DirectQuery to learn the architecture of solutions based on push datasets and how to create and populate a push dataset starting from a regular Tabular model.

In addition to reading that linked post, click through to see the three tools which are currently available.

Comments closed

Renaming Multiple Columns at Once in Power BI

Published 2021-08-19 by Kevin Feasel

Matt Allington wants to change a bunch of column names at once with Power BI:

This is not the first time I have shared this concept.  In my previous article I showed how it is possible to add a prefix to every column in a table. This article today is slightly different. Today I am removing text from multiple columns all at once using some M code. The trick you need to learn to solve this problem is “how to create a list of lists”.

Click through for a video to see it in action.

Comments closed