Press "Enter" to skip to content

Day: August 8, 2023

Recommendations for Snapshotting HDFS

Published 2023-08-08 by Kevin Feasel

Tsz Sze provides some recommendations:

The snapshots feature of the Apache Hadoop Distributed Filesystem (HDFS) enables you to capture point-in-time copies of the file system and protect your important data against corruption, user-, or application errors.  This feature is available in all versions of Cloudera Data Platform (CDP), Cloudera Distribution for Hadoop (CDH) and Hortonworks Data Platform (HDP). Regardless of whether you’ve been using snapshots for a while or contemplating their use, this blog gives you the insights and techniques to make them look their best.  

Read on for those tips and tricks.

Comments closed

Key Metrics for Kafka Clusters

Published 2023-08-08 by Kevin Feasel

Paul Brebner tracks some measurements:

Like a steam locomotive, Apache Kafka also has lots of “moving parts”. A Kafka cluster is actually a distributed system, a cluster, which consists of multiple brokers (servers)—messages are sent to Kafka with producers, and consumers receive messages. Topics are used to direct messages between producers and consumers—producers write to selected topics, and consumers subscribe and read from selected topics. Topics are divided up into partitions, and the partitions are distributed over the available brokers for high availability and concurrency. Partitions are replicated to other partitions (followers) from the leader partition (3 is common for production clusters).  

Click through for the description of metrics on brokers, topics, consumer groups, and more, as well as an extended analogy to steam locomotives which works surprisingly well.

Comments closed

Row Rank Min in KQL

Published 2023-08-08 by Kevin Feasel

Robert Cain continues a series on KQL:

In this post we’ll cover the next in our series on KQL Windowing Functions, row_rank_min. The row_rank_min function will assign rank numbers to an incoming dataset, with the rank of 1 being assigned to the minimal value in the dataset and increasing as the values increase.

First though, if you’ve not read the introductory post on Windowing Functions, Fun With KQL Windowing Functions – Serialize and Row_Number, you should do so now as it introduced several important concepts needed to understand how these Windowing Functions work.

Read on for Robert’s explanation.

Comments closed

Azure Data Studio 1.45 Released

Published 2023-08-08 by Kevin Feasel

Erin Stellato gives us an update:

In this release we updated to VS Code version 1.79.2, from version 1.70, bringing ADS nearly current with VS Code.  Keeping up with VS Code is an artifact of Azure Data Studio being a fork of VS Code, and the team accomplished a major feat with this latest merge.  Many of the changes are not immediately obvious to users, but there are improvements across security, performance, and the application interface.

Read on for the highlights.

Comments closed

Execution Plans in Postgres

Published 2023-08-08 by Kevin Feasel

Henrietta Dombrovskaya shows us an execution plan in Postgres:

In the last blog (When PostgreSQL Parameter Tuning is not the Answer), we compared several execution plans for a SQL statement as we made changes to parameters and indexes. Still, there was no mention of what an execution plan is, how one can obtain an execution plan for a query, and how to interpret the result. In this blog, we will take a deep dive into this topic.

Read on for the full article.

Comments closed