Joining two topics to aggregate the data is one of the fundamental operations in stream processing. But that’s not to say that it’s simple. Let me show you what can go wrong! This article chronicles my journey to join two Apache Kafka topics—stumbling into and out of various pitfalls. I‘m going to show you…
– How to debug co-partitioning with kcat (formerly kafkacat)
– How to avoid the number one pitfall of using kcat
– Stream–table join semantics in action
There’s a lot of useful information in this post.