Piotr Nowojski explains an important concept in streaming (and ELT/ETL) products:
The backpressure topic was tackled from different angles over the last couple of years. However, when it comes to identifying and analyzing sources of backpressure, things have changed quite a bit in the recent Flink releases (especially with new additions to metrics and the web UI in Flink 1.13). This post will try to clarify some of these changes and go into more detail about how to track down the source of backpressure, but first…
Read on for the full story, including a review of the concept and its importance.